Our 2011.2 release, codenamed "Cactus", was finally released early on
April 15. The Diablo merge window opened a few hours later, and in 8
days the developer community will gather in Santa Clara to discuss what
will be in it.
If you want to develop a feature for Diablo and get your design reviewed
or discussed at the summit, remember that you have until the end of
Tuesday, April 19 to submit a design summit session. The procedure to
follow is simple, please see http://wiki.openstack.org/Summit for more
details.
Your session blueprint will be reviewed as soon as possible by the track
leads. Given the very large number of sessions submitted, we might try
to group together related blueprints for a common discussion, or have to
refuse a few sessions. The April 19 is a soft deadline: you can still
submit sessions after that date, but there is no guarantee we will
review it, and they will be given less priority that the ones submitted
early.
See you all there !
Our time-based release cycles
are cadenced by a number of freezes and milestones, and we just
passed BranchMergeProposalFreeze
for the Cactus
cycle. Feature
branches should have been proposed by now, so let's see how well we did
compared to the original plan
we had for this release:
- Essential specs: 3 merged, 2 proposed (100% proposed in time)
- High-prio specs: 5 merged, 3 proposed, 3 deferred (73%)
- Medium-prio specs: 11 merged, 5 proposed, 4 late, 2 deferred (73%)
- Low-prio specs: 7 merged, 1 proposed, 6 late, 2 deferred (50%)
Compared to the Bexar
cycle,
the results are comparable (with less success on High specs, better
success on Medium specs). With 54 specs targeted (compared to 42 in
Bexar), that's a great achievement, so congratulations to all the
developers !
Now it's time to concentrate on review and get those proposed branches
merged before FeatureFreeze,
which happens at end of day March 24.
In a bit more than a week, we will hit
FeatureFreeze for OpenStack
"Cactus" cycle, so we start to have a good idea of what new features
will make it. The Cactus cycle focus was on stability, so there are
fewer new features compared to Bexar, but the developers still achieved
a lot in a couple of months...
Swift (OpenStack object storage)
The Swift team really focused and stability and performance improvements
this cycle. I will just single out the refactoring of the proxy to
make backend requests
concurrent,
and improvements on sqlite3
indexing
as good examples of this effort.
Glance (OpenStack image registry and delivery service)
Bexar saw the first release of Glance, and in Cactus it was vastly
improved to match standards we have for the rest of OpenStack:
logging,
configuration
and
options
parsing, use of
paste.deploy
and non-static
versioning,
database
migrations...
New features include a CLI
tool and a new
method for client to verify
images.
Glance developers might also sneak in an authentication
middleware
and support for HTTPS
connections
!
Nova (OpenStack compute)
A lot of the feature work in Nova for Cactus revolved around the
OpenStack API
1.1 and
exposing features through XenServer
(migration,
resize, rescue
mode,
IPv6,
file and
network
injection...). We should also have the long-awaited live
migration
feature (for KVM), support for LXC
containers,
VHD
images,
multipleNICs,
dynamically-configured instance
flavors
or volume storage on HP/Lefthand
SANs.
XenAPI should get support for Vlan network
manager
and network
injection.
We hope support for VMWare/vSphere
hypervisor
will make it.
The rest of the Nova team concentrated on testing, bugfixing (already
115 bugfixes committed to Cactus !) and producing a coherent release, as
evidenced by the work on adding the missing Ipv6 support for
FlatManager
network model. I should also mention that the groundwork for
multi-tenant
accounting
and multiple clusters in a
region
also landed in Cactus.
Over the three projects branches, last month we had more than 2500
commits by more than 75 developers. Not too bad for a project less than
one-year-old... We'll see the result of this work on Cactus release day,
scheduled April 14.
I'd like to quickly precise what will be expected, from a release
management perspective, from the Project Technical Leads (PTLs) some of
us will nominate and elect in the following weeks.
PTLs, like it says on the tin, will have to technically lead each
project. That comes with a few rights, but also with a lot of duties
that the candidates shouldn't underestimate...
Rights include the ability to decide between conflicting opinions on a
technical debate, or the authority to set the theme for the next
release. Exercising those rights will only be possible if the technical
opinions of the elected lead are widely respected in the project.
Duties of the PTLs, from a release management perspective, mainly
include coming up with a plan for the next release(s). That means
staying on top of what everyone proposes, selecting (and rejecting)
blueprints for a given cycle, setting priorities, approving designs
(potentially with the help of other project drivers), or making sure we
don't duplicate work. The PTLs should also be very impacted by the
design summit preparation, making sure we have sessions for what we need
to discuss, and encouraging people to submit corresponding blueprints.
As release manager, I can help with the process, but the decisions must
come from the PTLs, who have the legitimacy of being elected. During the
cycle, I will then help in making sure the defined plan is on track.
With a well-established project like Swift or a relatively-small project
like Glance, the PTL work can certainly be done at the same time as
regular development. For Nova however, the PTL should expect project
coordination work to take a large part of his time, so he could find
himself not being able to write as much code as he would want. That
should be kept in mind before you accept nominations :)
Hoping this helps in clarifying expectations... Happy nominating and
voting !
You can globally split open source projects into two broad categories.
Upstream projects develop and publish source code for various
applications and features. Downstream projects are consumers of this
source code. The most common type of downstream projects are
distributions, which release ready-to-use binary packages of these
upstream applications, make sure they integrate well with the rest of
the system, and release security and bugfix updates according to their
maintenance policies.
The relationship between upstream projects and distributions is always a
bit difficult, because their roles overlap a bit. Since I'm sitting on
both sides of the fence, let's try to find common ground.
Overlapping roles
In an ideal world, everyone would install software through distribution
packages, and the roles wouldn't overlap. In the real world though,
upstream projects need to deal with distributions that don't provide
packages for your software, or provide old buggy versions with no
mechanism for getting fresh ones. That's why they need to care about
manual installation or update mechanisms. On the other hand, in their
rush to release fixes, distributions sometimes carry patches without
sending them upstream immediately. Both want to provide bugfix updates
to stable versions. In all cases the overlapping roles end up
duplicating work and creating unnecessary friction.
Splitting the roles
In my (humble) opinion, upstream projects should encourage the use of
packaged software wherever possible, rather than resisting it. They
should concentrate on their core competency: working on producing new
releases of their code. Dealing with distribution issues, environment
specificities or maintaining stable branches is a different type of
work, and one that distributions excel in. So the key seems to be in
splitting the roles more cleanly.
Upstream projects should release code, together with good documentation
on how to manually deploy it: dependencies, startup and upgrade
mechanisms, open bug trackers with links to patches... This
documentation can be reused by manual deployers and distribution
packagers alike. They should stop short of providing installers,
auto-updaters, dependency bundles, etc. They should limit the release of
point release updates only to critical issues (data loss, security...).
Distributions should be responsible for proper packaging (easy way to
install the software and its dependencies, together with startup scripts
and other system integration), and would be responsible for more general
bugfix updates that match their maintenance policy.
With such a split, you obviously will end up with a subpar user
experience if you try to manually install the software from the released
code. But you facilitate packaging, so you should end up being packaged
in more distributions. I think time is better spent contacting
distributions to get packaged rather than trying to improve the manual
installation to the point where it is actually usable.
Freshness
One case where you end up doing manual installations (even on supported
distributions) is to get the latest released code running on
already-released distributions. Due to stable release policies in
distributions, they will release bugfix updates for the version that was
available when they released, but usually won't provide a whole new
version of a package.
The solution is in specific distribution archives that track the latest
upstream releases (like PPAs in Ubuntu) and make them available for
users of already-released distributions. Those are usually
co-maintained between distributions and upstream projects.
Reference distributions
At this point, it is worth taking collaboration one step further, and
have developers that are involved in both projects ! Those can make sure
the distribution includes the packages and patches you need for your
software to run properly. Those can make sure the distribution is one on
which your software is up-to-date, runs properly and gets appropriate
bugfix updates. Those can maintain the specific distribution archives
for the latest upstream releases.
That distribution can then become a reference distribution for the
upstream project, one that is tightly integrated with the upstream
project and lives in harmony.
Two closing remarks:
- You can have multiple reference distributions. That said, one way to
limit friction and increase freshness is to have
somewhat-synchronized release cycles, which may not scale very well.
- I realize the proposed role split and reference distro scheme might
not be generally applicable to all open source upstream projects. In
my experience it worked well with server software.
In OpenStack, having a few Ubuntu core developers in the project (and
the Ubuntu server team supporting us) allows us to use Ubuntu as a
reference distribution. We have packages up for other distributions, but
those are not (yet) official distribution packages. Any other distro
developers interested to join ?
