There was a bit of
discussion
lately on feature-based vs. time-based release schedules in OpenStack,
and here are my thoughts on it. In a feature-based release cycle, you
release when a given set of features is implemented, while in a
time-based release cycle, you release at a predetermined date, with
whatever is ready at the time.
Release early, release often
One the basic principles in open source (and agile) is to release early
and release often. This allows fast iterations, which avoid the classic
drawbacks of waterfall
development. If you push
that logic to the extreme, you can release at every commit: that is what
continuous deployment is about. Continuous deployment is great for web
services, where there is only one deployment of the software and it runs
the latest version.
OpenStack projects actually provide builds (and packages) for every
commit made to development trunk, but we don't call them releases. For
software that has multiple deployers, having "releases" that combine a
reasonable amount of new features and bugfixes is more appropriate.
Hence the temptation of doing feature-based releases: release often,
whenever the next significant feature is ready.
Frequent feature-based releases
The main argument of supporters of frequent feature-based releases is
that time-based cycles are too long, so they delay the time it takes for
a given feature to be available to the public. But time-based isn't
about "a long time". It's about "a predetermined amount of time". You
can make that "predetermined amount of time" as small as needed...
Supporters of feature-based releases say that time-based releases are
good for distributions, since those have limited bearing on the release
cycles of their individual subcomponents. I'd argue that time-based
releases are always better, for anyone that wants to do open development
in a community.
Time-based releases as a community enabler
If you work with a developer community rather than with a single-company
development group, the project doesn't have full control over its
developers, but just limited influence. Doing feature-based releases is
therefore risky, since you have no idea how long it will take to have a
given feature implemented. It's better to have frequent time-based
releases (or milestones), that regularly delivers to a wider audience
what happens to be implemented at a given, predetermined date.
If you work with an open source community rather than with a
single-company product team, you want to help the different separate
stakeholders to synchronize. Pre-announced release dates allow everyone
(developers, testers, distributions, users, marketers, press...) to be
on the same line, following the same cadence, responding to the same
rhythm. It might be convenient for developers to release "whenever it
makes sense", but the wider community benefits from having predictable
release dates.
It's no wonder that most large open source development communities
switched from feature-based releases to time-based releases: it's about
the only way to "release early, release often" with a large community.
And since we want the OpenStack community to be as open and as large as
possible, we should definitely continue to do time-based releases, and
to announce the schedule as early as we can.
Back at the OpenStack Design Summit in Santa Clara, we decided to switch
from a 3-month cycle to a 6-month coordinated release cycle, with more
frequent milestones delivery in the middle.
Lately we have been busy adapting the release processes to match the
delivery of the first milestones. Swift 1.4.0 was released last Tuesday,
and today sees the release of the diablo-1 milestone for Nova and
Glance.
What should you expect from diablo-1, just 4 weeks after the design
summit ? In this short timeframe lots of features have been worked on,
and the developers managed to land quite a few of them in time for
diablo-1.
Glance's API was improved to support filtering of /images and
/images/detail
results
and limiting and paging of
results.
This made support of API
versioning
necessary. It also grew a new disk
format ("iso")
that should ultimately allow to boot ISOs directly in Nova.
On Nova's side, the most notable addition is support for
snapshotting
and cloning
volumes with the EC2 API. The XenServer plugin now supports Open
vSwitch, and pause
and suspend capabilities have been added to the KVM
hypervisor.
Now keep your seatbelt fastened, because diablo-2 is set to release on
June 30th.
A few weeks after the OpenStack Design Summit in Santa Clara, we are
starting to get a better picture of what should
be in the next version of
OpenStack Nova, codenamed Diablo, scheduled for
release on September
22.
One big priority of this release is to separate
code
for the network and volume services, and refactor nova-network
code
to add a clear internal
API.
This will allow to plug separate
network
and
volume
service providers, and pave the way for integration with future
OpenStack projects like
Quantum/Melange/Donabe and
LunR. In preparation for this, we'll push
changes to rely more on the
queue (and
less on the database) to pass information between components. In the
same area, we need some more changes to support multiple
NICs and
should also provide a client OpenStack API for directly interacting
with
volumes.
A second theme of the Diablo release is the new distributed
scheduler,
which should be able to schedule across zones and taking capabilities
into account. This will need changes in the way we reference
instances,
as well as some changes for EC2 API
compatibility.
On the API side, we should finalize OpenStack API
1.1
support, including work on floating
IPs
and shared IP
groups.
For administrators, instance
migration
and account administration
actions
should be added. We'll also ensure good AWS API
compatibility
and
validation.
Support for
snapshotting,
cloning and
booting
from volumes should land early in this cycle, as well as
new
ways
of communicating configuration data between host and guest. We also need
to
integrate
with
AuthN/AuthZ with the new common Keystone authentication system. Lots of
other features are planned (and others might be added before the end),
you can check out the blueprints
plan for more detail.
Last but not least, on the QA side, we should have continuous automated
testing
across a range of reference
architectures
and increase our unittest and smoketest
coverage
among
other
efforts
to
build-in quality.
The first milestone for this cycle,
diablo-1, should be
released on June 2nd.
Last week I attended the Ubuntu Developer Summit for
Oneiric in Budapest. This was the first time I
attended UDS as an upstream representative rather than as a Canonical
employee. I very much enjoyed it: not being a track lead or a busy
technical lead actually gives you desirable flexibility in your agenda
:)
First of all, a quick comment on the big announcement of the week, which
the Twittersphere is not done retweeting yet: "Ubuntu switching from
Eucalyptus to OpenStack". I think it would be more accurate to say that
Ubuntu chose to use OpenStack as its default cloud stack for future
versions. Comparing Eucalyptus and OpenStack is like comparing apples to
apple trees: OpenStack provides several cloud infrastructure pieces (of
which only OpenStack Compute -Nova- covers the same space as
Eucalyptus). I suspect the wide scope of the project played a role in
OpenStack being selected as the default stack for the future. Eucalyptus
and OpenStack Nova should both be present as deployment options from
11.10 on.
On the UDS format itself, I'd say that the "one blueprint = one hour"
format does not scale that well. The numbers of hours in the week is
fixed, so when the project grows you end up having too many sessions
going on at the same time. Lots of blueprints do not require one hour of
discussion, but rather a quick presentation of plan, feedback from
interested parties and Q&A. That's what we do for our own Design
Summits, but I'd admit it makes scheduling a bit more complex. On the
good side, having the floor plan inside our UDS badges was a really good
idea, especially with confusing room names :)
The Launchpad and bzr guys were very present during the week, attentive
and reactive to the wishes of upstream projects. They have great
improvements and features coming up, including finer-grained
bugmail
and dramatic speed
improvements
in bzr handling of large repos.
Last week also saw the rise of creampiesourcing: motivation of groups of
developers over bets ("if the number of critical bugs for Launchpad goes
to 0 by June 27, I'll take a
creampiein
the face"). Seems to work better than karma points.
Finally, Rackspace Hosting was co-sponsoring the "meet and greet" event
on the Monday night, promoting OpenStack. I think offering cool
T-shirts, like we did at the previous UDS in Orlando, was more efficient
in spreading the word and making the project "visible" over time: in
Budapest you could see a lot of people wearing the OpenStack T-shirts we
offered back then !
Just a few days from now, lots of OpenStack community members from
around the world will gather in (hopefully sunny) Santa Clara for the
OpenStack Conference and Design Summit. As I explained
here,
those are two co-hosted events, and here are a few precisions on the
Design Summit part.
The Design summit starts on Wednesday at 9am and ends on Friday at 5pm.
Apart from the 25-minute opening plenary, we have three types of
sessions: design discussions, unconference sessions and lightning talks.
Design sessions are the meat of the Design Summit. Those 25-minute or
55-minute sessions were selected from proposals coming from developers
working on a feature for future releases of OpenStack. They are
organized in 8 different tracks:
- OpenStack infrastructure: discussions affecting all the projects
- Nova APIs: discussions around the OpenStack and EC2 API in Nova
- Nova volumes: discussions around volumes / block storage in Nova
- Nova networking: discussions around networking in Nova and the
proposed Network as a Service project
- Nova core: other discussions on Nova
- Glance: discussions on the Glance Image service project
- Swift: discussions on the Swift Object Storage project
- Other projects: discussions on OpenStack incubating projects
You can find a tentative schedule, organized by day or track, at the
following URL: http://summit.openstack.org/ods-d/. You should expect
it to change by next week though, as we tune it to ensure required
people are present where they are needed. So refresh often !
The second type of sessions in unconference sessions. On the Thursday
and the Friday, we'll have an openly-scheduled room available for all
types of presentations or discussions. We'll have a big whiteboard with
empty 30-minute slots: just mark your name and session title in your
preferred time slot. We expect quite a few educational presentations, as
well as discussions around peripheral projects to happen there. So watch
that space !
Finally, from Wednesday to Friday, after lunch and before the design
sessions restart, we'll have 25min of lightning talks. These will also
be openly-scheduled, but in 5-minute increments. Anything
loosely-connected to OpenStack is relevant, so step up and use your 5
minutes of glory :)
All the project technical leads and myself hope that this mix of
sessions will allow us to make the most of those three days together.
See you there !
