In less than two months the OpenStack development community will gather in
Paris to discuss the details of the Kilo development cycle. It starts after the
keynotes on the Tuesday of the summit week, and ends at the end of the day on
the Friday. We decided a number of changes in the Design Summit organization in
order to make it an even more productive time for all of us.
Tuesday
On the Tuesday we'll have cross-project workshops. These are 40-min or
90-min sessions on issues that span multiple programs, and where we try to
reach alignment and exploit that little window in time where all projects are
in the same place. These were very successful in Atlanta, so we'll do them
again. We started brainstorming potential topics on
this etherpad.
On the same day there will be time for scheduled sessions for incubating
projects, as well as some sessions for other OpenStack projects. I'll post
details on how to participate to that on the
openstack-dev mailing-list soon.
Wednesday and Thursday
On Wednesday and Thursday we'll have our usual scheduled sessions. Those
are 40-min sessions on a specific theme, and they appear on the Design Summit
schedule. The idea is to use those few visible time slots to gather feedback on
issues that need a lot of external input, but also to address some obvious
elephants in the room.
Friday
On the Friday we'll have contributors meetups. All programs will have
a space for a half-day or a full-day of informal meetup with an open agenda.
The idea is to use that time to get alignment between core contributors on the
cycle objectives, to solve project-internal issues, or have further discussions
on a specific effort. The format is very much like the mid-cycle meetups.
Program pods
Program pods are roundtables contributors can informally gather around to
continue a discussion when they don't have scheduled space at the same time. In
Paris we have limited space, so there will be a limited number of program pods.
Incubating programs will have a designated pod, but other programs will have to
share the remaining available space. Hopefully it won't be that much of a
musical-chairs issue, thanks to the contributors meetup day on Friday.
Planning the summit
We decided to abandon the formal session suggestion system in favor of a more
collaborative and open approach. Topics are suggested on an open document (see
the list here), discussed and
deduplicated at IRC team meetings, and finally split between scheduled session
slots and the contributors meetup agenda. This is done under the guidance of
the Kilo PTLs, which we'll be electing soon. Reminders will be posted on the
openstack-dev list.
So I invite everyone to join those documents and the future team meetings
discussing the summit agenda. Let's all make this Design Summit great !
Yesterday the
OpenStack Technical Committee
concluded the end-of-cycle graduation review for
Zaqar
(the project previously called Marconi), and decided Zaqar should stay in
incubation during the Kilo cycle.
What it does not mean
It does not mean that the Zaqar developers failed to deliver. The Zaqar team
crossed all the checkboxes in our
integration requirements
checklist. They consistently took into account our feedback and proved
themselves ready to make any required change. This is why the decision appears
so unfair to them: they could not really do more.
It does not mean that OpenStack wouldn't benefit from a basic queue/messaging
service. If anything, it means that we actually care a lot about such a
service, enough to have strong opinions about how it should be done, the scope
of the designed solution and what exact use cases it enables or prevents.
What it does mean
It does mean that our incubation process is broken. The Technical Committee
could not provide to the Zaqar team the feedback they needed to be successful.
It could not discuss Zaqar's scope and design well before the end-of-cycle
graduation review, when time constraints push for a quick decision. In this
cycle the TC focused on existing issues with existing integrated projects (like
the Neutron/nova-network gaps), which some might argue need to be fixed before
we add more integrated projects. We ended up not having the time to follow up
on incubated projects as much as we should have.
The good news here is that the gap coverage for existing integrated projects
during the Juno cycle was generally successful, so this was probably more of a
one-time thing. If we preserve the incubation process as-is, next cycle we plan
to assign a specific TC member mentor to every incubated project, so that we
provide consistent feedback and raise the necessary discussions in due time.
It does also mean that our "in or out" integrated release model is broken.
Having a single class you can graduate to means accepting a new integrated
project has a significant cost on existing horizontal teams. The integrated
release is therefore very conservative -- if there is any doubt, better leave
the incubated project to resolve those doubts while in incubation, rather than
in the first integrated cycle with the time pressure of the next release date.
And if it's not "in", it's just "out". There is no intermediary area. If we had
several classes (or layers), triggering different amounts of resources, then it
would be much less black or white and we could have a conservative class and an
inclusive class. We are just starting a discussion at the TC level on how we
could evolve that model (and the "programs" that back that model) to allow us
to scale.
The next steps
What's up next for Zaqar, then ? It is the duty of the Technical Committee to
explain its decision, and work out under which conditions it would accept Zaqar
in 6 months time. So now that we are free from the time pressure of the
integration decision, we need to determine which options are on the table and
which ones would be preferred. This should happen in the coming weeks, before
the Design Summit in Paris. Then we really need to follow-up on that feedback,
by designating a member from the newly-elected TC as the Zaqar mentor, tasked
with making sure the communication lines stay open at all times and we stay in
alignment. We need to learn from our mistakes, and not find ourselves in the
same position in 6 months.
Finally, depending on the outcome of the integrated release model evolution
discussion, it is possible that the decision to stay in incubation during the
Kilo cycle will not matter that much. If we set up a layered model for example,
it is very possible that Zaqar would directly appear on that map. So my advice
would be to follow closely how that discussion goes to see how it may affect
Zaqar in the near future.
Some people asked me to analyze the results of the recent TC election in
the same way I ran the analysis for the previous
one.
I finally found the time to do it, here are the results.
The April 2014 election was set up to renew 7 of the 13 TC members. We
had 17 candidates, you can find the official results of the election
here.
Condorcet spread
This graph shows how each candidate was ranked. The first bubble on the
left represents the number of people that have placed that candidate as
their only first choice. The last bubble on the right represents the
number of people that have placed this candidate as their only last
choice. If multiple candidates are ranked at the same level, we average
their "score".
We can see that once again the Condorcet algorithm preferred a
consensual candidate (Devananda) over less-consensual ones (Julien,
Sergey). It's also interesting to compare the spread between Michael,
Jim and Mark.
Proportional Condorcet
At the previous election, running the same ballot with CIVS so-called
"proportional mode" option altered the result. This time, the
proportional mode returns the exact same set of
winners.
Partisan voting
The goal of this analysis is to detect blocks of voters, who
consistently place a set of candidates above anyone else. I slightly
modified my script to reveal the most popular pairs: calculate how many
people place the same two people above anyone else, and try to detect
bias in the most popular pairs.
The most popular pair was Vish/Thierry (with 5.58% of voters placing us
above anyone else). We could call that the old-timers party. The
second most popular pair was Jay/Sergey (the Mirantis party) with
5.13%. 4.24% of the voters placed the Julien/Thierry pair on top of
anything else: that could reveal an influential French party ! The
Foundation party pair (Jim/Thierry) was preferred by 3.34%, same
result as the Foodie party (Mark/Thierry). The Jay/Vish pair was
preferred by 3.12% of the voters (a variant of the old-timers party).
It's also worth noting that 2.45% of the voters favored the
Flavio/Steven pair (best score of the RedHat pairs), 1.56% favored
Michael/JohnG (best score of the Rackspace pairs) and 1.11% favored
Joe/Devananda (best score of the HP pairs).
While this analysis shows some corporate bias in the vote, it's worth
noting that (a) it's extremely limited, (b) it's actually decreasing
from the previous election, (c) it did not affect the result in any
significant way and (d) it's comparable to other bias (old timers,
French people).
Run your own analysis
That's all I had to share. But don't blindly trust me, you can run your
own analysis by downloading the anonymized
ballot
!
Various governance models exist for free and open source software
projects. Most of those happen naturally, some of them are chosen...
Which one is the best ? Is there a best ? How could we judge the best ?
Like any ecosystem, I'd postulate that F/OSS project communities should
have long-term survival as their main goal: the ability to continue
operation as the same community over time, without fracture of fork.
Dictatorship
The "benevolent dictator for life" model usually happens naturally. The
project is often originally the brain child of a single, talented
individual, who retains final say over everything that happens to
their project. This usually works very well: the person is naturally
respected in the community. It also naturally allows for opinionated
design, and people who sign up to the project can't ignore what they
sign up for.
The main issue with that setup is that it's not replicable, it can't be
dictated. It either happens naturally, or it will never happen. You
don't choose someone to become your dictator-for-life after the fact.
Any attempt to do so would fail to get enough legitimacy and natural
respect to make it last. The second issue with that setup is that it's
not durable. If the dictator stops being active in the community, their
opinion is not as much respected anymore (especially by new
contributors), which usually triggers a painful fork or governance model
switch (that's what happened in Gentoo). Even in the rare cases where
the original dictator manages to retain interest and respect in the
project, it's inherently brittle: the "natural" dictator can't really be
replaced in case something bad happens. Succession is always dirty. So
from a long-term survival standpoint, this model is not that great.
Aristocracy
Aristocracy is used to solve the perceived drawbacks of the
dictator-for-life model. Instead of focusing on one person, let's have a
group of people in control of the project, and let that group
self-select successors in the wider pool of contributors. That's the
role of "committers" in certain projects, and it's also how Apache
project management committees (PMCs) usually work. It also works quite
well, with self-selection usually ensuring that the members share enough
common culture to reach consensus on most decisions.
The drawback here is obviously the self-selection bias. Aristocracies
all fall after getting more and more disconnected from the people they
control, and revolution happens. Open source aristocracies are no
different: they fall after gradually growing disconnected from their
project contributors base. Whenever contributors to an open source
project feel like their leaders are no longer representative of the
contributors or relevant to the present of the project, this disconnect
happens. In mild cases, people just go contribute somewhere else, and in
difficult cases this usually triggers a fork.
Direct democracy / Anarchy
The obvious way to solve that disconnect is to give the power directly
to the contributors. Direct democracy projects give ultimate power to
all the contributors. Anarchy projects let contributors do whatever they
want. Debian is an interesting mix of the two: developers vote on
general resolutions, but maintainers also have a lot of control on their
packages.
While these models have a certain appeal, those projects usually have a
hard time taking necessary decisions that affect the whole project, so
they tend to linger not taking any critical decision. It's also a model
that is difficult to evolve: when you try to add new layers on top of
it, they are never really accepted by the contributors base.
Representative democracy
That leaves us with representative democracy. You regularly designate a
small group of people and trust them to make the right decisions for the
governance of the project. It can happen in cases where there was no
natural dictator at the beginning of the project. It's different from
aristocracy in that they are chosen by the contributors base and
regularly renewed -- ensuring that they are always seen as a fair
representation of the contributors to the project. It's more efficient
than direct democracy or anarchy in making clear and opinionated
decisions.
Now it's far from perfect. As Churchill famously said, it's the worst
form of government, except all those other forms that have been tried
from time to time. It also only works if the elected people are seen as
legitimate and representative, so it requires good participation levels
in elections. So here is my plea: the OpenStack Technical Committee,
which oversees the development of the OpenStack open source project as a
whole, is being partially renewed this week. If you're an OpenStack
contributor, please vote: this will ensure that elected people have the
legitimacy necessary for making the decisions that need to be made, and
increase the health of the project.
Since the very beginning of OpenStack we fulfilled our Open
Design promise by
organizing a developer gathering open to all OpenStack contributors at
the beginning of all our development cycles, called the Design
Summit. Those events have
proven to be an essential part of OpenStack success and growth.
Design Summits are a set of discussion sessions under the auspices of a
given OpenStack program. There are no formal presentations or speakers,
just open discussions around a given development theme. The elected
Program Technical Leads are responsible for picking a set of discussion
topics, and they take suggestions from the rest of the community on our
session suggestion website at
summit.openstack.org.
Improvements
One of the last sessions at the Icehouse Design Summit in Hong-Kong was
about the Design Summit format, and how we should improve on it. Several
issues were reported during that session, like:
- the inability for technical people to attend (or present at) the
rest of the OpenStack Summit
- the (in)visibility of the Unconference track, making it difficult
for nascent projects to attract interested contributors
- the difficulty to have cross-project discussions in a schedule
strictly organized around project-specific topics
- the difficulty for incubated projects to continue to collaborate
during the week, outside of the limited scheduled slots allocated to
them
I'm happy to report that we acted on that feedback and will implement a
number of changes in the upcoming Juno Design Summit in Atlanta in May.
First, we started staggering the Design Summit from the rest of the
OpenStack Summit. The main event starts on Monday and ends on Thursday,
while the Juno Design Summit starts on Tuesday and ends on Friday. This
should allow our key technical assets to attend some of the conference
and maybe present there.
Second, the Unconference track is abandoned. It will be replaced by
several initiatives. Part of the Unconference was traditionally used
by open source projects related to OpenStack to present themselves and
recruit contributors. We'll have an Other projects track at the
Design Summit to cover for those. This will be limited to one session
per project, but those will appear on the official schedule. If you are
an open source project related to OpenStack and would like one of those
slots, please head to the session suggestion
site !
Another classic use of the Unconference was ad-hoc continuation of
discussions that started in scheduled sessions, or coverage of lesser
topics that couldn't find a place in scheduled sessions. To cover for
that, we'll set up a roundtable for each program (complete with
paperboard) to serve as a rallying point for contributors around that
program. This designated space (codenamed project pod) can be used
to have additional discussions and continue collaboration outside of the
limited scheduled sessions.
Last but not least, we'll dedicate the first day of the summit (Tuesday)
to cross-project workshops. During that day, no other integrated
project sessions will be running, which should facilitate presence of
key stakeholders. We'll be able to discuss OpenStack-wide goals,
convergence, integration and other cross-project issues.
We hope that those changes will let us make the most of those 4 days all
together. At the end of the Juno Design Summit we'll discuss whether
those changes were an improvement and whether we should do them again
for the K design summit in Paris in November. See you all in Atlanta in
8 weeks!
