On several occasions over the last months, I heard people exposing truths
about the OpenStack Technical Committee. However, those positions were often
inaccurate or incomplete. Arguably we are not communicating enough: governance
changes and resolutions that are brought to the TC are just approved or
rejected. That binary answer is generally not the whole story, and with only
the headline, it is easy to read too much in a past decision. We should do a
better job at communicating beyond that simple answer when the topic is more
complex, and at continuously explaining the role and limits of the TC.
Hopefully this blogpost will help, by busting some of those myths.
Myth #1: "the TC doesn't want Go in OpenStack"
This one comes from the recent rejection of a resolution proposing to add
golang to the list of approved languages, to support merging the hummingbird
feature branch into Swift's master branch. A more accurate way to present
that decision would be to say that a (short) majority of the TC members was not
supporting the addition of golang at that time and under the proposed
conditions. In summary it was more of a "not now, not this way" than a "never".
There were a number of reasons for this decision, but the crux was: adding a
language creates fragmentation and a support cost for all of the OpenStack
community, so we need to make sure "OpenStack" as a whole is successful with
it, beyond any specific project. That means for example having a clear plan
for integrating Go within all our standing processes, while trying to prevent
duplication of effort or gratuitous rewrites. The discussion was actually
recently restarted by Flavio in
this change. We'll need
resources to make this a success, so if you care, please jump in to help.
Myth #2: "the TC doesn't like competition with existing projects"
I'm not exactly sure where this one comes from. I can't think of any specific
TC decision that would explain it. Historically, back when we had programs,
a given team would own a given problem space and could essentially lock
alternatives out. But the Big Tent project structure reform changed that,
allowing competition to happen within our community.
Yes, we still want to encourage cooperation wherever it's possible, so we
still have a requirement which says "where it makes sense, the project
cooperates with existing projects rather than gratuitously competing or
reinventing the wheel". But as long as the new project is different or
presents a different value proposal, it should be fair game. For example we
accepted the Monasca team in the same problem space as the Telemetry team.
And we have several teams working on various deployment recipes.
That said, having competitive solutions in OpenStack on key problem spaces
(like base compute or networking services) creates interesting second-order
questions in terms of what is "core", trademark usage and interoperability.
Those are arguably more downstream concerns than upstream concerns, but that
explains why the deeper you go, the more difficult the community discussion
is likely to be.
Myth #3: "the TC does not set any direction"
There are multiple variants on that one, from "OpenStack needs a benevolent
dictator" to "a camel is a horse designed by a committee". The idea behind it
is that the TC needs to have a very opinionated plan for OpenStack and somehow
force everyone in our community to follow it. Part of this myth is trying to
apply single-vendor software development theory to an open collaboration, and
misunderstanding how other large open source projects (like the Linux kernel)
While the TC members are all well-respected in our community, we can't
unilaterally decide everything for 2500+ developers from 300+ different
organizations, and expect them all to execute The Plan. What the TC can do,
however, is to define the mission, provide an environment, set principles,
enforce common practices, and arbitrate conflicts. In painting terms, the TC
provides the frame, the subject of the painting, a color palette and the
techniques that can be used. But it doesn't paint. Painting is done at each
project team level.
In this cycle, the TC started to drive some simple cross-community goals. The
idea is to collectively make visible progress on a given topic over the course
of a release cycle, to pay back technical debt or to implement a simple feature
across all of OpenStack projects. But this is done as a goal the community
agrees to work on, rather than a top-down mandate.
Myth #4: "the TC, due to the Big Tent, prevents proper focus"
This one is interesting, and I think its roots lie in some misunderstanding of
open source community dynamics. If you consider a finite set of resources and
a zero-sum-game community, then of course adding more projects results in less
resources being dedicated to "important projects". But an open community like
OpenStack is not a finite set of resources. The people and the organizations
in the OpenStack community work and cooperate on a number of projects. Before
the big tent, some of those would not be considered part of the OpenStack
projects, despite helping with the OpenStack mission and following the
OpenStack development principles, and therefore essentially being developed
by the OpenStack community.
Considering more (or less) projects as being part of our community doesn't
decrease (or increase) focus on "important projects". It's up to each
organization in OpenStack to focus on the set of projects it considers
important. For more on that, go read Ed Leafe's
brilliant blogpost, he expressed
it better than I can. Of course there are some efforts (like packaging)
where adding more projects results in diluting focus. But with every added
project comes new blood in our community (rather than artificially keeping it
out), and some of that new blood ends up helping on those efforts. It's not
a zero-sum game, and the big tent makes sure we are open to new ways of
achieving the OpenStack mission and have an inclusive and welcoming community.
This week we are renewing 6 seats
in the 13-member OpenStack Technical Committee. This election has attracted
a large number of candidates, which is a great sign that people care about
the Technical Committee. At the same time, there is a lot of misconceptions
in our community about what the TC is for, what it can or cannot do, and the
overlap with other groups. I'll try to clarify that in this post.
A misleading name
Part of the reason why there are so many misconceptions about the role of the
TC is that its name is pretty misleading. The Technical Committee is not
primarily technical: most of the issues that the TC tackles are open source
project governance issues. The TC is not really a committee either: it is a
group of elected people who will vote on resolutions and changes that are
proposed and which affect OpenStack as a whole. In the US, it is closer to
the Supreme Court than to anything else.
The primary role of the TC is to act as the final decision stage when it comes
to decision-making in the OpenStack open source project. It is extremely
important in an open source project to have a "bucket stops here" body that is
empowered to make decisions for the project if consensus and agreement cannot
be reached elsewhere -- otherwise it risks complete stand-still. I have been
involved in projects which were stuck in such governance grey area, and it
is a pretty ugly situation. It's interesting to note that the mere existence
of the final decision-making body is enough to achieve consensus at the lower
levels: two teams will prefer to find agreement between themselves rather
than call for final arbitration by the TC. This is a feature, not a bug.
An evolving mission
The mission of the TC is to lead the development of "OpenStack". As OpenStack
evolved into one framework with a lot of collaborating components, each of
which developed by project teams with their own governance, the mission of
the TC also evolved. Those days we focus on the "OpenStack" experience. What
does it mean to be an OpenStack project ? What does it imply in terms of
development practices, general principles, common goals, cooperation, minimal
QA or user experience ? Is this new group applying to become an official
OpenStack project team following enough of those rules to be considered "one
of us" ?
Some people would like the TC to single-handedly solve upgrades, scalability,
interoperability or end user experience. Some other people would like the
TC to let the individual project teams do as they want. Reality is, neither
of those extremes are likely to happen. The TC is just 13 (usually busy)
people, they can't solve all the issues in OpenStack by themselves. They are
elected because we trust them to make the right decisions for OpenStack, not
because they are the ultimate 100x engineers who can fix everything. On the
other hand, the TC is not powerless: by continuously refining what it means to
be an "OpenStack project", it can influence OpenStack project teams to address
the right topics. We did that through assert tags, which encouraged teams to
improve their upgrade story or adopt a sane feature deprecation policy. We'll
do that through release cycle goals, to drive visible improvements across
all the components in OpenStack.
This is also why the TC needs all the help it can get. I'm excited we now have
the Architecture workgroup,
an open group of people interested in addressing long-standing architecture
issues in OpenStack as a whole. I'm thrilled that we have a
an open group of people encouraging leaders in OpenStack to adopt Servant
Leadership practices and tools. We are electing 13 members to vote and make
the final calls, but all the ideas don't have to come from those 13 members.
If you're a candidate and you're not elected, it doesn't prevent you from
working on governance problems and propose changes.
So... If you are an eligible voter, please take the time to read the candidates
platform emails and vote. Whoever gets elected, they will need the legitimacy
that only a good turnout in elections can give them. And if you are a candidate
and don't get elected, please consider joining those open workgroups, and
propose governance changes -- keeping "OpenStack" together is definitely not
a 13-people task.
I was recently privately asked how I expected the split of the current
OpenStack Design Summit into two different events to enhance the overall
OpenStack development process. Here is the answer I gave...
We are working on splitting the "Design Summit" into a more open
requirements-gathering and feedback forum at the Summit on one side,
and a separated event for project team members on the other side. I expect
this change will greatly enhance the development process, for the following
Key upstream developers and PTLs were too busy during the Summit week
to actually watch and listen. Having them more available during the Summit
week (where all of our community is present) should help a lot in getting
the right priorities across. The feedback sessions were also not very well
balanced (being organized as part of the upstream developer-centric event
called the "Design Summit"). Making it more like a regular Summit event will
help make it a neutral exchange forum between community peers (rather than
dev kings listening to grievances in their throne hall).
Project teams were lacking time to work together as a team: building
trust, organizing the work, agreeing on priorities and assigning tasks. The
current "Design Summit" doesn't work so well for that because it doubles as
a general forum and a lot of people outside the team members were attending
the sessions. There were also too many distractions to hang out between team
members and build social bonds. This is why a lot of teams organized specific,
separated events (the "midcycles"): to get more time together. The new
"Project Teams Gathering" event is all about providing that time to work
together as a team.
Partly as a result of separate midcycle events, project teams operated in
silos and were unlikely to be exposed or take on critical cross-project work.
They would also skip the cross-project workshops at the Design Summit since so
much is happening at the same time. The new event should provide specific time
for cross-project work, without anything running against it. It should
encourage team members from a vertical team (Nova, Neutron...) to join and
participate in horizontal / cross-project efforts (QA, Infra, technology
convergence, release theme...), to break out of their silo and become true
The timing of the "Design Summit" event was inefficient. It was too late
to organize work, too early to get feedback on the recent release. By splitting
the events, we put the main Summit further away from the release -- giving time
for packagers, deployers, solutions builders to build a product and start
experimenting with the new release. The quality of the feedback we get should
therefore improve a lot. It also lets us put the new event closer to the start
of the development cycle (closer to the previous cycle feature freeze),
ensuring there is less "down" time between cycles (almost two months in the
So, in summary, I expect the new split event format to solve long-standing
issues that made the "Design Summit" no longer efficient. It should increase
our productivity, but also greatly improve the feedback loops between
downstream and upstream.
In a global and virtual community, high-bandwidth face-to-face time is
essential. This is why we made the OpenStack Design Summits an integral
part of our processes from day 0. Those were set at the beginning of each
of our development cycles to help set goals and organize the work for the
upcoming 6 months. At the same time and in the same location, a more
traditional conference was happening, ensuring a lot of interaction between
the upstream (producers) and downstream (consumers) parts of our community.
This setup, however, has a number of issues. For developers first: the
"conference" part of the common event got bigger and bigger and it is
difficult to focus on upstream work (and socially bond with your teammates)
with so much other commitments and distractions. The result is that our
design summits are a lot less productive than they used to be, and we
organize other events ("midcycles") to fill our focus and small-group
socialization needs. The timing of the event (a couple of weeks after the
previous cycle release) is also suboptimal: it is way too late to gather any
sort of requirements and priorities for the already-started new cycle, and
also too late to do any sort of work planning (the cycle work started almost
2 months ago).
But it's not just suboptimal for developers. For contributing companies,
flying all their developers to expensive cities and conference hotels so
that they can attend the Design Summit is pretty costly, and the goals of
the summit location (reaching out to users everywhere) do not necessarily
align with the goals of the Design Summit location (minimize and balance
travel costs for existing contributors). For the companies that build products
and distributions on top of the recent release, the timing of the common event
is not so great either: it is difficult to show off products based on the
recent release only two weeks after it's out. The summit date is also too
early to leverage all the users attending the summit to gather feedback on
the recent release -- not a lot of people would have tried upgrades by
summit time. Finally a common event is also suboptimal for the events
organization : finding venues that can accommodate both events is becoming
Time is ripe for a change. After Tokyo, we at the Foundation have been
considering options on how to evolve our events to solve those issues. This
proposal is the result of this work. There is no perfect solution here (and
this is still work in progress), but we are confident that this strawman
solution solves a lot more problems than it creates, and balances the needs
of the various constituents of our community.
The idea would be to split the events. The first event would be for upstream
technical contributors to OpenStack. It would be held in a simpler, scaled-back
setting that would let all OpenStack project teams meet in separate rooms,
but in a co-located event that would make it easy to have ad-hoc cross-project
discussions. It would happen closer to the centers of mass of contributors,
in less-expensive locations.
More importantly, it would be set to happen a couple of weeks before the
previous cycle release. There is a lot of overlap between cycles. Work on a
cycle starts at the previous cycle feature freeze, while there is still 5
weeks to go. Most people switch full-time to the next cycle by RC1.
Organizing the event just after that time lets us organize the work and
kickstart the new cycle at the best moment. It also allows us to use our time
together to quickly address last-minute release-critical issues if such
The second event would be the main downstream business conference, with
high-end keynotes, marketplace and breakout sessions. It would be organized
two or three months after the release, to give time for all downstream
users to deploy and build products on top of the release. It would be the best
time to gather feedback on the recent release, and also the best time to have
strategic discussions: start gathering requirements for the next cycle,
leveraging the very large cross-section of all our community that attends
To that effect, we'd still hold a number of strategic planning sessions at
the main event to gather feedback, determine requirements and define overall
cross-project themes, but the session format would not require all project
contributors to attend. A subset of contributors who would like to participate
in these sessions can collect and relay feedback to other team members for
implementation (similar to the Ops midcycle). Other contributors will also
want to get more involved in the conference, whether that's giving
presentations or hearing user stories.
The split should ideally reduce the needs to organize separate in-person
mid-cycle events. If some are still needed, the main conference venue and
time could easily be used to provide space for such midcycle events (given
that it would end up happening in the middle of the cycle).
In practice, the split means that we need to stagger the events and cycles.
We have a long time between Barcelona and the Q1 Summit in the US, so the
idea would be to use that long period to insert a smaller cycle (Ocata) with
a release early March, 2017 and have the first specific contributors event
at the start of the P cycle, mid-February, 2017. With the already-planned
events in 2016 and 2017 it is the earliest we can make the transition. We'd
have a last, scaled-down design summit in Barcelona to plan the shorter cycle.
With that setup, we hope that we can restore the productivity and focus of
the face-to-face contributors gathering, reduce the need to have midcycle
events for social bonding and team building, keep the cost of getting all
contributors together once per cycle under control, maintain the feedback
loops with all the constituents of the OpenStack community at the main event,
and better align the timing of each event with the reality of the release
NB: You will note that I did not name the separated event "Design Summit" --
that is because Design will now be split into feedback/requirements
gathering (the why at the main event) and execution planning and
kickstarting (the how at the contributors-oriented event), so that name
doesn't feel right anymore. We can bikeshed on the name for the new event
This was also posted to the openstack-dev ML:
please comment and follow-up there if you have thoughts to share.
We are 5 years into the OpenStack ride (and 3 years into the OpenStack
Foundation ride), and the challenges for our community are evolving.
In this article I want to talk about what I consider the most significant
threat for our open source community today: the loss of our common culture.
Over the past year we evolved the OpenStack project model to adopt an
inclusive approach. Project teams which
work on deliverables that help us achieve the OpenStack Mission, and which
follow our development and community practices should generally be accepted
under the "big tent". As we explained in
this presentation in Vancouver,
we moved from asking "is this OpenStack ?" to asking "are you OpenStack ?".
What does it mean to be OpenStack ? We wrote down a
set of principles,
based on the original four opens
that were defined at the very beginning of this journey. But
"being OpenStack" goes beyond that. It is to be aligned on a common goal,
be part of the same effort, be the same tribe. OpenStack relies on a number
of individuals working cross-project (on infrastructure, QA, documentation,
release processes, interoperability, user experience, API guidelines,
vulnerability management, election organization...). It is because we belong
to the same tribe that some people and organizations care enough about
"OpenStack" as a whole to dedicate time to those essential cross-project
This is why we standardize on logged IRC channels as a communication medium,
why we ask that every project change goes through Gerrit, and why we should
very conservatively add new programming languages to the mix. Some people
advocate letting OpenStack project teams pick whatever language they want,
or letting them meet on that new trendy videoconferencing app, or letting
them track bugs on separate JIRA instances. More freedom sounds good at first
glance, but doing so would further fragment our community into specific silos
that all behave differently. Doing so would make recruiting for those
essential cross-project efforts even harder than it is today, while at the
same time making the work of those cross-project efforts significantly more
complex. Doing so would make our community crumble under its own weight.
We started this journey with a pretty strong common culture. It was mostly
oral tradition. We assumed that as OpenStack grew, our culture would
naturally be assimilated by new members. And it did, for quite some time.
But today we are at a point where we dramatically expanded our community
(we doubled the number of project teams over the last year) and our common
culture did not naturally transmit to newcomers. Silos with local traditions
have formed. Teams don't all behave in the same way anymore. Most team
members only care about a single project team. We struggle to move from one
project to another. We struggle to provide common solutions that work for
everyone. We struggle to recruit for cross-project efforts more than we ever
did. OpenStack's future as a community is at risk. It's time to hold the
culture line, rather than time to further relax it.
It is also more than time that we document our common culture, so that it
can be explicitly communicated to everyone in the OpenStack ecosystem
(current and prospective members). We started a workgroup at the
Technical Committee, held a
virtual sprint to get a
base version written, and now here it is: the first version of the
OpenStack Project team guide.
Read it, refer to it, communicate it to your OpenStack community fellows,
propose changes to it. It is an essential tool for us to overcome this new
challenge. It's certainly not the only tool, and I hope we'll be able to
dedicate a cross-project session at the Mitaka Design Summit in Tokyo to
further discuss this topic.