How splitting the Design Summit enhances the development process

I was recently privately asked how I expected the split of the current OpenStack Design Summit into two different events to enhance the overall OpenStack development process. Here is the answer I gave...

We are working on splitting the "Design Summit" into a more open requirements-gathering and feedback forum at the Summit on one side, and a separated event for project team members on the other side. I expect this change will greatly enhance the development process, for the following reasons:

  1. Key upstream developers and PTLs were too busy during the Summit week to actually watch and listen. Having them more available during the Summit week (where all of our community is present) should help a lot in getting the right priorities across. The feedback sessions were also not very well balanced (being organized as part of the upstream developer-centric event called the "Design Summit"). Making it more like a regular Summit event will help make it a neutral exchange forum between community peers (rather than dev kings listening to grievances in their throne hall).

  2. Project teams were lacking time to work together as a team: building trust, organizing the work, agreeing on priorities and assigning tasks. The current "Design Summit" doesn't work so well for that because it doubles as a general forum and a lot of people outside the team members were attending the sessions. There were also too many distractions to hang out between team members and build social bonds. This is why a lot of teams organized specific, separated events (the "midcycles"): to get more time together. The new "Project Teams Gathering" event is all about providing that time to work together as a team.

  3. Partly as a result of separate midcycle events, project teams operated in silos and were unlikely to be exposed or take on critical cross-project work. They would also skip the cross-project workshops at the Design Summit since so much is happening at the same time. The new event should provide specific time for cross-project work, without anything running against it. It should encourage team members from a vertical team (Nova, Neutron...) to join and participate in horizontal / cross-project efforts (QA, Infra, technology convergence, release theme...), to break out of their silo and become true "OpenStack" contributors.

  4. The timing of the "Design Summit" event was inefficient. It was too late to organize work, too early to get feedback on the recent release. By splitting the events, we put the main Summit further away from the release -- giving time for packagers, deployers, solutions builders to build a product and start experimenting with the new release. The quality of the feedback we get should therefore improve a lot. It also lets us put the new event closer to the start of the development cycle (closer to the previous cycle feature freeze), ensuring there is less "down" time between cycles (almost two months in the current setup).

So, in summary, I expect the new split event format to solve long-standing issues that made the "Design Summit" no longer efficient. It should increase our productivity, but also greatly improve the feedback loops between downstream and upstream.

Splitting out the OpenStack Design Summit

In a global and virtual community, high-bandwidth face-to-face time is essential. This is why we made the OpenStack Design Summits an integral part of our processes from day 0. Those were set at the beginning of each of our development cycles to help set goals and organize the work for the upcoming 6 months. At the same time and in the same location, a more traditional conference was happening, ensuring a lot of interaction between the upstream (producers) and downstream (consumers) parts of our community.

This setup, however, has a number of issues. For developers first: the "conference" part of the common event got bigger and bigger and it is difficult to focus on upstream work (and socially bond with your teammates) with so much other commitments and distractions. The result is that our design summits are a lot less productive than they used to be, and we organize other events ("midcycles") to fill our focus and small-group socialization needs. The timing of the event (a couple of weeks after the previous cycle release) is also suboptimal: it is way too late to gather any sort of requirements and priorities for the already-started new cycle, and also too late to do any sort of work planning (the cycle work started almost 2 months ago).

But it's not just suboptimal for developers. For contributing companies, flying all their developers to expensive cities and conference hotels so that they can attend the Design Summit is pretty costly, and the goals of the summit location (reaching out to users everywhere) do not necessarily align with the goals of the Design Summit location (minimize and balance travel costs for existing contributors). For the companies that build products and distributions on top of the recent release, the timing of the common event is not so great either: it is difficult to show off products based on the recent release only two weeks after it's out. The summit date is also too early to leverage all the users attending the summit to gather feedback on the recent release -- not a lot of people would have tried upgrades by summit time. Finally a common event is also suboptimal for the events organization : finding venues that can accommodate both events is becoming increasingly complicated.

Time is ripe for a change. After Tokyo, we at the Foundation have been considering options on how to evolve our events to solve those issues. This proposal is the result of this work. There is no perfect solution here (and this is still work in progress), but we are confident that this strawman solution solves a lot more problems than it creates, and balances the needs of the various constituents of our community.

The idea would be to split the events. The first event would be for upstream technical contributors to OpenStack. It would be held in a simpler, scaled-back setting that would let all OpenStack project teams meet in separate rooms, but in a co-located event that would make it easy to have ad-hoc cross-project discussions. It would happen closer to the centers of mass of contributors, in less-expensive locations.

More importantly, it would be set to happen a couple of weeks before the previous cycle release. There is a lot of overlap between cycles. Work on a cycle starts at the previous cycle feature freeze, while there is still 5 weeks to go. Most people switch full-time to the next cycle by RC1. Organizing the event just after that time lets us organize the work and kickstart the new cycle at the best moment. It also allows us to use our time together to quickly address last-minute release-critical issues if such issues arise.

The second event would be the main downstream business conference, with high-end keynotes, marketplace and breakout sessions. It would be organized two or three months after the release, to give time for all downstream users to deploy and build products on top of the release. It would be the best time to gather feedback on the recent release, and also the best time to have strategic discussions: start gathering requirements for the next cycle, leveraging the very large cross-section of all our community that attends the event.

To that effect, we'd still hold a number of strategic planning sessions at the main event to gather feedback, determine requirements and define overall cross-project themes, but the session format would not require all project contributors to attend. A subset of contributors who would like to participate in these sessions can collect and relay feedback to other team members for implementation (similar to the Ops midcycle). Other contributors will also want to get more involved in the conference, whether that's giving presentations or hearing user stories.

The split should ideally reduce the needs to organize separate in-person mid-cycle events. If some are still needed, the main conference venue and time could easily be used to provide space for such midcycle events (given that it would end up happening in the middle of the cycle).

In practice, the split means that we need to stagger the events and cycles. We have a long time between Barcelona and the Q1 Summit in the US, so the idea would be to use that long period to insert a smaller cycle (Ocata) with a release early March, 2017 and have the first specific contributors event at the start of the P cycle, mid-February, 2017. With the already-planned events in 2016 and 2017 it is the earliest we can make the transition. We'd have a last, scaled-down design summit in Barcelona to plan the shorter cycle.


With that setup, we hope that we can restore the productivity and focus of the face-to-face contributors gathering, reduce the need to have midcycle events for social bonding and team building, keep the cost of getting all contributors together once per cycle under control, maintain the feedback loops with all the constituents of the OpenStack community at the main event, and better align the timing of each event with the reality of the release cycles.

NB: You will note that I did not name the separated event "Design Summit" -- that is because Design will now be split into feedback/requirements gathering (the why at the main event) and execution planning and kickstarting (the how at the contributors-oriented event), so that name doesn't feel right anymore. We can bikeshed on the name for the new event later :)

This was also posted to the openstack-dev ML: please comment and follow-up there if you have thoughts to share.

OpenStack Common Culture

We are 5 years into the OpenStack ride (and 3 years into the OpenStack Foundation ride), and the challenges for our community are evolving. In this article I want to talk about what I consider the most significant threat for our open source community today: the loss of our common culture.

Over the past year we evolved the OpenStack project model to adopt an inclusive approach. Project teams which work on deliverables that help us achieve the OpenStack Mission, and which follow our development and community practices should generally be accepted under the "big tent". As we explained in this presentation in Vancouver, we moved from asking "is this OpenStack ?" to asking "are you OpenStack ?".

What does it mean to be OpenStack ? We wrote down a set of principles, based on the original four opens that were defined at the very beginning of this journey. But "being OpenStack" goes beyond that. It is to be aligned on a common goal, be part of the same effort, be the same tribe. OpenStack relies on a number of individuals working cross-project (on infrastructure, QA, documentation, release processes, interoperability, user experience, API guidelines, vulnerability management, election organization...). It is because we belong to the same tribe that some people and organizations care enough about "OpenStack" as a whole to dedicate time to those essential cross-project efforts.

This is why we standardize on logged IRC channels as a communication medium, why we ask that every project change goes through Gerrit, and why we should very conservatively add new programming languages to the mix. Some people advocate letting OpenStack project teams pick whatever language they want, or letting them meet on that new trendy videoconferencing app, or letting them track bugs on separate JIRA instances. More freedom sounds good at first glance, but doing so would further fragment our community into specific silos that all behave differently. Doing so would make recruiting for those essential cross-project efforts even harder than it is today, while at the same time making the work of those cross-project efforts significantly more complex. Doing so would make our community crumble under its own weight.

We started this journey with a pretty strong common culture. It was mostly oral tradition. We assumed that as OpenStack grew, our culture would naturally be assimilated by new members. And it did, for quite some time. But today we are at a point where we dramatically expanded our community (we doubled the number of project teams over the last year) and our common culture did not naturally transmit to newcomers. Silos with local traditions have formed. Teams don't all behave in the same way anymore. Most team members only care about a single project team. We struggle to move from one project to another. We struggle to provide common solutions that work for everyone. We struggle to recruit for cross-project efforts more than we ever did. OpenStack's future as a community is at risk. It's time to hold the culture line, rather than time to further relax it.

It is also more than time that we document our common culture, so that it can be explicitly communicated to everyone in the OpenStack ecosystem (current and prospective members). We started a workgroup at the Technical Committee, held a virtual sprint to get a base version written, and now here it is: the first version of the OpenStack Project team guide. Read it, refer to it, communicate it to your OpenStack community fellows, propose changes to it. It is an essential tool for us to overcome this new challenge. It's certainly not the only tool, and I hope we'll be able to dedicate a cross-project session at the Mitaka Design Summit in Tokyo to further discuss this topic.

The Age of Foundations

At OSCON last week, Google announced the creation around Kubernetes of the Cloud-Native Computing Foundation. The next day, Jim Zemlin dedicated his keynote to the (recently-renamed) Open Container Initiative, confirming the Linux Foundation's recent shift towards providing Foundations-as-a-Service. Foundations ended up being the talk of the show, with some questioning the need for Foundations for everything, and others discussing the rise of Foundations as tactical weapons.

Back to the basics

The main goal of open source foundations is to provide a neutral, level and open collaboration ground around one or several open source projects. That is what we call the upstream support goal. Projects are initially created by individuals or companies that own the original trademark and have power to change the governance model. That creates a tilted playing field: not all players are equal, and some of them can even change the rules in the middle of the game. As projects become more popular, that initial parentage becomes a blocker for other contributors or companies to participate. If your goal is to maximize adoption, contribution and mindshare, transferring the ownership of the project and its governance to a more neutral body is the natural next step. It removes barriers to contribution and truly enables open innovation.

Now, those foundations need basic funding, and a common way to achieve that is to accept corporate members. That leads to the secondary goal of open source foundations: serve as a marketing and business development engine for companies around a common goal. That is what we call the downstream support goal. Foundations work to build and promote a sane ecosystem around the open source project, by organizing local and global events or supporting initiatives to make it more usable: interoperability, training, certification, trademark licenses...

Not all Foundations are the same

At this point it's important to see that a foundation is not a label, the name doesn't come with any guarantee. All those foundations are actually very different, and you need to read the fine print to understand their goals or assess exactly how open they are.

On the upstream side, few of them actually let their open source project be completely run by their individual contributors, with elected leadership (one contributor = one vote, and anyone may contribute). That form of governance is the only one that ensures that a project is really open to individual contributors, and the only one that prevents forks due to contributors and project owners not having aligned goals. If you restrict leadership positions to appointed seats by corporate backers, you've created a closed pay-to-play collaboration, not an open collaboration ground. On the downstream side, not all of them accept individual members or give representation to smaller companies, beyond their founding members. Those details matter.

When we set up the OpenStack Foundation, we worked hard to make sure we created a solid, independent, open and meritocratic upstream side. That, in turn, enabled a pretty successful downstream side, set up to be inclusive of the diversity in our ecosystem.

The future

I see the "Foundation" approach to open source as the only viable solution past a given size and momentum around a project. It's certainly preferable to "open but actually owned by one specific party" (which sooner or later leads to forking). Open source now being the default development model in the industry, we'll certainly see even more foundations in the future, not less.

As this approach gets more prevalent, I expect a rise in more tactical foundations that primarily exist as a trade association to push a specific vision for the industry. At OSCON during those two presentations around container-driven foundations, it was actually interesting to notice not the common points, but the differences. The message was subtly different (pods vs. containers), and the companies backing them were subtly different too. I expect differential analysis of Foundations to become a thing.

My hope is that as the "Foundation" model of open source gets ubiquitous, we make sure that we distinguish those which are primarily built to sustain the needs or the strategy of a dozen of large corporations, and those which are primarily built to enable open collaboration around an open source project. The downstream goal should stay a secondary goal, and new foundations need to make sure they first get the upstream side right.

In conclusion, we should certainly welcome more Foundations being created to sustain more successful open source projects in the future. But we also need to pause and read the fine print: assess how open they are, discover who ends up owning their upstream open source project, and determine their primary reason for existing.

New OpenStack component versioning

Yesterday we reached the liberty-1 development milestone. You may have noticed from the announcement that the various components released were all using new, different version numbers. What's going on here ?

Once upon a time

Since the beginning of OpenStack we've been using two versioning schemes. One was for projects released once every 6 months and following a schedule of development milestones and release candidates. Those would be using a YEAR.N version number (like 2015.1 for Kilo).

Another was used by Swift, which was already mature when OpenStack started, and which released intermediary versions as-needed throughout the cycle. It would use a X.Y.Z version number which looked a lot more like semantic versioning.

At the end of the cycle, we would coordinate a final release that would combine both. For example the "Kilo" release would be made of Nova 2015.1.0, Swift 2.3.0, and everything else at 2015.1.0.

Recent developments

A few things happened over the last two cycles. First, we released more and more libraries, and those would follow a strict X.Y.Z semantic versioning. Those would also have an final release in the cycle, from which a stable branch would be maintained for critical bugfixes and vulnerability fixes. So the portion of commonly-versioned YEAR.N deliverables was fast decreasing.

Second, some projects got more mature and/or more able to release fully-functional intermediary releases as-needed. As a community, we still can't support more than one stable branch every 6 months, so those intermediary releases won't get backports, but past a given maturity step, it's still a great thing to push new features to bleeding-edge users as early and often as we can. For those a YEAR.N synchronized versioning scheme would not work.

The versioning conundrum

At that stage we had three options to handle those projects switching from one model to another. They could keep their 2015.2.0 version and start doing semantic versioning from that -- but that would be highly confusing, when you end up releasing 2017.9.4 sometimes in 2016. The second option was to reset the version for projects as they switch. So Ironic would adopt, say, 3.0.0 while all other projects still use 2015.2.0.

The third option was to bite the bullet and drop the YEAR.N versioning at the same time, for all the projects that were still using it. Switching them all to some arbitrary number (say, 12.0.0 since that would be the 12th OpenStack release) would create confusion as projects switching to intermediary releases would slowly drift from the pack (most projects publishing 13.0.0 while some would be at 12.5.2 and others at 13.1.0). So to avoid that confusion, projects would pick purposefully distinct version numbers based on their age.

The change

After discussions at the Vancouver Design Summit and on the mailing-list, we opted for the third option, with an initial number calculated from the number of past integrated releases already published.

It's a clean cut which will reduce on-going disruption. All components end up with a different, meaningful version number: there are no longer "normal" and "outliers" projects. Additionally, it solves the weird impression we had when we released 2014.2.2 stable versions sometimes in 2015.

As far as impact is concerned, distributions will need to make sure to insert an epoch so that package versions sort correctly in their package management systems. If your internal CI pipeline relies on sorting version numbers, it will likely need an adjustment too. For everyone else, it should not have an impact: when Liberty is out, you will upgrade to the liberty version of the components, as you always did.

Liberty-1 and the future

The change in versions was pushed last week, and that is why for liberty-1 we published for Nova, for Keystone, and for Designate, etc... Those are still on a milestone-based 6-month release cycle, but their "Liberty" final version won't be all versioned "2015.2.0", but 12.0.0 for Nova, 8.0.0 for Keystone, etc.

To reduce the confusion, the release management team will provide tooling and web pages to describe what each series means in terms of component version numbers (and the other way around).

We hope this future-proof change will bring some more freedom for OpenStack project teams to pick the release model that is the most interesting for them and their user base. For a cycle named "liberty", that sounded like a pretty good time to do it.