Tuesday, March 27, 2012

Fresh fork

The ZeroMQ project just got ripely forked [fork]

I find forks interesting because I'm interested in the way that open-source projects break down.

Martin Sustrik appears to be the lead author and architect of ZeroMQ, at least up until March 2010 [buyout, switch].  He apparently developed ZeroMQ as part of the open-source company FastMQ.   Around October 2009, a Belgian company called iMatix bought FastMQ [buyout] and switched from its own messaging protocols to ZeroMQ [switch]. Pieter Hintjens was and still is the CEO of iMatix.

From around 2010, Martin Lucina was "co-maintainer" of ZeroMQ with Martin Sustrik [cv]. He and Sustrik both live in Bratislava [beer-mail]:
Also, a lot of discussion between Martin Sustrik and myself got done in person, simply because we live and work in the same city (Bratislava, Slovakia).
Lucina and Sustrik wrote an article about ZeroMQ in LWN [lwn].

There were many discussions of process from at least August 2010 [workflow-proposal, releases, ego-per-git, first-fork, process1].

In at least one of these, Hintjens complains about Lucina's "ego", but asserts that ego is a major or the major motivation for work on ZeroMQ [first-fork]:
I'll just point out that your own ego is dominant and not always
pleasantly for others. I vaguely recall an original proposal for
multiple release branches that was quashed without sympathy (though
now you are arguing for exactly that). However without egos, and the
desire to dominate the work we do, this community would not exist, so
let's embrace that rather than deny it.
In the same email he loses patience with one of Lucina's objections, points out that iMatix owns ZeroMQ, and suggests Lucina forks the code if he is unhappy [first-fork]:
Shrug. You are free to fork any 0MQ repository and make your own
versions and releases. It is LGPL licensed. The packages distributed
from the zeromq.org site, which iMatix owns, and labelled ZeroMQ, a
name that iMatix also owns, ultimately fall under iMatix's purvey. If
you feel iMatix has been a bad host to the 0MQ community, feel free to
fork. This is an essential freedom, don't hesitate if you think it's
Luchina objected to proposals requiring pull requests on github, on the basis that it would lead to vendor lock-in [lucina-lockin], with some sympathy from Sustrik [sustrik-lockin].

Hintjens wrote a tradmark policy for the ZeroMQ name in May 2011 [tm-email, tm].  A long thread developed from the initial announcement [tm-email]

In December 2011, there was a private email conversation between Hintjens,  Sustrik and Lucina [metadict].  During this conversation, Lucina says that Sustrik "resigned as the benevolent dictator".  Lucina [lagree] and Sustrik [sagree] agreed to release the contents of this private email conversation, but Hintjens did not [hdisagree].

Around January 2012 Hintjens proposed a radically open model of development [policy]. He had the view that "Maintainers are not developers and they have no opinion on patches". Thus maintainers should push all or nearly all patches after confirming basic process [policy].

In January 2012, Lucina wrote to the ZeroMQ mailing list [wtf] pointing out some of the more radical aspects of Hintjens' proposal. He gave the opinion that this way of working was already causing a decline in code quality and finished with an appeal:
Thoughts? Most of us here are software engineers by trade, surely you can see where this is leading.
In an email on 4 February 2012, Lucina refers to a conversation with Hintjens the previous day in which Hintjens apparently asserted his leadership of the ZeroMQ community [metadict]. Lucina again appealed for support from other members of the community [metadict]:
If you care about this, the only way [4] to achieve change today is to be
vocal, and to criticise and argue those points of the process which you
care about. Having known Pieter personally for more than 10 years now, and spent many many hours arguing with him, I wish you luck!
This is footnote 4 from the same email:
[4] The other option is a fork (in the formal sense, not the Github sense). This is an option of *last resort*, and is not something to be taken lightly. However, it does put you on a fair playing ground; build your own community and process, and ultimately users will follow the leader with their feet.
Later in the same thread Hintjens withdrew from further discussion with [hdisagree]:
I'm retiring from this thread now and will by necessity ignore any
further discussion that isn't forward-focused and constructive.
Sustrik announced the release of Crossroads I/0 on March 15 2012 [fork]:
Crossroads I/O is a fork of the ZeroMQ project.
While we acknowledge forking can be a painful process, we felt the
ZeroMQ trademark policy to be overly restrictive.

Furthermore, the ZeroMQ community has also recently chosen to institute a light review process, which we feel is at odds with the technical quality and long-term goals we desire for the project.
The Crossroads IO website gives some explanation of the trademark dispute:
To grow [a thriving commercial] ecosystem the project must be fully vendor neutral, and implement a liberal (e.g. Linux-style) trademark policy allowing use of the trademark for third party distributions of the software, as well as for plug-ins and extensions.
The development policy for the new project is hardline Linux kernel style : all changes must be submitted to the mailing list as a patch [harddev].  They do not accept pull requests because [nopull]:
Pull requests can change while being reviewed. This makes it impossible to ensure that the code being merged is the same code that has been reviewed and discussed, which compromises integrity of the codebase.

Pull requests are meant for delegation of work to sub-maintainers and require an established web of trust. We may consider moving to this model in future.
There are two interesting questions.
  1. Was there anything that the community could have done to avoid this fork?
  2. Which project will succeed?

[buyout] http://lists.zeromq.org/pipermail/zeromq-dev/2009-November/001362.html
[switch] http://lists.openamq.org/pipermail/openamq-dev/2010-March/001598.html
[beer-mail]  http://lists.zeromq.org/pipermail/zeromq-dev/2012-February/015768.html
[policy] http://www.zeromq.org/docs:contributing
[cv] http://lucina.net/cv:lucina
[lwn] http://lwn.net/Articles/370307/
[workflow-proposal] http://lists.zeromq.org/pipermail/zeromq-dev/2010-August/004963.html
[releases] http://lists.zeromq.org/pipermail/zeromq-dev/2011-February/009220.html
[sustrik-lockin] http://lists.zeromq.org/pipermail/zeromq-dev/2011-February/009245.html
[ego-per-git] http://lists.zeromq.org/pipermail/zeromq-dev/2011-March/009988.html
[first-fork] http://lists.zeromq.org/pipermail/zeromq-dev/2011-March/009996.html
[process1] http://lists.zeromq.org/pipermail/zeromq-dev/2011-October/013986.html
[lucina-lockin] http://lists.zeromq.org/pipermail/zeromq-dev/2011-October/014010.html
[wtf] http://lists.zeromq.org/pipermail/zeromq-dev/2012-January/015524.html
[metadict] http://lists.zeromq.org/pipermail/zeromq-dev/2012-February/015744.html
[lagree] http://lists.zeromq.org/pipermail/zeromq-dev/2012-February/015767.html
[hdisagree] http://lists.zeromq.org/pipermail/zeromq-dev/2012-February/015774.html
[sagree] http://lists.zeromq.org/pipermail/zeromq-dev/2012-February/015790.html
[tm] http://www.zeromq.org/docs:trademark-policy
[tm-email] http://lists.zeromq.org/pipermail/zeromq-dev/2011-May/011141.html
[fork] http://lists.zeromq.org/pipermail/zeromq-dev/2012-March/016429.html
[harddev] http://www.crossroads.io/dev:start
[nopull] http://www.crossroads.io/faq