Guido van Rossum talks about Python 3

Yesterday Guido van Rossum came to Berkeley to talk about Python and science.

Listening to him, I understood for the first time the importance of porting to Python 3K.

Open-source is not the same as commercial software. If Microsoft puts out Vista, and we don't see the need to change, we just don't buy it. It's not the same for Python 3K.

For Python, we benefit from the enthusiasm and - yes - enjoyment - of the Python core team. We want them to enjoy what they do, we want to share their enjoyment of something well-done, well thought out. That's how good work gets done, and it's the energy of a project.

Python 3K is important, because it's important to the Python core team. It may not yet be obvious that we need it, but we should port, because it's important for Python.

The five dysfunctions of a team

The five dysfunctions of a team is a book by Patrick Lencioni. There are no data to justify its conclusions - but it's a convincing and enjoyable read. The five dysfunctions are:

1. Absence of trust
2. Fear of Conflict
3. Lack of Commitment
4. Avoidance of Accountability
5. Inattention to Results

See the five dysfunctions summary. The numbering goes in the order these problems must be solved (trust first).

The solutions for the team leader to try are:

1. Absence of trust -> Go first!
   
2. Fear of Conflict -> Mine for conflict
   
3. Lack of Commitment -> Force clarity and closure
   
4. Avoidance of Accountability -> Confront difficult issues
   
5. Inattention to Results -> Focus on collective outcomes

See the five dysfunctions model.

An explicit society needs a constitution

We hold these truths to be self-evident...

There is something mysterious about a healthy society, in open source as elsewhere. From outside, it looks as if it just happened that way, particularly in open-source. It may be that some projects and societies are lucky with their people and the cultures they make. Maybe NIPY is such a project. But, our job is to defend our society, and our project, from decaying.

To do this, we need to understand the sources of decay, and that is what my previous posts were trying to describe. Without this understanding, we can make no sensible defense.

In the case of NIPY, we started with a naive model (one person in charge). For many reasons, that model has not worked well. We then need to make another model that will work better. To do this, we summarize the state we want to avoid, and then we regulate.

Law plays a fundamental role in making the rules of the society clear, and making agreement to those rules explicit. It provides a defense against arbitrary decisions, bullying and abuses of power; it reduces anxiety, and with it the motivation to seek power. It draws people together who have a shared vision of how society works, and what it looks like when it works well.

We need a constitution. What should go in the constitution? It should be set of principles that we consider important as a society, and the rules and decision making process that will hold us to these principles.

NIPY as an explicit society

When I started writing this blog I didn't know what I would write, but I found myself writing about politics.

I'm interested in politics for many reasons, but there have certainly been politics in the distributed society of NIPY.

I notice, that, one strong opinion that comes up, expressed in various ways is:

• We should not discuss politics in public
  

I suppose the reasons that might be given for this are:

1. it looks bad to outsiders
   
2. it's irrelevant to the problem we have
   

It is dangerous not to discuss politics. Open discussion is a defense against many forms of political game. When there are power games in play, the essential mechanism for keeping them in play, is to prevent them from being discussed. I leave the proof as an exercise for the reader.

The strong pressure not to discuss politics usually makes it impossible to discuss the fact that we are not discussing politics. Expressed with more poetry, the language of irrationality is silence.

In the spirit of free discussion then:

• it looks bad to outsiders

This is an interesting point of view. There are two branches to go down, depending on whether the proposer agrees that there is a political problem. Imagine the proposer does think that the society suffers from political divisions and that this is a problem. Then the assertion that 'it looks bad to outsiders' is an expression of the hope that the outsiders will not notice the problem, and that the problem will go away if we don't talk about it. It contains an implicit judgment that outsiders will be alarmed by, and intolerant of, open discussion of politics. I am reminded of the famous parable of the grand inquisitor. The alternative point of view is that good citizens are drawn to honest and open discussion.

Taking the second branch means that there is in fact no political problem, and, by implication, the person who thinks there is a problem, is either oddly mistaken or possibly up to no good. This soon leads to the classic 'no man, no problem' fallacy.

• it's irrelevant to the problem we have

This leads back to the first objection ('I don't want to discuss it'). Of course, it might be irrelevant to the problem we have, but how will we know, if we don't discuss it? Back then, to Python, because we know that:

• explicit is better than implicit

in the realm of society as for code.

Next - navigating towards a healthy society.

Politics and power

I think there are two modes in the way that we react to other people in organizations:

• trust
• power

Trust: the state we reach when we assume that our colleagues are competent and want the project to succeed. We may work independently, but we know that, when we need something to help our work from the others, they will listen and help. We wish that they feel valued in their jobs, and they wish the same for us.

Power: the other members of our team may not want what we want. Our job is to make it more likely that we get what we want. Others will surely do the same; the winner is the person who does it with the most success. Success comes from having powerful people agree with you, so that you can make less powerful people do what you want.

An interest in power comes from anxiety. This in turn comes from a potentially unlimited threat from a poorly regulated society.

The two do not mix; that is, a significant part of the team working for power will destroy trust. Increase in trust makes the desire for power irrelevant or ugly.

The clearest example I can think of is the analysis of companies that become successful in Good to Great by Jim Collins. Companies that shift from mediocrity to prolonged success differed markedly from comparison companies in the character of their leaders. The book describes the successful leaders as being 'level 5' and elsewhere 'servant-leaders'. These leaders seemed to have an unusual lack of interest in personal power. From the Wikipedia article on servant leadership :

The highest type of ruler is one of whose existence the people are barely aware. Next comes one whom they love and praise. Next comes one whom they fear. Next comes one whom they despise and defy. When you are lacking in faith, Others will be unfaithful to you. The Sage is self-effacing and scanty of words. When his task is accomplished and things have been completed, All the people say, ‘We ourselves have achieved it!’

From Lao Tzu, Tao Teh Ching, trans. John C. H. Wu (Boston, Massachusetts: Shambhala, 2006), 35.

Open-source as a self-regulating society

Maybe community open-source projects have social advantages over projects developed by organized work teams.

• it is usually obvious who is charge because most projects start as an individual effort
  
• being a member of the team depends very much on how much work you do
• because of the above, the person in charge has usually done a lot of the practical work
• the team members rarely meet so that working relationships are almost exclusively about work, and power games are hard to sustain.
  

I like Joel's analysis of why programmers don't like politics - half way down his field guide to developers.

Meditating impatience

Is impatience a bad thing?

"I need to be impatient with myself to get work done".

I think this convincing statement is false. Consider that, the times that we have been particularly impatient, have been times when we have found it very hard to work. Without thinking more, we might assume that we are very impatient because we have so much to do. Then roll it round the other way, and ask, what was the cause of all the time we wasted when we could have been working? Think of the times we have had, that were most productive, and worked best, and think of the difference.

In practice, our habits of mind are so overwhelming, that these thoughts are difficult to have.

It may be easier to say this:

• impatience is a work-style that you may prefer for yourself
• impatience is destructive of teamwork, and we should make every effort to make sure our own impatience does not affect those we work with

How do we recognize the effects of impatience?

• lack of productivity
• blame

This is so, because impatience is the symptom of blame of self. It is natural that impatience in teams leads to blame of others. Blame differs from constructive criticism in that it:

• is most clearly expressed in private to people other than the target of the criticism
• concentrates on the working or personal characteristics of the people responsible for the problem, rather than the specifics of the problem
• is self-sustaining, because blame leads to poor working relationships and low productivity
  
• tends to come from the people doing the least practical work on the project
  
• is difficult to express coherently in writing (because the content is more emotional than the originators want to believe).

The expression of blame has a characteristic tone. It sounds like a pompous teenager impersonating an adult. There is a good example in this absurd email that started the famous split in the NetBSD project.

The three great sins of a open-source programmer

The engine of open-source is enjoyment of work well done. As more people join the project, the enjoyment increases, because we share the pleasure of work well done. If we are lucky, then our society is a happy and ordered place for new members. The written and unwritten rules seem sensible and straightforward. The group members are respectful, helpful and generous. Enthusiastic but ignorant new-comers are treated with indulgent warmth, and irritable or ungenerous emails are dealt with firmly but politely.

I suspect that is the natural history of a successful project.

What happens though, when our society is not yet like that? What do we do? How do our societies deteriorate and why?

I suspect it is because the team has not yet learned to deal with the three great sins of a open-source programmer.

• laziness
• impatience
• hubris

Maybe you have read something like this before: LazinessImpatienceHubris

Laziness is the what happens when we know that someone has worked on this problem before, but we lack the energy to look at what they have done, and we start again. We deprive our fellow programmer of the nectar of shared work Laziness is the dark side of the desire for something efficient and well-made.

Impatience is the output from our distress because of laziness. Impatience is the name of thing that causes problems on mailing lists. You want to use someone's software, but you have not got time to contribute. The software does not work the way you want and you send an irritable email. The small attack makes the society feel just that little bit more tired and less enthusiastic to help you, or each other. Impatience is what we feel when we realize that our problem is hard, but we lack the energy or hope to solve it. Impatience is the atmosphere of a failed desire for productivity.

Hubris is to confidence as vanity is to courage. Hubris is when we get so worried by the possibility of failure, that we start making up a whole new fantasy world where we are succeeding. Of course this happens often, but when we see it, in ourselves or others, we know that there is a large problem, and, unless we are brave, we walk away.

Luckily, there seem to be many brave people in our open-source world. Luckily, it is possible to return to efficiency, productivity, and courage.

What to expect?

I started this blog without being sure what would go in it. I expect that I will update it rarely with thoughts about the problems that we have, and the solutions we have tried.

NIPY is a rather ambitious project to make a new library for functional brain imaging analysis using Python. It's a collaboration between Stanford, Berkeley, MIT and Neurospin (in Paris).