Software Psychology

…And then a Miracle Occurs by Bruce Nielson

There is an old Far Side comic where a professor is working complex math on a white board. At one point he’s written “and then a miracle occurs” and then successfully finishes his difficult problem.

I’ve talked to a lot of programmers that feel software actually tries to do just that. They complain that that no matter how many problems in the past we have, we just don’t learn our lessons. We always assume that just because our approach failed last time that this time it will be different.

This sentiment has even lead to another joke I often hear: what’s the definition of insanity? Doing the same thing you did last time but expecting a different result.

But Miracles are Real!

Both of these jokes underscore to me that we’re missing something. And, of course, it’s the psychology of software projects we’re leaving out. If we are continually doing the same ‘bad’ things over and over, there is probably a reason for it. If it really was all bad and we just kept repeating the same bad behavior merely expecting a different result, then we’d really be collectively insane.

So I’m going to start with a different assumption. I’m going to suggest that maybe we are not insane. The logical corollary to this is that the ‘bad’ results we hate are in fact better than the perceived alternatives.

I once had a business analyst friend come running up to me in a huff, concerned that once again his company, that was consulting with, was going to try to do the impossible. He listed out to me the very long list all the projects and the due dates. Then he said: “They’re doing it again! We failed the last time we tried to do this, why are they trying to force us to make impossible due dates this time? Do they think a miracle is going to occur this time?”

I laughed and he looked like he was going to smack me. Then I said, “Relax, Jeremy. A miracle will occur and you’ll be okay.”

Now he really looked like he was going to smack me. So I went on to explain:

Why We Accept Due Dates We Know We Can’t Make

Your managers undoubtedly have good reasons for why they want to promise dates that everyone knows we can’t make. They, in turn, get you and the team to – at a minimum – at least not publicly admit you can’t make it. In fact, there is generally good reason to believe you could make it if ‘this time we communicate well, work hard, and there are no mistakes.’ The real problem is that you will have miscommunications, someone will have something happen that messes up the work, and there will be mistakes. Making these dates are not impossible, just highly improbable.

But tell me honestly, Jeremy, what is going to happen when the dates are missed? Will everyone be fired? Of course not. Instead, everyone will start the play the blame game. This will cause the blame to redistribute around so that the managers can’t possibly find who to blame. The managers don’t want to admit that they don’t know what caused the failure, so they’ll find the best reasons for why things changed from back when they made the estimates and will argue that the slip of the date is for good reasons. The executives up the chain will have mostly have forgotten by then why they had to have it by that date and they’ll calmly accept the date slide based on the reason given.

If I’m wrong about this, and instead the executives decide to come down on everyone – and that does sometimes happen, though rarely – then a witch hunt will begin. Eventually, because specific blame is basically impossible to assign, someone will become the token person that failed. That person will be fired and everyone else will be forgiven since now we’ve decided that it’s primarily that person’s fault. Eventually, we’ll all come to believe that was actually the case.

Now we all know that most likely the schedule will slip and nothing will happen. But if we’re wrong, then the odds that we’ll be the token person this time are very slim. So it really does make sense to not do any major objecting over the unreasonable due dates and just wait and see what happens. Because odds are things will change and it won’t be a big deal.

Oh, and isn’t that what happened last time? And the time before? And the time before that? In other words, the reason we keep repeating the same pattern of accepting unreasonable due dates is because we perceive that miracles normally occur and we’re betting they will this time too. The miracle won’t be that we make the due date. The miracle will be that something changes so that we don’t have to or that someone who isn’t us gets in trouble for missing it.

Conclusion

Jeremy’s face lit up. “You’re right! I feel so much better!” he exclaimed. And off he went to go ahead and do his best and not worry so much about the unreasonable due date.

Now, I’d argue that there is a downside to this reality. For one, we are giving up on the fact that software can be more predictable than this. And also, we are giving up our chance to do real prioritization to make sure we are doing the high value items first. So I am not advocating for complacency on this issue. But it does help put it into perspective.

We are not being stupid when we repeat our “errors.” We do it because it makes sense to do so because the results are optimized (compared to the perceived or real alternatives) to maximize our chance of personal success (i.e. not lose our reputation or source of income.)

But there is one more important point here. Now that we understand the psychology of accepting (or at least not strongly objecting to) unreasonable due dates, we can now understand why grass root efforts to fix the problem are doomed to failure. If this isn’t how you wish to see your company run, it’s the managers and executives that have to decide to fix the problem top down.

Software is Not Coding, It’s A Cooperative Game of Communication by Bruce Nielson

In my post on “code is really design” post, I mentioned that I would further address the paradox that software is created in “thoughts units” but the most important people to the success or failure of a project are the sponsor and customer. In this post, I’ll explore that paradox further by discussing Alistair Cockburn’s idea of Software as a cooperative game.

In Agile Software Development, Cockburn says:

Software development is therefore a cooperative game of invention and communication. There is nothing in the game but people’s ideas and the communication of those ideas to their colleagues and to the computer.

In Crystal Clear: A Human-Powered Methodology for Small Teams, he writes:

[Successful teams] view developing software as an evolving conversation – an economic-cooperative game, actually. In any one move they go forward and create reminders for each other.

But in what sense is software a game and how does this help us understand software development better?

I’ve never been fully comfortable with Cockburn’s calling it a “game.” I’m not sure images of Monopoly and Halo (which is what comes to my mind, anyhow) is what he intended.

It’s helpful to refer to other non-software examples that Cockburn uses. One of my favorite of his examples is making laws in a legislature.

Can You Estimate How Long it Will Take to Pass a Law?

Imagine if the republicans asked the democrats, “How long will it take you guys to pass that new health care law of yours?”

Now in theory, the democrats should be able to estimate this. It’s just a bunch of words on a pieces of paper, right? Someone has to sit down to write it, they have to talk to experts, they have to run a few numbers and come up with a budget, then they have to pass it through a committee, then bring it to the floor, and finally get a vote. So the question the republicans are asking seems reasonable on the surface.

But in reality, part of what the democrats have to estimate is how long it will take them to convince enough republicans to pass a version of the bill. If this step were unnecessary, making an estimate would probably be easy. But if the democrats need to actually convince some key republicans to vote for the bill to get it passed, then part of their estimate will have to be an estimate for how long it takes to keep revising the bill until the republicans accept it – if ever.

Once we realize that is the case, the republican’s question seems less legitimate. The Republicans have a significant say in determining when the bill is “done” and the democrats don’t actually control that part of the process. The democrats cannot, even in principle, make an estimate for how long it will take to pass the law.

I see software much like this. We have customers and sponsors on the one hand and the development team (programmers, testers, designers, project managers) on the other hand. But instead of capturing ideas into specific laws to be interpreted by other humans, they are cooperating together to capture business rules into an automated programs that will be interpreted by computers.

There is nothing else going on when we make software. There is no analogy here to building a house or a wall, or anything else physical. When my customers ask me “how long will it take to program the fitz-bart module?” I am literally incapable of answering them because part of my estimate would have to be how long they will choose to iterate before they accept the software as matching their needs. There is literally no limit to how many times they might choose to keep tweaking the requirements and I’ve been in cases where they never become satisfied with it.

If I didn’t have to worry about customers having to accept the software, estimates would be a lot easier. But with customer acceptance required, it’s impossible to estimate anything beyond the first iteration of the creation of the module.

An Alternative – Cooperation

Now consider an alternative example. Lets say that both the republicans and democrats both are anxious to pass a health care bill and they both have more or less synchronous goals. Now is it possible to estimate how long it will take to pass a health care law?

Absolutely!

Why? Because now both sides are working together to meet a target date. This means both sides are motivated to make the date and, more to the point, both will be willing to accept some compromises to meet that date.

Software works the same way. If the customers/sponsors and the development team both want to make a certain release date or a specific budget target, there is almost always a way to come up with a workable (though usually manual, and thus less than ideal) solution to any software problem such that you can meet those goals. It just requires both sides to agree that they will make the target date or the target budget and to be willing to negotiate on scope until they match those goals.