Code is Design by Bruce Nielson
The Story So Far
Let’s return to the primary concern with Software. We’ve talked about how most software projects either fail altogether or run significantly over budget. Along with that thought, we considered the statistics that show that the development team itself plays little role in the success and failure of the project in comparison to users and sponsors. However, contrasting that we’ve looked at how software development is really developed in “thought units” more so then actual code.
These two facts seem on the surface to be at odds with each other. If software is really developed in “thought units” then why do users and sponsors have more control over the success and failure of the project then the developers?
Coding As Design
I will now explore this question further in a series of posts. But first, we need to look at the concepts of “software design” vs. “software coding”
Rumor has it that software is an engineering discipline and because of that, our industry adopted much of the vocabulary and process used by real engineers. This led to the creation of the both famous and infamous “waterfall methodology” where we pass through water tight stages that start with Requirement Gathering, move on to a documented Design, and then finally Construct what was designed, finishing off with Testing and then Release.
It’s all a bunch of posh, of course.
Actually, despite it’s recent (and partially deserved) bad reputation, the waterfall methodology has a lot to recommend it. To this day, I’ve never come across any methodology, Agile or otherwise, that doesn’t owe a great debt to the Waterfall methodology. So let’s not throw out the baby with the bath water.
However, what we software developers do is not really engineering in the true sense and probably never will be. This is why a strict waterfall approach, which serves so well in other engineering fields, has failed us miserably.
Jack W. Reeves classic article, “What is Software Design?” addresses this directly. (See also this link, for further information.) What Reeves correctly points out is that what we call “Software Design” is not a complete design at all, but is rather what he calls a “high level structural design” yet we treat it as if it’s some sort of complete detailed design (even often calling it “The Detailed Design”).
Now maybe you’re now thinking, “So it’s a high level design instead of a detailed design. So what, who cares?”
Yet, as I’ll show in future posts, this seemingly small distinction might just be the entire difference between understanding the real problems of software development and turning a blind eye to them.
But let’s let Reeves speak for himself. He says:
For almost 10 years I have felt that the software industry collectively misses a subtle point about the difference between developing a software design and what a software design really is. … This lesson is that programming is not about building software; programming is about designing software.
…the software industry has created some false parallels with hardware engineering while missing some perfectly valid parallels. In essence, I concluded that we are not software engineers because we do not realize what a software design really is
A program listing is a document that represents a software design. Compilers and linkers actually build [manufacture or construct] software designs.
…it is cheaper and simpler to just build the design and test it than to do anything else. We do not care how many builds [constructions or manufactures] we do—they cost next to nothing in terms of time.
[The] designs will be coded in some programming language and then validated and refined via the build/test cycle.
Is Programming a Form of Engineering at All?
Reeves questions if Programmers should be called Engineers if they don’t even realize that programming isn’t “manufacturing” or “construction” at all, but is really “detailed design.” But Reeves never quite questions if programming might not be Engineering at all.
But this leaves one uncomfortable. If our industry is full of programmers that do detail design and think they are doing “manufacturing” (i.e. “construction or building of a design”) how could they possibly misunderstand their own engineering discipline so severely?
Agile Maven, Alistair Cockburn, suggests how by questioning the very notion of “software engineering”:
Software development is not “naturally” a branch of engineering. It was proposed as being a branch of engineering in 1968 as a deliberate provocation intended to stir people to new action [Naur-Randell]. As a provocation, it succeeded. As a means for providing sound advice to busy practitioners, however, it cannot be considered a success. After 35 years of using this model, our field lacks a notable project success rate [Standish], we do not find a correlation between project success and use of tidy “engineering” development processes [Cockburn 2003a], and we do not find practitioners able to derive practical advice to pressing problems on live projects. (See link)
Elsewhere, Cockburn points out that the whole notion of “software engineering” was actually just an attempt to provoke less than obvious parallels between programming and true engineering fields. “The term ‘software engineering’ was coined in 1968, in the NATO Conference on Software Engineering,” Says Cockburn, “It is fascinating to read the verbatim reports of that conference and compare what those attendees were saying with what we have created since then.”
He then quotes from that conferences notes on the background of the conference:
The Study Group concentrated on possible actions which would merit an international, rather than a national effort. In particular it focused its attentions on the problems of software. In late 1967 the Study Group recommended the holding of a working conference on Software Engineering. The phrase ‘software engineering’ was deliberately chosen as being provocative, in implying the need for software manufacture to be based on the types of theoretical foundations and practical disciplines, that are traditional in the established branches of engineering.
It may well be time for us to abandon the very notion of “Software Engineering” and to base our development techniques off of some more natural match. Reeves agrees:
In many ways, software bears more resemblance to complex social or organic systems than to hardware.
Ramifications of Software As Design
Rethink, for a moment, how accepting “Coding” as really being “Design” changes the way we think of software development. The ramifications are substantial.
First of all, it means software development, being a design activity, will always change after the requirements gathering and high level design are “completed.” We would no more expect software to merely follow the “original design” (which we now know is only a high level design without all the needed details) then we’d expect Intel to design their next generation process only at a high level and then lock it in permanently. Any attempt to do so would be met with the same sort of frustration that plagues the software industry.
Another ramification is that it often makes sense to “get to coding” (i.e. detailed design) as quickly as reasonable. Contrary to popular belief, not all rushes to code are “hacking.”
Indeed, any methodology that encourages “design” (i.e. high level design) to be simultaneous with “coding” (i.e. detailed design) will be superior to one that doesn’t. This explains the popularity of methodologies such as “spiral,” “rapid prototyping,” and now “agile.”
Furthermore, testing software is actually a design activity as well. It’s how we validate the design details. Thus any methodology (a la Agile) that encourages testing during “coding” will be more successful than one that doesn’t.
Conclusion: It’s All Design
Wait! Did you read that right? Is it really true that design, construction, and testing phases are all really just design?
As Reeves so aptly put it:
The overwhelming problem with software development is that everything is part of the design process. Coding is design, testing and debugging are part of design, and what we typically call software design is still part of design. Software may be cheap to build, but it is incredibly expensive to design.