The Law of Lossy Requirements by Bruce Nielson

Lossy Compression

In computer science, compression is an indispensible tool. Anyone familiar with .zip files knows what I mean. Interestingly, there are two kids of compression, lossless and lossy. Lossless compression is like .zip compression, you put a file of, say, 100kb in and the end result is a file of, say, 50k. But when you reverse the process, you get back your original 100kb file.

Lossless compression relies on the fact that in real life information contains patterns. Just imagine taking this post and finding the most common used words in it. Then replace the most common word with a the tag “<1>” and the next most common with “<2>”, etc. The end result would generally be a smaller file, yet by replacing “<x>” with the original words you could recover the original file in its entirety.

I used to wonder what possible use lossy compression could ever be. Why in the world would I ever want to save off a compressed version of my files if I couldn’t get them back to their original state ever again?

Of course for files, lossy compression is useless but for computer graphics it’s quite useful. Because the human eye is not capable of picking up all the details in an image it’s possible to reduce the details in the image without the human eye being able to tell the difference.

What’s even more interesting is that you can continue to increase that lossy compression on an image until the human eye can tell the difference, yet the brain will still be able to tell what the image is. It just looks a bit more blurry.

In fact, you can continue to compress and compress using lossy compression to any size. At some point you end up with a single solid color, of course, which isn’t useful, but even if you take a very large image – let’s say 1024 x 1024 pixels – and shrink it via lossy compression down to the equivalent of a 16 x 16 image, the human brain fills in the details and still makes sense of the image.

Requirements Are Lossy Compression

If Code is Design, one of the most under appreciated points of software development is that requirements are equivalent to lossy compression of graphics, only in reverse. (See also Tell Me What To Do, Not How to Do It for an example of this.) 

What we call “requirements documents” (or even “design documents”) are really a blurry picture of what is to be built. In other words, requirements and design documents by definition do not contain all the details.

Why do we do this? There are actually many reasons, but here are two key reasons:

1. Until the software is actually built, it doesn’t exist. So there isn’t really a choice but to imagine it with broad brush strokes and then refine it into reality.

2. As was discussed in a previous post, human beings more easily understand abstractions rather than precision. If we didn’t have abstract and blurry views of the design, no one but the programmer would ever understand what is to be done. Actually, even the programmer would only understand it in chunks and not holistically.

The Law of Lossy Requirements

If you followed my argument so far, then The Law of Lossy Requirements will now make sense to you. This law states:

All requirements documents are a lossy compression of the business rules. If you made it non-lossy, the cheapest way to document it would be in code.

Therefore there are two corollaries to this law:

Corollary 1: There will never be a point where you know all details of what software is to be created until the software is completed.

Corollary 2: There is no such thing as documenting everything in the software because to do so would require synchronizing essentially a second copy of the code.