Try mentally taking apart a car. Can you point to the component that supplies the transportation functionality?

Of course not. Transportation emerges from the interactions between air, fuel, cylinder, drive shaft, wheels, etc.1¹ This is one of my favourite observations about product design: the air just over the road surface is a critical part of an explosion-engined car’s operation. You won’t find that on any blueprint, but it is an assumption built into the entire thing. When we take apart well-designed products, we rarely find their primary functions installed as components. Instead, their functions emerge in the interaction between components, and between the product and its environment.2² A wristwatch does not have a time-telling component – that happens when the human interprets the pointers on the dial.

As Dijkstra noted3³ Hierarchical Ordering of Sequential Processes; Dijkstra; Acta Informatica; 1971, our computers are the same. The cpu sees a monotonous sequence of instructions which have no inherent meaning on their own. Our ram is filled with a homogeneous sequence of bits which have no meaning as far as the memory controller is concerned. Any meaning is created when a higher-level process interprets the sequence in a particular way – often depending on other parts of the memory itself.

Meaning emerges from the interactions of meaningless details.

This ignorance is what makes the computer such a flexible tool. It’s not built to support any specific feature, it’s built in a way that supplies the primitives for supporting many possible features, whichever are desired by higher-level code.

Hold on to this idea, and we’ll look at how people commonly design software.

When given a specification for what a program should do, a common instinct among many programmers is to think of a recipe that would produce the desired outcome, and then formalise this as a sequence of steps, conditionals, loops, and subprogram calls, and then write the code for those.4⁴ At first I wanted to say this is typical of beginners, but to be fair, I have seen very experienced (not to say expensive!) consultants do the same thing. We will call this a flowchart based design because we could draw the recipe as a flowchart, even though few people do that these days.

What are the attractions of flowchart based designs? There are at least three, and they overlap:

• The recipe is how we as humans would accomplish the task, and a good introduction to programming is to translate the manual process quite literally into instructions the computer can perform instead. We teach this in introductory programming classes.5⁵ Consultants can also more easily sell this type of design to non-technical management because “it’s just like cooking!”
• The first version of the program is fairly easy to write following this approach – it looks like quick progress.
• For the reason above, this approach works really well as long as the total program size is not significantly greater than 1000 lines of code. On that scale, we can just throw out all the code and start over when the requirements change, so it doesn’t matter much how we built it.

If we go back to the design space tree from the previous article, we’ll note that a flowchart based design both makes decisions in the wrong order, and locks in design decisions that are likely to change later.

Even if we suggest6⁶ As the aforementioned expensive consultants did. that each node in the flowchart is a module that can have its implementation swapped out, we still have the problem of rigid interfaces between the modules, and this prevents easily changing how the flowchart nodes are wired up – a likely change request.

A flowchart based design uses modules to hide implementations, rather than design decisions, which often span multiple flowchart nodes.7⁷ This is the essence of why I dislike the mvc pattern also. It draws boundaries according to technical concerns rather than design decisions. If you want to have all your views together, by all means run find . | grep View, but it’s no way to organise the functionality of a product. If we use modules to hide design decisions instead, we will often get hidden implementations as a byproduct, but we are focusing on the right thing.

This ties into the larger problem with the flowchart based approach: it starts from the outcome we want. As sensible as that sounds, the outcome we want is usually not as well defined as we think. The desired outcome is going to change as the development progresses.8⁸ Not to mention that we have the ability to change what the desired outcome is, by talking to the users and finding better ways to solve their problems.

We learned in the previous article that our early decisions should be about things that are not going to change. If there’s one thing we know will change, it’s user requirements – and this, generally speaking – is a good thing! So let’s not base our design on the user requirements, or other things that change.

David Parnas9⁹ On the Criteria To Be Used in Decomposing Systems into Modules; Parnas; Communications of the acm; 1972.,10¹⁰ Information Distribution Aspects of Design Methodology; Parnas; Air Force Office of Scientific Research; 1971.,11¹¹ Designing Software For Ease of Extension and Contraction; Parnas; ieee Transactions on Software Engineering; 1979. recommends something he calls inside-out design.

Whereas flowchart based designs tend to be outside-in, i.e. start with what is expected and then iteratively refine the implementations, inside-out design goes the other way around. When designing inside-out, we start from a known and general core and gradually increase convenience while removing generality until we have components that can be connected together so a solution satisfying the user requirements emerge.

This approach is also known as a vm-based design, because we think of it as starting from pure machine instructions, and then we gradually construct more and more convenient outer shells on top of it. We gradually add vocabulary to the instruction set until the solution to the original problem is easy to express.

This may sound like bottom-up design, but Parnas is careful not to call it that. Sometimes inside-out design is top-down, sometimes it is bottom-up. It simply depends on whether the high-level or low-level parts constitute the known and general core. For example, in embedded applications, the fixed core may be high-level operations we want to perform, whereas the outer shell are the specifics of the interface used to speak to the rest of the system, which may be in flux. This means the design is still inside-out, but it’s also top-down rather than bottom-up.

In other words, to design for a broad range of outcomes (a large program family), we start with the core mechanisms that will be common to all programs in the family. We leave the outermost parts – the user requirements – undecided for as long as possible. This lets us do what Allen Ward12¹² Lean Product and Process Development; Ward & Sobek; Lean Enterprises Institute; 2014. suggests and aim for ambitious but flexible targets. I’ve long been puzzled by how that is possible, but the simple trick is to not fix the shape of the product early.

The inside-out approach also aids useful modularisation, because it’s easier to take a useful subset or superset of functionality when looking at somewhat general vm instructions than it is to subset a rigidly connected flowchart. When we build components based on fundamental principles rather than features, we make it easier to change how we wire together components later, and let new features emerge.

Even the most junior programmer immediately gets handed a very dangerous tool: the power to create abstractions. Wielded correctly, this power can make otherwise impossible things possible. Used incorrectly, it can turn conceptually simple things into endless mud.

I want us all to move closer to making the impossible possible, and further from mud. David Parnas lights the way.

I recently wrote code to simulate a card game. The details are irrelevant, but it was a great opportunity to put inside-out design into practise, because when I started programming, I didn’t actually know the rules of the game, nor what all the card types were. Thus, I started from a very general core that I did know with certainty:

Card
  - name      : string

  - valid_play : Card -> bool

A card has a name and it gets to decide on its own whether it’s legal to play this card on top of another card. Naturally, we also have

Player
  - name       : string
  - hand       : List Card

  - add        : Card -> ()
  - remove     : Card -> Maybe ()

A player has a name and a few cards on their hand. If we give the player a card, we get a new player with more cards. If we try to remove a card from a player, we either get nothing (if the player didn’t have that card) or a new player with one fewer cards. Simple, but extremely general.

Then we have

Table
  - pile       : List Card
  - previous   : Card

  - draw       : Player -> Maybe ()
  - play       : Player -> Card -> Maybe ()

In other words, the table has a pile of cards that are not yet in play, and there’s the previously played card. The table lets a player draw a card from the pile, or play a card from the player’s hand. Again, these do minimal validation.13¹³ The draw method ensures there are still cards in the pile, and the play method obviously fails if the player does not have the card that they are attempting to play.

Then to start to impose some structure on this, we can create

Ply
  - table      : Table
  - player     : Player

  - draw       : () -> Maybe ()
  - play       : Card -> Maybe ()
  - end_ply    : () -> Maybe ()

A ply is game-speak for the portion of a turn in which one player is allowed to act. In some games, the player may perform multiple actions during their ply, which is why it is a separate action to end_ply. This may only be permitted under certain circumstances, e.g. there may be a requirement to play a card before ending one’s ply.

This abstraction also starts to control the sequences of actions that are allowed within a ply. For example, if a player draws a card they may be forced to play that specific card rather than any other they have in their hand. This can be enforced by the Ply by returning nothing when a player attempts to play any other card.

The Ply module can do whatever bookkeeping it wants internally to produce these rules. We don’t really care, because it presents a highly general interface to the rest of the application. What’s neat is that by swapping out the implementation of Ply for something else, we can create widely different games, using the same basic building blocks.14¹⁴ However! The way I have designed this, all types of actions permitted by the game need to be exposed as methods by the Ply module’s interface. I was able to make that assumption because I could fairly quickly enumerate all the actions that would ever be supported under any set of rules for the game I was studying, but if that’s not the case for you, you may need to find a way to modularise the set of allowed actions also.

Going further, there is usually a ring of players around a table.

Ring
  - table      : Table
  - players    : List Player

  - draw       : Player -> Maybe ()
  - play       : Player -> Card -> Maybe ()
  - end_ply    : Player -> Maybe ()

This abstraction has the responsibility of managing the order in which plies are created, i.e. in which order the players act. If a player acts out-of-turn, this is where that would be detected as a mismatch between acting player and the player whose ply it currently is.

Here we have a clear illustration of an abstraction that reduces the power of an underlying abstraction. By re-exporting the actions from the Ply in the Ring, we can force players to go through the Ring to act, which means we can enforce more rules on them, which means we provide even more structure to the game.

At this point, we could practically write code that plays a game by creating a few players, a table, passing it all into the ring and then calling methods on the ring to find what the legal moves are and perform them.

We have hidden the decisions around what the actual rules of the game are inside the Ply and Ring abstractions, so if we learn of a new rule we can stick that in there and get a slightly different game. We don’t have to change anything else, because nothing else can depend on the specific ruleset we hade in mind when making the design.

If we learn of a new card type, we can instantiate that type of card and all the code we’ve seen so far continues to work. Nothing in this design depends on the specific cards we had in mind when writing the code.15¹⁵ Aside from the caveat from before: if we discover a card type that can do something other than being drawn and possibly played, i.e. a card that has some other effect, we need to expose that effect as a method on the card. Since we haven’t designed with that in mind, that would be a more invasive change.