Deduction happens when we combine theory and observation to make a prediction. If the theory is “everyone who pays to attend a conference will arrive at the venue” and we observe that Patrick has paid to attend a conference, then we – necessarily – conclude that Patrick will arrive at the venue.

Deduction can never generate any new knowledge, because it all it does is apply general principles or abstractions to concrete cases. It may give surprising or counter-intuitive results, but they are not, strictly speaking, new knowledge – just new applications of existing knowledge.

What we can do with deduction is destroy knowledge: at the date of the conference we may note that Patrick never shows up. This forces us to discard our hypothesis that everyone who pays will come. Hypothesis testing is a deductive process.

Induction happens when we draw conclusions from statistical arguments, rather than cold, hard logic. If Patrick doesn’t show up at the conference he paid to attend, we draw the conclusion that Patrick got sick, because this is the most common explanation for people not showing up when they have paid to attend.

Induction is an uncertain process, but it does create new knowledge. We now have a hypothesis we didn’t have before: Patrick is sick.

Like induction, abduction is an uncertain process that creates knowledge.

Abduction happens when we draw conclusions from what we think are the best explanations for observations, rather than what is statistically most likely. For example, if we know that Patrick is incredibly afraid of flying, then we might draw the conclusion that Patrick was overcome by fear on the day he was supposed to fly to the conference, rather than got sick.

Finding the best explanation for something is a somewhat subjective judgment, and may require balancing properties like parsimony (Occam’s razor) and generality.

The main difference between induction and abduction is that induction makes references to statistics when creating knowledge, whereas abduction makes references to causal mechanisms. The two often overlap: if Emmy has lung cancer, the best causal explanation is smoking – but that is also the best statistical explanation. This is unsurprising, since causal mechanisms tend to stand out also statistically. As Reichenbach taught us: No correlation without causation.

Unfortunately, the effect is that people often confuse induction with abduction. Theory-building, in the sense of creating general abstractions to explain specific observations, is an abductive process. It is sensible to draw clues from statistical properties, but in the end, we are looking for a causal mechanism, meaning we need to abduct.

What does this have to do with designing abstractions? In the example with Patrick and the conference, we created knowledge about a specific case (Patrick) from general principles (fear of flying.) We can also go the other way around, abstracting from specific cases to a more general explanation.

This requires first observing some pattern. One pattern that often shows up in code is that of iterating through an array. Or a list. Or a set. Or a dictionary’s keys. Or the lines of a file, and so on. We can start to guess that the underlying principle is something that is capable of generating one element at a time from something else – an enumerator2² Or generator, or iterator, or whatever you want to call it! Of course, the enumerator is just one of the abstractions we could make from the pattern. Another possible abstraction based on the same concrete cases is “everything is a file” – and some people have pursued that abstraction with great success, too.

In scientific terms, we can say that we have seen two competing theories of iteration: the enumerator and “everything is a file”. We may find an edge case that falsifies either of these theories but not the other3³ Through deduction – trying to apply the theory to a specific case and running into some sort of contradiction with our experience., in which case we know which theory is more general. Or we find that one edge case falsifies one of the theories, and another edge case falsifies another of the theories, and then either our two theories are both insufficient, or the theories actually different, but only seem similar because we came up with both in response to the same set of specific cases. The best explanation can be abducted from the specific cases that invalidate our theories.

This is why theory-building thrives on finding deviant cases that aren’t fully explained by the existing theories: they allow us to advance our theories to explain things more fully.

This framework for thinking about abstraction design is also another link between software development and knowledge-building.