The most common mistake we make as Systems Thinkers.

Author: Rodolfo Hansen, Philips Software Competency Lead

At the core of the developer experience, we find the models we use to understand our code, the world, and we interact with it. These models, be they design patterns, hexagon, or micro-service architectures, have an almost visceral impact on how we go about getting things done, how we provide value to our users, and how we appreciate and evolve the codebase.​

How cumbersome, or unnecessary a given model or architecture is, turns out to be a key part of the developer experience. Personally, I find it painful to work on systems with too many layers, or in situations where I find myself having to sift through hundreds of lines of boilerplate code. But at the same time, it seems that my own systems tend to grow into these models that have some artificial complexity or ‘leaky abstractions’.​

​Given how the models we choose are so important to the developer experience, and how we deliver value and maintain our products, then we should at least have a few mechanisms for evaluating the quality of these models; are they effective at abstracting away needless details, yet powerful enough to express the concrete concerns of our customers: do they encode well problem domain?

Category theory has a fundamental result which enlightens how we deal with our models and their expressive power. Specifically, we now know that the syntax and semantics for a given language form an adjunction. That is to say: as we attempt to express more details or functionality in our models, we necessarily lose breadth of applicability. Conversely, the broader and more encompassing (abstract) our models are, the less they are able to house the details and nuances the model is supposed to represent.

Think of the detail with which you describe the chairs in a given furniture store. If we specify that a set of chairs are reclinable or brown, we can distinguish more details but we are narrowing down to a smaller fraction of the chairs that are at the furniture store. Additionally, if we pick “chairs” and forgo their color, we expand our selection to the things we are expected to sit on, but not necessarily all the furniture we would wish to buy for our home.

This tension is what gives rise to the Reification Fallacy, which happens when we encode an abstraction in our code that ends up deteriorating the developer experience, increases coupling and generally making the architecture more brittle towards changes. It ultimately lowers the quality of our code, reducing developer productivity as well as our ability to deliver.

Let’s look at three ways the abstractions we’ve chosen can fail:

• Leaky abstractions
• Sparse abstractions
• Useless abstractions

Leaky abstractions

Leaky Abstractions (source: Dall-E OpenAI)

Let’s explore a hypothetical codebase for consuming and producing goods in a town:

We have come up with the following abstraction: a common Producer interface which, given a set of parameters will produce a Good.

Producer façade

We have defined a clean interface / façade that brings together these four producers: PancakeMaker, PizzaMaker, TelephoneMaker, and ComputerMaker. Now each consumer will somehow pick up their goods list and work things out from there.​

This abstraction, although quite clear and useful in the graph above, could very well not match our project’s actual reality. In reality we might need to know what each good is; and from where it is coming from. For example, we may need to store pizzas in a freezer and ensure phones are in their own safety boxes.

Abstractions leak when users need to peek inside them

A leaky abstraction is one that forces us to continuously disregard it and look at a more concrete version of what the abstraction was meant to represent. Think every time you have to write: ​

if (x == null)

Or

if (x isIntanceOf ConcreteType)​

Or ​

if (x.hasCapabilityY)​

Every time our code gets littered with these concretizations, it means that whatever our value x meant to be an abstraction of; is leaking some capability, specific concrete representation or even its validity, and we are being forced to defensively confirm each of these cases. ​

Here is another example where a JUrl abstraction was leaking a detail. To read the JUrl this code needed to first distinguish if it was dealing with a local JUrlor a remote JUrl.

Real-world example of the JUrl leaking

​The realization that the JUrl type was too broad meant we needed to reify things with a narrower semantic. We need a wider syntax that would provide support for the two, more specific meanings: LocalURL and RemoteURL.

Leaking abstractions (checking for particularly concrete versions / conditional workarounds, etc,) point to having gone too far up the semantic coverage slope, to the point that the abstraction has lost its utility.​

How removing a leaky abstraction simplifies the model

You can tell an abstraction is leaking when:

• The code is littered with conditionals to check details hidden by the abstraction.
• Rules of operation are ad-hoc forcing you to go through each usage site in order to confirm that otherwise hidden details remain valid once you have added another instance of said abstraction.

Abstraction leaks, as evidenced by the need to introduce conditional workarounds or to disambiguate between specific instances, indicate that we have made things TOO abstract. In this case, the abstraction in question has lost its purpose: it is no longer beneficial to developers.

Sparse Abstraction

Sparse abstraction (source: Dall-E OpenAI)

If we consider the same producer façade, we already see there is an inherent cost to making this abstraction. It comes later whenever we need to distinguish between the different makers. We must pay it every time we use those if statements to inform us which path we are taking. ​

​Sparse abstractions, on the other hand, don’t leak: they don’t have the telltale sign of additional if statements, nor in any way require an expansion of syntax.

Sparse abstractions fail for another reason: they raise the coupling in the model since they cover just a few syntactical elements in such a way that it doesn’t make up for the additional indirections. This gives rise to a paradox, since a common goal of abstraction is the reduction of coupling.

Less lines without the producers

Once we remove the producer, we notice that there are actually two separate ​domains for consuming goods; Reseller and WholeSaleConsumer which consume only devices, and RetailConsumer and OnlineConsumer which only consume food. This distinction is lost when we introduce the abstraction, as is intended. But at the same time, we now have additional lines that couple the consumers of food and the consumers of devices with themselves, which weren’t there before.

We introduce new abstractions to reduce coupling. Sparse abstractions unintentionally and counterintuitively raise the amount of coupling in the system.​

This is yet another way an abstraction can degrade the developer experience. A way in which it seems to subtly not carry its weight. When an abstraction is simply joining a bunch of goods that really didn’t need to be joined, you have a sparse abstraction.

Removing sparse abstractions improves the developer experience by reducing the mental load required to understand the superfluous layer of abstraction. It also makes it easier to understand the purpose and nature of specific interactions. Developers will recognize this now allows the previously unnoticed separate domains to evolve independently.​

Lines of coupling go down from 8 to 6 when removing a sparse abstraction.

Sparse abstractions are harder to detect, but once removed you can confirm:

• There is less code overall.
• The overall coupling in the system goes down.​

Having sparse abstractions carries the burden of:

• Having to implement methods from interfaces that make little or no sense in the given class.
• Generate the feeling of having to jump through pointless hoops or red tape in order to get something done.

Eliminating sparse abstractions enhances the developer experience by lightening the cognitive burden associated with comprehending extra layers of abstraction. This results in a clearer understanding of the purpose and characteristics of individual interactions which had been previusly grouped. Developers will observe needlessly bundled domains can now evolve independently.

Useless Abstraction

Useless abstraction (source: Dall-E OpenAI)

Sometimes an abstraction is simply done along an incorrect axis. This is best observed as needing to refactor from one pattern to another.​ A useless abstraction happens as the intention of a system drifts away from the system’s chosen models, or its architecture.

As a system evolves and its concerns change we may find ourselves fighting against the existing original paradigm in order to deliver new features. Sometimes it is simply better to revisit existing abstractions and notice all the duplication they are generating, and possibly select another axis that reduces the amount of boilerplate or duplicate code. It turns out that in the previous example distinguishing between online consumers and retail consumers didn’t make sense.​ If instead we merge them and express their difference as two separate methods eatIn() and orderOut() lots of otherwise duplicate code disappears.

Less classes after splitting up the consume() method

The best and most accurate measure of bad architecture starts to show as soon as we see ourselves forced to repeat the same boilerplate; or the same patterns in our code. We somehow find ourselves repeating a ritual that is mostly or often even wholly unrelated to the value we intend to provide our users.​

The effects of a useless abstractions can be spotted by:

• CPD or other duplication tools to highlight consistently repeated patterns.
• Finding yourself having to fix the same bug in multiple places in the codebase instead of one.
• Having no place where to a section of code that may even feel pre-existing and that you have repeated multiple times before.

Multiple instances of the same repeated conditional

Noticing these things means there is another, different, or perhaps new abstraction that can better serve your model.

Splitting I into a new, different pair of abstractions (J, K) that prevent leaks

Improving your Models

In summary, we’ve seen three ways abstractions may fail:

• Leaky abstractions: The model is too abstract, and leaks make the code harder to predict.​
• Sparse abstractions: The model is needlessly increasing coupling and moving forward is slower than necessary.​
• Useless abstractions: The model is forcing you to repeat yourself, and boilerplate needs to be consistently added everywhere to support an abstraction that no longer aligns with the intention of the code.​

Software architecture is intrinsically linked to the code’s purpose. The moment we depict architecture in a diagram, relying solely on our good judgment and self-discipline to maintain adherence with our ongoing coding efforts is the instant at which the architecture diverges from our actual code.

This is true once you understand William Lawvere‘s statement that syntax and semantics are adjoint. That is to say: when we expand our abstractions to accommodate more elements, we inevitably lose the ability to describe them coherently.

In software, this principle holds true, underscoring the importance of embracing the simplest abstractions to prevent the accrual of technical debt in the long term. Errors in choosing abstractions are often dubbed ‘leakage’ or ‘abstraction leaks,’ but there’s more at play here. This concept ultimately leads us to confront the challenge of ‘reification,’ which occurs when an abstraction becomes itself concrete in the context of scarce representation (i.e., when too powerful an abstraction is holding only a handful of elements),

The fallacy of reification is one of if not THE most common mistake a software architect can make and is what developers allude to when they complain architects operate from an ivory tower and cannot fathom the consequences and implications of their designs once they need to be implemented.

​Next time you feel annoyed with how things are organized in your project, and you have a suspicion it’s related to one of these cases, you should now have the tools to successfully demonstrate objectively how your abstraction is failing the system’s purpose.

Curious about working in tech at Philips? Find out more here​