Disclaimer: I've use OpenAI's Codex to proofread and to help me with the diagram. But all the content come from me.
Introduction
I've heard people saying that software engineering is not a real engineering domain, based on the idea that software development doesn't require as much math as other engineering fields. At least, that's what I tend to understand from their comments. Maybe it's because software can be tied more to abstract logic and linguistics, which feels less like an engineering domain compared to those that are based more on hard science. Software engineering seems to have a foot in both types of science, soft and hard. But I think that it still has an impact on our lives in funny ways. Like the code in airplanes, which is written in a language that has the lives of hundreds of people in its hands. Imagine how many times somebody used language and it was misunderstood, and that's what we decided to put in charge. And similar to other engineering fields, software has a real impact on the physical world and, as a bonus, on the virtual world.
It can also cost at a similar scale, sadly. There was a great article[1] talking about the fact that software is indeed pricey, but compared to what? It goes on to say that developing software is way cheaper than trying to implement the same function with hardware, either mechanically or electrically.
And so the question is: what can we do to make it cheaper and better? Either to avoid having a negative impact on the world or, at minimum, to be useful. Which I think most engineering domains boil down to. And so I try to educate myself as much as possible on techniques to achieve that: better software. I did study software engineering in university, but I never felt like I could really master software architecture on a solid foundation. We had classes about software patterns, architecture, software requirements, and we did plenty of projects with peers. But in the end I didn't feel armed with the tools to really lead a software project and guide it with a good architecture.
Thinking back to those years, I remember a professor telling me that any programming language that is not Object Oriented is out of date. Which frustrates me today, because what he meant was that any software that doesn't allow you to manage dependencies easily is out of date. This has sadly been a pattern throughout my schooling and even in my professional and personal life, but that's a discussion for another blog post.
And so in my professional life, I did some architecture, learned about some, saw some fail, and sometimes felt like I succeeded. But I never felt like I had rules I knew should be followed. After a couple of years working, I started working with a friend from university and he always tried to fit patterns that he knew into the different projects we did. So as soon as there was a bit of architecture, he kind of went to the GoF patterns and tried to see which one fit. Which is not something I ever did, and that had a lasting impression on me. And so I tried to listen to more experienced peers or friends, and I was recently suggested a book and decided to give it a try. The book is Clean Architecture: A Craftsman's Guide to Software Structure. It is written by Uncle Bob, who is kind of a polarising figure in the software world. I think his book on Clean Code is not always considered beneficial, and he seems to leave little room for nuance.
But I looked up the reviews for the book on Amazon and there are a lot of good ratings. So I thought I might as well read it on the subway. In the end, I think it's a good book that helped give me a better foundation for software architecture, which I was longing for.
Summary
The book has 7 parts. We start with the introduction, and then we go into the meat of the subject. We get Programming Paradigms, where we look at OOP languages and other types of language design, with their respective pros and cons and what impact they have on the software you're writing. Then we get into 3 parts that are mainly principles. There are the Design Principles, where we talk about SOLID principles. Then Part 4 is about component principles and Part 5 is Architecture concepts. The two last chapters are for "Details" and then the Appendix.
I'm thinking of summarizing by part, because I took a bunch of notes that I want to distill per part.
Part 1
I think the first chapter gives the tone for the book well. It's not overly complicated and it does give some good lines such as:
[T]he goal of software architecture is to minimize the human resources required to build and maintain the required system
We then dive into the dynamics of software development, mostly about competing values. There is, on one hand, the business and then the engineering. One mental exercise he suggests that I thought was interesting was to push those values to the extreme:
Every software system provides two different values to the stakeholders: behavior and structure
The first value of software is its behavior. Programmers are hired to make machines behave in a way that makes or saves money for the stakeholders. The second value of software is software must be soft - that is it must be easy to change. When the stakeholders change their minds about a feature that change should be simple and easy to make.
And so which has the most value? The one that behaves but cannot change, or the one that doesn't behave but can change? He makes the argument that if it's harder to answer new requirements (he assumes there are always going to be new requirements, which is sensible), then at some point it'll be impossible. But if you have something that can easily change, then the new requirements can be more easily answered.
Then there's the Eisenhower matrix idea that he shares, where there's not urgent and not important, urgent and not important, not urgent and important, and urgent and important. He argues that managers fail to distinguish between what is urgent and important versus what is urgent and not important. Which results in the architecture not being considered important enough, which then makes the software harder to change.
He also argues that it's part of the software team's responsibility to assert the importance of architecture against the urgency of features.
My thought on this part is that there is indeed a problem with decision makers hearing the rationale of architects about this. It's not always clear, and it's also an investment for business managers. Because let's say I tell the team to build the software right away to answer the needs, and then we make money, we can invest more into making the software easy to change later on. And I understand there's always a balance, like what if the investment is more than the money made, or making it easy to change takes longer to answer the needs and then there's no money left to be made.
And the issue of people changing jobs to get better conditions doesn't really follow the second mentality of listening to the software team. So it's not an easy decision to make. And there's nothing guaranteeing that the team making the software will take the right decision when doing the architecture. I would argue the same thing for the business team choosing the right feature to make money. But a company doesn't really exist except to make money. Otherwise, that's an organisation that's not for profit. In my mind that's what it comes down to.
But I think that the responsibility of the software team is to find compromises that'll try to answer both concerns. There needs to be good communication between the business side and the engineering side.
And another thing that I think is interesting in the book is that:
Effective software development teams tackle that struggle head on. They unabashedly squabble with all the other stakeholders as equals. Remember, as a software developer, you are a stakeholder. You have a stake in the software that you need to safeguard. That's part of your role, and part of your duty. And it's a big part of why you were hired.
So indeed, we are stakeholders because, well, if it makes us miserable to work with, we probably won't stay and all our institutional knowledge will be lost. Which is sometimes bad for business.
Part 2
After those philosophical interrogations, we delve into programming paradigms. And the big point of those chapters is that OOP languages allow you to invert dependencies. If I were to give the single most important concept of the book, it is this. At the beginning I was talking about the fact that my teachers were saying that non-OOP languages are out of date, even modern ones. Well, that's what they meant: OOP languages allow you to powerfully manage dependencies and invert them.
Having that ability to control where the dependencies are, and adding abstraction to prevent coupling, is something we were taught. But I guess for me, without the context, or maybe because I wasn't interested in just conventional software, it didn't settle in my mind. But this introduction makes so much sense to me now and is now part of the foundation of my software architecture tool belt.
Then there's the idea of functional programming. It's mostly that there's immutability for the variables, which helps with sharing resources or concurrency. I think that's also a powerful tool, but less important through the lens of architecture. I would consider this to fall into the Details section of the book.
But I want to do another blog post about memory safety and I would like to include some of those quotes in it. I think that memory safety is paramount in technology choices.
Part 3
In part 3, the book mostly talks about the design principles. They're mostly tools to make concepts easy to understand and guide you into making good architectures. We talk about SOLID principles.
The ones that I have the most notes on are the Interface Segregation Principle (ISP) and the Dependency Inversion Principle (DIP).
For ISP, the conclusion summarizes it well:
The lesson here is that depending on something that carries baggage that you don't need can cause you troubles that you didn't expect.
And for DIP, I think that's the most important principle of the book, or at least it's one that gets mentioned a lot throughout the book. And I think it's because that's at the core of the initial idea of reducing the amount of resources required to build software.
The Dependency Inversion Principle (DIP) tells us that the most flexible systems are those in which source code dependencies refer only to abstractions, not to concretions.
It also goes back to the thought exercise about software that can change versus software that works but doesn't change. It also tells us that:
It is the volatile concrete elements of our system that we want to avoid depending on. Those are modules that we are actively developing, and that are undergoing frequent change.
So DIP helps us have a flexible and changeable system by not depending on the volatile and concrete elements of the system.
And that also means that we ought to avoid changing interfaces because that'll incur modifications to all the concrete elements that implement that interface.
Here's a list of coding practices to respect DIP:
- Don't refer to volatile concrete classes. Refer to the abstract interfaces instead. […]
- Don't derive from volatile concrete classes. […]
- Don't override concrete functions. Concrete functions often require source code dependencies. When you override those functions, you do not eliminate those dependencies – indeed, you inherit them. To manage those dependencies, you should make the function abstract and create multiple implementations. […]
- Never mention the name of anything concrete and volatile. […]
And to me this can pretty much summarize the book right there. Once you have that tool, most of the work of software architecture is right there.
Part 4
In this part of the book, we take a closer look at the pieces that compose a software system, or should I say, the relations between those components. It mentions some new principles, such as: Reuse/Release Equivalent Principle, Common Closure Principle, and Common Reuse Principle.
And the one that I found the most interesting was the Common Closure Principle (CCP), which states: "Gather into components those classes that change for the same reason and at the same times. Separate into different components those classes that change at different times and for different reasons."
Which I feel like now we are talking at a high level of abstraction. We are not at the class level but the group of classes, which is the essence of the component idea. And the CCP is similar to the SRP that we saw earlier but at the component level.
Next is the Common Reuse Principle, which states: "Don't force users of a component to depend on things they don't need." This to me seems like software architecture hygiene, where we want to make sure we don't have waste dangling from our components.
And so we should make sure that those components use all of the classes that another component depends on. And also make sure that the grouping of the classes is coherent, such that "the components are inseparable - that it is impossible to depend on some and not on the others. Otherwise, we will be redeploying more components than is necessary, and wasting significant effort"
And the last part of that quote is something that Uncle Bob mentions a couple of times and I thought was interesting. I feel like that should be studied more, or in essence studying the whole software life cycle, but what I'm trying to say is that to deploy a new component, we shouldn't have to output a new .deb package or a new docker image. It should just be a new component that's loaded at runtime and then used. Which is doable in C++ and Java, at least those are the examples he gives in the book. But it made me think that we don't necessarily think about those aspects in software engineering courses. Maybe it's because it's always dependent on the technology and we don't want to have courses depending on that.
Uncle Bob mentions that depending on the state of the project, some of the principles will evolve in importance. Less mature projects will draw more on the CCP and the CRP, while as they become more mature they'll depend more on the reuse principle. The idea is that as they become more dependable for other projects, they require more stability.
The next chapter of this part is "Component Coupling", and there's the Stable Dependencies Principles (SDP), which is again similar to the DIP.
But the "Stable Abstractions Principle" (SAP) is interesting. It states that "[a] component should be as abstract as it is stable". I find this interesting, because the fundamental parts of the system should be the most abstract and the most stable. I wonder if that is also something we can draw from writing requirements, where the abstract actions we do with the system should be the most stable to answer the use case.
There's maybe a little correction that I need to mention: when talking about the DIP, Uncle Bob is talking about classes, while here it's components. Which have the same idea. The SAP and the SDP are essentially the DIP.
Then there's a bit of math at the end of the 4th part. It's there to help evaluate more objectively where such components should be positioned. There's also some more diagrams, but it's again pretty much what is explained in the 3rd part about DIP.
Part 5
In this part, there is a bit of reflection on what software architecture is and its different facets.
The architecture of a software system is the shape given to that system by those who build it. The form of that shape is in the division of that system into components, the arrangement of those components and the ways in which those components communicate with each other.
The strategy behind that facilitation is to leave as many options as possible, for as long as possible.
Here we are definitely more in the process or reflection on architecture, such as the development of the software, its deployment, and its maintenance.
And software that is easy to work with is often a good sign of software that's going to be more successful than not. There's also mention of the way teams are structured: smaller teams can go faster and need less structure, while larger ones go slower and need more structure.
And another good sign is that if it's easy to deploy, it'll often have better quality and longevity. Just because if it's easy to send fixes, your turnaround time for making that software better is faster.
And then we also get into the fact that if it's easy to change, it'll be able to live longer and provide more value through time. There's also a funny quote: "a good architecture maximizes the number of decisions not made."
And in the end, the architecture needs to answer these criteria: * The use cases and operation of the system. * The maintenance of the system * The development of the system * The deployment of the system
And we can also see the value of being flexible and not making assumptions. If an application is a monolith and it's hard to break down into multiple pieces, it might have its days counted. But one that doesn't assume the way of communication between the components, and has proper isolation between the components, is much easier to transition through the spectrum of threads, processes, and services as the operational needs of the system change over time.
We should make it easy to change in all the ways it must change by leaving options open.
There's also an interesting reflection on how to attain the initial goal of minimum human resources to build and maintain a system, which can be seen as reducing the coupling of the system.
Boundaries are the axis of change
What are boundaries? They are the limits of the component and other elements, such as an interface, abstraction, or another element. And they help us define where we need abstraction to help us in the future as the requirements change.
It's important to define those since it's going to help us define the components and then define the abstractions that we need.
Part 6
The thing that I found most interesting in this part is the part about frameworks, which says:
The relationship between you and the framework author is extraordinarily asymmetric. You must make a huge commitment to the framework, but the framework author makes no commitment to you whatsoever.
Which I think is something that people should be more worried about. I've used ROS code for a long time, and that was one of my pet peeves of the framework. It becomes tentacular and your whole software stack is dependent on it. It makes me remember when they decided to switch to ROS2 and there were die-hard people staying on ROS1, which from experience was more robust. But it creates that relationship where you don't control what you are creating as much as you would like.
Another idea that I thought was interesting was how can we enforce architecture with the compiler? Or in a fast feedback loop. And that's what Uncle Bob suggests, instead of either self-discipline or post-compilation tools. I've been very curious about the Vale language lately, and there are a lot of simple techniques that they added to make the language safer. And I'm wondering if it would be possible to have something like that. Like could you define a specific annotation that specifies a relationship? Or a level of abstraction that you want to keep? There's probably something in academia about that, but none that I know of. Papyrus exists, but it's not exactly the dynamic that I'm thinking of. I'm thinking more about a hybrid, where you can do both. But it's just an idea, I'm not sure how good it is.
Analysis
Strengths & Weaknesses
What I think the book does well is explain most concepts very simply. It's easy to read. Often when I see stuff about abstraction or more computer science topics, it's harder to digest, but I found this book very easy to read and enlightening. While I did 4 years of study in software engineering, this concept of dealing with abstraction that way and building software with those principles would have been way more valuable than learning a bunch of patterns and criteria. I also think that the ideas behind the different relationships that he laid out were very interesting, like the disproportionate commitment of adding a framework to your project.
But what I found it could have done better was to show other architectures than Clean Architecture and be more critical of it, or at least explain the pros and cons better. At least I don't remember it doing so. Clean Architecture is relatively heavy and causes a bit of frustration from what I read about it online. I would also have shown Hexagonal Architecture, Layered Architecture, etc., as counterparts to the solution he's sharing.
Relevance from own experience
I've tried to apply the bigger picture in one project at work, where I tried to delay decisions as much as possible and make it as flexible as possible. I thought it was useful because some requirements could change, which would cause us to do rewrites, but doing it with the right architecture prevented that situation.
Conclusion
Overall, I really liked the book. It was short enough to read in the metro. It gave me a few rules that are very powerful and helped me avoid having to remember and shoe-horn patterns. It gives you the right tools to protect yourself and your project, most of the time at a small cost.
I think most people who work with software ought to read it, just because it gives you the right mindset and doesn't force you into having all that extra structure. It focuses on the end result and the abstract goal of software architecture.