Abstract
My zen art as a software developer hones that model of the Universe safely housed in my head through the practice of modeling lesser domains than the Universe itself.
We are sentient objects somehow flourishing in the impermanence of a relentlessly changing world. Our brains are the model of that world which determines our interactions with the world. If the model doesn’t evolve along with the changing world, that disconnect results in suffering.
Likewise, a business flourishes in a business world that is in constant change. Without acceptance of that reality when building software, a business is destined for a life of dukkha. That is, there will be a gap between how it conducts business and how the world really is.
This blog may sound more like an IT whitepaper than a piece on Zen. Towards the last quarter, it can get a bit too techie. But this post is at the heart of what Zen from my software developer’s perspective is all about.
However, as it is with enlightenment on Bodhi Day, you will unlikely will walk away from this post armed with belt of silver bullets that will lead your software project to success. What I hope you walk away with is that this post is really a parable of the Eightfold Path, particularly the business equivalent of Awakened View.
Here is the SBAR of the post:
- Situation – Modern software projects are implemented in complex environments, but are managed as if they are merely complicated. So many projects ultimately fail. Even if they succeed, the win is under-whelming.
- Background – The ability of sentient people to invent, design, and strategize is based on a model of the world we maintain in our brains. Our brain evolves along with the evolving world it’s modeling. Likewise, a business is an entity that must continuously strategize. Shouldn’t it have an evolving, integrated model too?
- Assessment – For a software development project, if we turn our attention from merely satisfying requirements towards the ability for businesses to adapt to an ever-changing present, we’ll close the gap between what we build and reality.
- Recommendation – Taking the time to thoroughly model the subject of the software being built, its domain, mitigates show-stopping misunderstandings. A methodology for building such a model is Domain Driven Design implemented with just the type and function definition system of a functional programming language – understandable by non-programmers with reasonable effort.
By the time you get towards the end of this post, you’ll see this is an abstract about a post on abstraction!
Building a Complex System in a Complicated Way
My biggest current challenge as a software developer is conveying to my co-workers that the complexity of even moderately-sized software projects are now beyond the capability of being managed solely through human effort. Today’s enterprises have so many intertwined moving parts and external relationships – people, machines, and other organizations – that even a small software project comes with so many external dependencies that it’s really much more involved than it appears.
During the days before the well-functioning networking of the Internet, the scope of software projects were well within the domain of a single business process, a department. The limitations of hardware and software tools back then naturally limited what could be accomplished. Additionally, a far less “global economy” limited what needed to be done.
The limited scope meant that software was still in the satisfyingly deterministic realm of complicated. There weren’t webs of dependencies between many objects with rules of their own that form complex systems. For example, dependencies between a number of systems on different hardware platforms, different software frameworks, policies of various companies, across clusters of servers, in different countries with different laws … on and on. Back then integration of software consisted of humans manually typing data from one system to another.
Typical software projects are now in well in the realm of complex. As an example of complicated versus complex, an F-22 fighter itself is a complicated object. It has a ton of really sophisticated parts, but it is still deterministic. The operation of an F-22 is complex, involving much more that the aircraft and the pilot. Complex software, or any other complex system, can’t be managed through tactics that work for merely complicated solutions.
Actually, large software projects have been beyond the paradigm of traditional top-down control by a small team of managers for a long time. It’s not like this issue hasn’t been addressed over the years. Around 2005 or so, Unified Modeling Language (UML), was one of the big over-hyped buzzwords of the time. Domain Driven Design was born in 2005 with the publication of Eric Evans’ book. There are the “user stories” of the Agile Methodology.
However, at of the time of this writing, most of my colleagues have heard of UML but don’t know much about it, maybe heard of user stories (despite much experience with Agile), and hardly any have even heard of Domain Driven Design. Yet, software projects at best succeed with under-whelming enthusiasm, and worse, years of really tough, expensive maintenance.
The challenge isn’t understanding the value of gathering requirements and documenting use cases. The challenge is to integrate these fragmented chunks of information into a coherent, integrated, fully-connected view that is understandable by all. Then to convince everyone that this obviously tedious task is worthwhile, not just some fluffy activity.
Alternatively, we could continue fighting complexity through tightened control. Which just makes things more rigid and paradoxically more complex. Rather than trying to conquer complexity in this complex world, we embrace complexity. That’s what Buddhism is all about. We embrace complexity by learning to adapt to an ever-changing present.
Automating Real World Processes
Software is about automating processes. That is, it drastically enhances what we currently do so we can focus on the more creative, more intellectually challenging aspects of work – the thing we humans currently do better than machines. Occasionally it can do things we never could do, even with all the time in the World. In either case, software augments our human activities.
The problem is that since the world is in constant change and software as it is today is incapable of self-evolving as how we humans are able to evolve by learning new skills. Or at least, software is incapable of self-evolving without making a big mess of things. Heard of “herding cats”? This is more like herding toddlers as big and mean as bears. Learning (evolving) is enough of a struggle as it is for we sentient humans – which is the main idea the skill of Zen – without the additional chore of manually evolving our software.
Yes, software too is cursed with dukkha. Meaning, the programming of the software doesn’t match reality. Except the non-sentient, non-evolving software doesn’t actually suffer pain from the turbulence it causes. It just tosses out an error message and passes the suffering to us.
Models versus Requirements
How can we help our software systems to evolve along with an evolving reality? What if we use our human intelligence to encode and link the large collection of narrow-scoped fragments of the world that we collect from lots of users into a single, connected, wider-scoped model? And these models should be readable to a all that are involved, human programmers, human non-programmers, and machine.
This post primarily address why we need to build a model of an enterprise in order to effectively engineer complex software undertakings. What is it about the way we currently approach large, modern software projects that result in under-whelming results or even failure?
A full explanation of the methodology of how we build this model is well beyond the scope of this blog. For the moment, I would like to point out that there is at least one viable methodology that has been in practice for over a decade – Domain Driven Design, which I previously mentioned.
It’s imperative to understand that models are not the same thing as requirements. Requirements are about what we need from the solution. The model is about how the domain works.
Requirements are relatively easy to gather. We just ask. However, we programmers don’t know what the business folks know and the business folks don’t know what we know. So requirements we’ve gathered can’t be all that good.
Now, it probably sounds very obvious that we need the requirements of what we’re building. How can we build something without knowing what it’s supposed to do? But this model thing probably sounds like another huge activity that has value but not enough to warrant the large investment of time and resources. “Requirements” always sounds so pragmatic, model sounds superfluous.
“Do we need that model? We can piece that together from the requirements.”
“It sounds really tedious. Do we really need to know the in-the-weeds details?”
“Sure it helps, but the users and test scripts will tell us if it’s not working.”
“We don’t have the time for ‘nice to haves’.”
The problem is that your understanding of what is done in a domain is a model you built in your head, and that version exists only in your head. Others on the team don’t have the same experiences as you, so the models in their heads won’t be like yours. Surely, you can communicate what you know to the others. But human-human communication isn’t really an exact science void of ambiguity.
The next two sections describe the model of the world in our brains and the resulting dukkha from the disconnect with the real world, followed by the counterpart for the “brain of a business”.
Our Brain’s Model of the World
Every thought we have and action we take is based on a model of the Universe that has been evolving in each of our brains since the appearance of our first neuron. This model is a network of rules, cause and effects, and goals we’ve been exposed to over the course of our lives. It empowers us with the ability to analyze a potential action in the safety and privacy of our heads before taking physically irreversible actions in the world.
The evolution of our brain’s model of the Universe is driven by rules encoded in our DNA, what we passively learn, what others train into our heads, and conclusions we make for ourselves (based on what we’ve learned). It’s the blueprint of how the world works and how we interact with it. However, as I’ve written before, this model is a pitifully inadequate model of the Universe.
There are two primary reasons the model in our heads is pitifully inadequate as a model of the Universe. First, we cannot possibly know everything. As Ringo said, “Only Everythin’ can know Everythin’.” The only adequate model of the Universe is the Universe. In the normal scope of our human animal lives, we’re not everything, so we don’t know everything, so we compromise – if we can’t know everything, we can at least know what seems important to us for the moment.
Therein lies the second reason; everything is constantly changing, so what’s important to us for the moment is a moving target. Therefore we learn; adapt by reflecting the present and mending the disconnects between what we think and what we’re experiencing. Most animals have some ability to learn but none like we humans can learn, at least on Earth.
However, unlike lower-sentience animals, learning for humans is about adapting to a perpetually changing world – in addition to building the skills we need for a role. Lower-sentience animals mostly adapt by passing on genes that seem to offer competitive value to their offspring. We humans can adapt our behavior and environment to suit present needs.
Dukkha arises from the disconnect between a change-resistant model of our world in our heads and the relentlessly changing world that is exceedingly bigger than any of us.
This model of the Universe we hold in our brains is so important that as obvious as it sounds, it’s worth the reminder that without it, the physical object we know as ourselves does not exist within Life on Earth anymore.
A Business Entity’s Model of its World
Businesses are very much like organisms. I’d argue that businesses are organisms because they are run by humans – where each human is the equivalent of a neuron in our brain. Except that a neuron isn’t nearly as opinionated.
Both individual humans and businesses are made up of thousands to many quadrillions of parts with a surprisingly large numbers of roles. For humans, it’s not just a matter of a couple hundred bones and a couple dozen organs. It’s trillions of cells and bacteria and thousands of different proteins.
For a large corporation, it’s not made up of a few dozen departments. It’s tens of thousands of employees, working at a few hundred locations, across thousands of unique roles, with thousands to millions of physical assets, millions of electronic assets. At least a human has a brain that organizes all of that in one place, where it’s well protected by a thick skull.
Both human animals and corporations compete in complex ecosystems against predators and competitors for limited resources (prey and customers). To succeed, humans and corporations must perform such that their whole is greater than the sum of their parts. For humans, that “whole” is sentience.
For business “organisms”, there isn’t the level of coherent sentience we humans enjoy, but there is a seed of sentience. We multi-celled organisms were once just a motley collection of cells, which over hundreds of millions of years coalesced into the coherently sentient beings we are today.
For corporations, success involves more luck than executives would care to admit, and a whole lotta dukkha. The knowledge of a corporation is fragmented across all employees, dozens to hundreds of software applications, millions of documents. The parable of the “Wise Men and the Elephant” is a ridiculous understatement. It’s more like “The Cells and the Human”.
What About A.I.?
Before continuing onto an exploration of the current state of software modeling, I need to point out that none of this post is about Artificial Intelligence.
Today’s A.I. can dig through massive databases for answers to posed questions with mostly rote algorithms taught to it by people. Today’s A.I. is far from capable of discovering linear algebra and calculus on its own. It’s far from organizing solutions for ambiguously defined problems. Strategic planning is still well within the realm of human brains.
Someday an A.I. or highly augmented people may render this blog moot. Shakyamuni Buddha’s audience of pure humans, will no longer exist, and his teaching will fall on deaf cyborg ears. At this point, what we need and can obtain is a model of the system that we can all look at (again, human and machine participants) and “observe” in a minimally ambiguous manner.
Current State of Software Modelling – Heap of Documents
As of today, whether or not a business possesses such a model, businesses somehow do plod along. All over the world, billions of people show up to work, work gets done, money is collected, money is paid, the doors open the next business day. So again, is this model thing just a “nice to have”?
Businesses do actually work without a model because there is a model. It exists in fragments distributed across all the employees. It’s often called “tribal knowledge”.
It’s long been fashionable to say, “Most of an enterprise’s data is stored on thousands of Excel spreadsheets on hundreds of desktops and laptops.” It’s probably true. But even worse, most of the knowledge of how things work in the enterprise is stored in peoples’ heads. It’s much harder to transfer that tribal knowledge brain to brain than to emailing a spreadsheet.
If a long-time employee leaves, the business takes a huge hit like the hole in a ship made by a torpedo. It’s usually possible to heal the business by piecing together fragments held in the heads of other people and assorted documents somewhere. Sometimes it can’t be reconstructed, so things need to be reinvented.
If we’re lucky, someone’s tribal knowledge is mostly redundant or easily re-constructed from a few readily identified sources. It would be great if someone, a small team, or something held the big picture at any given time. But unless the enterprise is a really small organization, no thing has that Big Picture.
It’s easy to counter that someone does indeed have the big picture – the CEO. But don’t mistake a 50,000 foot view of an enterprise as knowing everything. The CEO trades-off detail for scope. One way or another, no one person sees the entire elephant.
Requirements from Fragmented Sources
Before we can build anything we need to know what we’re building. Software, temple, Space Shuttle? Once we know what we’re building, an ERP system for example, we’re at least pointed in a direction and can now bring in experts to dive into requirements gathering.
Some teams are better at requirements gathering than others. Some teams dig deeper into the requirements than others. Some still jump straight into coding – often something no one yet asked for.
Requirements gathering is an iterative process with three stages.
The first is an overall idea of a problem we’re trying to solve. At this stage, we need enough of an idea of what we need so we know if it’s even possible, and if it’s really the right approach to solving our problem. For example, if I have a legal problem, I can’t expect to show up at the attorney’s office and expect to walk out the same day with a resolution. My reasonable expectation is the attorney will accept or not accept my case.
The second stage is where we dive deeply into the design and planning of what we’re building. We’re pretty sure this is the right approach. Continuing with the legal problem example, this is where my attorney has thoroughly researched the facts, formulated a strategy, and we have a court date.
The 3rd stage is really on-the-fly discovery as we actually get into the build. Ideally, there would be no 3rd stage, and we often over-confidently act as though there isn’t one. We delude ourselves into believing everything was fleshed out in the 2nd stage.
But complex systems are rife with unknown unknowns. Hopefully, whatever gotchas we discover in this 3rd stage are minor enough where we’re not hurled back to square one. To conclude the legal problem example, throughout a trial, things will come out that require altering the plan. However, if by some miracle a trial or project goes exactly as planned, we believe more in our planning skill than luck.
Software is a virtual thing. That’s a good thing and a bad thing. It’s good in that as an engineering project errors are much more forgiving than say for building a bridge. If software breaks, in most cases, a ticket is filed and someone fixes the bug. If a bridge collapses, people may die. And, re-building a bridge is much more expensive than rolling out a hot fix.
However, in the context in which software is built, potential problems aren’t as obvious as it would be for something built within the real physical world. The engineers and architects of bridges design to the model of physics. For a bridge, a physical thing, the requirements are more straight-forward, more in our face. The thoughts of our brains are just one step removed from the physical actions we execute in reality. But software is at least two steps removed from the reality of the physical world – jumping from programmer, to software, to the real world.
The Dukkha of Requirements
For the most part, when building software, requirements are gathered and organized by people who don’t actually do the work that we’re attempting to automate or augment. These are typically “white collar” folks such as managers of the people who do the work or business analysts who may not have ever lifted a shovel.
The dukkha of requirements is the disconnect between what the software does and how things actually are.
I need to label the two classes of workers from which requirements are gathered. As mentioned, there are the variety of “white collar” managers, business analysts, architects, and programmers. Managers can include those who manage the folks who actually do the work as well as “managers” of software projects such as the project manager, product owner, leads, etc. These folks are highly educated, creative, but at least one step removed from the “real” work.
Second, there are the ones who actually do the work the software is meant to automate or augment. They are the ones “in the weeds” who physically touch the details that are lost in the abstraction of requirements gathering and chains of translation. However, the closer to the weeds the more fragmented the knowledge.
The Concretes and the Abstracts
For the sake of brevity, I’ll refer to those who actually do the work as “Concretes” and those who don’t currently do the actual work being automated as “Abstracts”. I hope those terms don’t offend anyone. Since none of us knows everything, all of us at different times and places play the role of concretes or abstracts.
Considering the mix of abstract and concrete folks, there are at least three ways that requirements are gathered. All have their pros and cons.
The Way It Really Is
The concretes are the only ones who really know what goes on in the real world. The concretes are able to articulate their little piece to full fidelity. However, the problem still remains that the concretes can’t see the big picture. The concretes are the blind men feeling different parts of the elephant.
To clarify, concretes can be line workers “in the weeds” who only see a small part of the entire business. Or they could be the CEO with the 50,000 foot, Big Picture View – like being able to see the larger state-wide map, but not the map of her neighborhood.
The knowledge of the concretes is imperative for the design of software that reflects reality. For software to be in sync with the real world, we must know of the real world. However, a design authored purely by concretes very much risks disconnects between theory and reality due to their fragmented knowledge.
The Way We Think It Should Be
Concretes and abstracts will have opinions on how things should be, apart from how things currently are, because we must always yield to better ways. Even if they are in agreement, the fact remains that the work of the concretes is still manifested in a concrete world that currently isn’t doing things “the way it should be”.
If we design software to the way things should be, as opposed to the way things are, the real world of the concretes will need to evolve into that software reflecting the idealized way things should be. Of course, part of the software rollout will include plans for training the concretes to the new way as well as re-tooling the equipment and processes.
It does seem to make sense to build a software system towards the way we think things should be, along with a rollout plan to mold the real world to the idealized world. We are after all sentient humans gifted with the power to shape the world to our liking. However, writing software to an idealized world that doesn’t yet exist is a bold, maybe arrogant, all-in move. Do we assume there aren’t any unknown unknowns?
The Way We Think It Is
The dominant contributors to a software application’s design typically never actually did the work they are tasked with automating. But they probably have some notion of the nature of a domain. For example, programmers, architects, and project managers probably have a good idea of how healthcare, restaurants, or insurance companies work. However, aside from maybe a stint at McDonalds during high school, they unlikely were ever nurses or insurance agents.
Even though we engage subject matter experts (SME) culled from the ranks of those who actually do the work being automated, they typically are interviewed by the abstracts who mostly ask the questions, then pass those answers to the architects, who pass those answers to the programmers, who encode it in software.
So What Do We Do?
As a starting place for having our cake and eating it too, let’s start with this ridiculous unicorn fantasy: The people who actually do all the work, also happen to be crack software developers, who are infinitely patient. They get into a room for a couple of months and hammer out the perfect code.
Easy, huh? Such people do exist, but even Elon Musk can’t do everything. Back in “the day”, the 1980s, when software was merely complicated, not complex, I knew many professionals such as dentists and lawyers who learned how to program and founded successful software companies. Those days are somewhat gone. Things are very complex and low-hanging fruit is hard to find.
The Domain Model Pidgin
In these days where software is complex, it’s not possible to find people who can play all roles. So the roles must be distributed among a number of specialists.
Each of those specialized roles all have their own language. Bob speak nursing, Alice speaks architecture. But Paul speaks “Business” and sort of understands architecture and nursing, so Paul is the intermediary between Bob and Alice.
The problem is that communication risks being lost in translation from Bob to Paul to Alice – then back from Alice to Paul to Bob. The left side of the illustration above shows that communication from those who actually do the work takes a few hops before it’s encoded in software – to the Business Analysts to the Architects to the programmers.
But what if we could bridge those gaps with some sort of common language? Something we could all look at and understand to a workable level. Some sort of pidgin.
When people from different cultures are tossed together, for example Hawaii and New Orleans, they don’t have a way to communicate. They may initially just stare at each other. But soon enough a rudimentary language emerges that they all understand – a pidgin. The syntax is terrible, the vocabulary is sparse, but it’s enough where they get the workable gist of what each other is trying to say.
Over time, that pidgin evolves. A formal syntax emerges. New words are formed to fit new concepts unique to an emerging culture blended from that initial mishmash of people. That pidgin becomes a “creole” – a bonafide language. That language is the model, the expression, of a brand new culture. And it will continue to evolve along with the evolution that culture. Without that evolving model, there is no culture.
New State – Reflecting the Evolving Present
Businesses are all unique cultures. They have unique strategies, serving unique clients, and staffed with unique humans. Like the cultures of Hawaii and New Orleans, a model emerges that captures the culture and evolves along with it. However, as stated above, that model doesn’t exist in one place. Rather, it’s distributed in fragments among the employees and documents.
Is it possible to encode an evolving model? If not the entire thing, at least encode the domain of a software project?
The sophistication of such a model lies somewhere between a lifeless collection of project documents and genuine Artificial Intelligence. Unlike our real intelligence, this model will have practically no ability to self-learn.
No matter how we gather requirements, the software we develop will reflect a distorted image of reality to varying degrees. But that’s not the worst problem. The worst problem is the world is constantly changing. Similar to how house doesn’t clean itself up, software doesn’t evolve itself, we humans do that chore (for now) – and it’s a chore.
As the world changes around us and the software we use to automate our processes lags behind or even remains static (no new versions), the software limits our agility, our ability to compete. We find ourselves molding our real-world business processes to fit the software etched in silicon.
Remember, by “model”, don’t think of model airplanes, model trains, or even business models. Such models are just snapshots. Models are usually just shadows of the real thing – capturing the general shape, but masking away the details. What we’re looking for is a reflection of the real thing. The model we hold in our brains evolves to reflect our understanding of the world around us.
Similar to the brain reflecting our current understanding of the way the world works, this model reflects our current understanding of how the business works. The methodology that I feel best encapsulates this modeling is Domain Driven Design (DDD). DDD refers to such models simply as “domain models”.
I call this model a “Domain Reflection Model”, DRM, to better capture the notion that the model is meant to be a reflection the ever-evolving business.
Domain Driven Design
Domain Driven Design is a huge and deep topic. It’s also rather counter-intuitive in the software development world at the time of this writing. However, it’s actually very simple if you embrace the counter-intuitive aspects. With apologies to Sensei Eric Evan and other established DDD practitioners, here is my perhaps slightly off-kilter take on a few major points I feel are relevant to this post:
- Thorough upfront vetting of a business’ processes illuminates unseen potholes – the unknown unknowns. Such an exercise will be a severe pain in many ways, but a well-lit road makes for better odds of getting home safely.
- Sometimes it’s best to let apples be apples and let oranges be oranges. Instead of forcefully conforming one to the other, let them work as they are and handle any necessary translation as a separate translation component.
- Communication between folks of different project roles are hampered due to confusion caused by the insistence of folks using the terminology of their respective fields. With DDD, people of all roles defer to the language of the concretes, the people who actually do the work. That vocabulary is at the finest level of detail since it’s the closest to what really happens.
A Few Major DDD Definitions
Domain – This is the subject of the software being developed. If we’re building software for servicing customers in a restaurant, the domain involves all of the processes from seating to billing along with all the things (waiter, customer, orders, etc) involved. However, be cognizant of the fact that servicing customers in a restaurant doesn’t exist in a vacuum. There are processes for things like procuring ingredients (no ingredients, no food to serve) and payroll (tips are part of the payroll process).
Domain Model – This is the documentation of the domain, the model, the prime object of this post. Whatever the method of documentation, the primary piece is the thorough mapping of process flows within the domain. It has three primary characteristics:
- It’s a model that folks across all roles can “watch” and so agree upon what’s going on.
- It’s a living structure that reflects the evolving present.
- Aspects of this model can be projected into many fundamental things – such as documentation, code, visualizations …
We document the process in excruciating detail. For example, if we note that the waiter picks up an order from the kitchen counter are the results simply that the food will get to the table? The waiter could trip on the way, the waiter could find that it’s the wrong order, the customers could be gone. This is critical since this is real life. Each of those possibilities results in another process that the software must somehow handle.
Of course, we shouldn’t go overboard. Listing that a possibility is the police will storm into the restaurant is a bit too far. Those sort of overboard errors could fall into the typical catch-all “Unknown Error – See your Administrator” … haha.
Bounded Context – In software, we attempt to abstract concepts to remove the fuzzy details in order to simplify things. That’s good. But sometimes forcing a square peg into a round hole isn’t worth the trouble. Or at least, we can punt forcing them together to a later time. For example, in a restaurant, the process for serving a sit-down customer is different enough from drive-through where they should be handled separately.
Ubiquitous Language – This is the language within a bounded context. It’s the language the people who actually do the work use – not the jargon or the programmers, project manager, or that of another domain. We depend on the concretes, the people who actually do the work, to provide the knowledge as close as possible to reality. Therefore, we need to speak in their terms.
Analysis Paralysis
At this point, especially with phrases like “thorough upfront vetting of the processes”, it’s easy to imagine the reader horrified with visions of being mired in the quicksand of over-analysis – the so-called “analysis paralysis”. I don’t think this is the same thing.
Vetting of the processes is a learning activity while the kind of over-analysis that can mire you in quicksand is a decision-making process. The latter is especially true when choosing the lesser of two evils. We’re so afraid of making the wrong decision that we are caught in the limbo of analysis.
Analysis Paralysis is seductive because analysis usually seems more like “real work” than learning. It’s easy for people to assume you already know what they know about the business.
However, it’s not that people don’t already go through extensive discovery and mapping phases. The problem is that these discovery exercises, which can go on for months, is highly fragmented. Meetings are held with various parties, piles of information are gathered, people (usually abstracts) disappear for a few weeks or months, and then return from the mountains with The Plan.
Domain Driven Design Using Functional Programming
Development of a domain model itself may get all the humans of a project aligned with a common understanding of what is being built. However, what brings “Pinocchio” to life is to encode the domain model into something the computer pieces understand as well.
Of course, I’m not suggesting we provide “code as documentation”. Although computer program code does unambiguously describes the process, that’s a very silly notion. Even programmers have a hard enough time reading their own code a few weeks after last looking at it. So we need to do better.
One option is if we can build the domain model using the type and function definitions (only the definitions, not the gory code implementation) alone we can end up with what is genuine code. Of course, “code” is “understandable by computers, as well as programmers and architects.
But it can plausibly be readable by the “non-technical” executives, business analysts, and people who do the actual work as well! The type and function definitions of a programming language should be readily learnable by anyone who knows how to spell “Excel”.
It happens that the strongly-typed type system and function definition syntax of Microsoft’s F# is such a language. Thorough discussion of DDD and F# are well beyond the scope of this blog. But there are wonderful references listed below my signature. In particular, there is the fantastic work of Sensei Scott Wlaschin.
However, even though a model encoded using the F# type and function syntax should be plausibly readable by anyone involved in a software project for whatever role, we can easily express the model into whatever format works best for particular roles.
From the Domain Reflection Model encoded with F#, we can write simple code transcribing many views. For example, code to transcribe the DRM to a network view is very simple. For example, simple C# code that formats Cypher statements loading into a Neo4j graph database. Simple C# code could “normalize” the DRM into a relational database schema or Excel sheets.
The transcriber of the domain will be the architect, the person traditionally in between the programmers and everyone else. For the architect, the coding aspects of transcribing a model to F# types and definitions is child’s play.
The architect can hand over the DRM to the programmers as a genuine blueprint, DNA. For the case of F#, these are .NET classes with names matching the ubiquitous language into which programmers unambiguously fill in the implementation.
Now, I understand that the notion of DDD coupled with FP probably sounds insane. Both turn the current corresponding ways of thinking, upside down:
- Spend a ton of time building a model we all can look at before even starting? It can sound very indulgent.
- Very roughly speaking, the difference between the very familiar (at the time of this writing) object-oriented programming and functional programming is like focusing on the nodes of a network, then instead focusing more on the characteristics of the relationships.
Functional Programming should be more innately in tune with a Zen practitioner. Object-oriented centers around things. Functional programming centers around the transformations between things. There are no things, but their is constant change.
Data Governance and the Continued Evolution of the Domain Model
The Domain Reflection Model begins its life reflecting the business at the time of the project which gave birth to it. It needs to start somewhere. But if the DRM doesn’t evolve along with the changes of the business ecosystem it’s reflecting, it becomes like our brain stuck in ’82. All the business’ energy is spent in futile attempts to bring back those glory days instead of bravely flowing towards the future.
Therefore, a DRM should be under the authority of a Data Governance Office. It should be a first class citizen, right up there with Master Data Management, Security, Compliance, Data Quality, Business Rules, and Metadata Management. It should be governed by DRM Stewards.
Process Management and Project Management folks should work tightly with DRM Stewards to leverage the knowledge in the DRM and evolve the DRM to continue to reflect the current mechanisms of the business.
Like the data stewards of Master Data, the DRM should be maintained by “DRM Stewards” who update what is known of the processes.
Domain Driven Design and Business Intelligence
DDD is usually associated with software automating processes that can be well-defined within reason. Those are the the actual activities of a business such as manufacturing automobiles.
Business Intelligence is about a special business process. It’s about the very nebulous process of learning – specifically learning about how to optimize the processes or at least to evolve the processes to match the changing world.
Indeed, Business Intelligence is really supposed to be the “brain” of an enterprise. Not just the glorified reporting it seems to have become. Like the brain, it integrates information from all over into a single place where it can be centrally analyzed. I’ve also compared domain models to the human brain. Therefore, DDD has a naturally better fit in the learning-oriented realm of BI than simply the automation of business processes.
The DRM serves the effectiveness of BI by identifying the major drivers of a certain problem. In this Big Data era we store everything. For any given problem, most of that will be noise that we can tune out once we know what’s important.
Conclusion
As I mentioned at the beginning, this really is a Zen post that may seem more like an IT whitepaper. The primary message is that the notion of creating and maintaining a comprehensive model of an enterprise is the business equivalent of the model of the Universe in your brain. The disadvantages are the same – it’s a major pain in the ass to maintain. But the rewards are great – a coherent, evolving model of the world that minimizes the disconnect, thus minimizing the dukkha.
The model in our brain or the DRM will never be perfect. The world is in constant change. The point of Zen as well as the point of upkeep of a DRM is that the price of our Gift of Sentience is perpetual polishing towards perfect reflection.
The dorje cookie pictured above features an enso, the black circle in the middle. The enso represents our journey. They are usually depicted with a gap which means something along the lines of “not-yet-perfect” – that the journey still goes on.
I look at it from a subtly different point of view. The gap in the enso pictured above reminds me of the constant change in the World and that the model of the Universe in our heads will always be a little out of sync. That gap is the small price we pay for Our Gift of Sentience and all we accomplish that are miracles to non-sentient things.
Lastly, notice that the enso starts out as a rather straight line at the fat end, goes straight up along the left, and takes an abrupt turn at the upper-left. There’s something poetic about the smoothness of curve after that abrupt right turn. Is that kink the day you or your company realized the Oneness Nature of knowledge.
A Profoundly Sincere Happy New Year to You!
Faith and Patience to you,
Reverend Dukkha Hanamoku
Appendix
Miscellaneous Advice
- When learning about Functional Programming, don’t get caught up in the mathy stuff – monads, functors, category theory – or the zealous FP dogma. For the purpose of encoding a domain model, just F#’s type and function definition system seems to be the easiest on the eyes for the purpose of encoding a domain model for all to read. For developers, the familiar C# is actually well-equipped for writing FP-style code. The FP capability of C# may constitute a “pure” functional language, but hardly any real-world software projects are.
- If you wish to go hard-core with functional programming, you’ll figure out you need to learn category theory. But study Abstract Algebra before hitting Category Theory, especially if you have no idea what “abstract math” is.
- The technical advice in this post is most useful for the architect role of a project since that role sits between everyone, and the architect is a “technical” person. Of course, programmers are all at least part architect too.
Further reading
Domain Driven Design and Functional Programming References
I’m a big fan of Scott Wlaschin’s work combining DDD and Microsoft’s FP language, F#.
- Domain-Driven Design: Tackling Complexity in the Heart of Software, Eric Evans
- F# for Fun and Profit, Scott Wlaschin
- Functional and Algebraic Domain Modeling – Debasish Ghosh
Related Fishnu or BodhiDay Posts
- Bodhi Day 2019 – Awakened View – Complicated versus Complex.
- Corporate Dukkha
Fishnu 