Mastery of some skill is a type of project we all take on multiple times during our life. If we chart our progress towards that mastery, it follows an interesting pattern. There is first that long, slow, boring, frustrating, humiliating period of building foundational skills. During this time there is little growth and return on investment. Then one day, almost magically, it all gels, everything falls into place, and our progress skyrockets … at least for a while.
The beginning phases of mastering a skill is a long stretch of tedium with little evident progress. But you must have faith and patience that things will gel at some point.
It’s better to be expert at a few related skills than a master of one.
Learning is an iterative process, not a sequence of progressively more advanced topics.
The world is in constant change, so yesterday’s master is today’s dinosaur.
In this blog, we’ll dive into the pattern of learning and mastery as conveyed by the fahkee. The Eternal Fishnu says the Neanderthals realized that learning how to learn is the primary skill of sentient beings of Earth. They created a symbol for that pattern they call a fahkee. Fahkee was their word for “learning”.
The fahkee starts out like a sigmoid function (s-curve shown below), from the start at the left to the peak around half way. In contrast, the sigmoid function shown below points outwards to the right, straight ahead forever.
But systems on Earth don’t quite work that way. Life on Earth is a complex system of interacting phenomena, each playing a part in regulating wanton growth. The nature of Nature is its self-policing through a a web of interaction. Otherwise the world we know would be a big ball of mud.
The fahkee illustrates that as it continues on from the peak of mastery into a decline. It ends pointing slightly downwards suggesting a slow but eventual end.
The fahkee – like linear lines, hockey sticks, and s-curves – is a 2D graph. Along the x-axis (horizontal, left to right) is the Path of Mastery itself. The further we go right, the further along we are on the path.
Along the y-axis (vertical line, top to bottom) is a composite value consisting of Master of the Art, Value towards your Spirit, Value to society. Mastery of the Art, of course, refers to your level of skill. Value towards your Spirit refers to your spiritual growth. Tony Robbins says, “growth is happiness”. Value to society refers to your “marketability” in the world of jobs.
There are distinct phases we pass through on the Path to Mastery. Starting at the far left is the first phase of mastery. This is a time of preparing, filling your mind from the fire hose. Obviously there isn’t much mastery. Without that mastery, there is no value to society. But maybe there is value for your spirit.
Moving right, we see the curve take an abrupt turn upwards. At this point, we’re prepped enough to actually practice what we’re attempting to master. When we’re properly prepped, our mastery rises along with value to your own spirit, and the value of that mastery to society.
Moving further right our mastery and value are at its height – pun intended.
At the right, the last major phase shows a decline in mastery/value. In a nutshell, the world changes over time. Things that were at one time very important and worthy of mastery will eventually fade into the past. For example, sword fighting was once a very practical Zen Art. As I often mention, software development is my Zen Art, and one that seems to be in more demand.
In the next section, we’ll dive deeper into the structure of the fahkee, the Path of Mastery. We’ll use what is roughly the belts of judo as labels for those phases. And we’ll use my recent learning experience with Apache Spark as an example.
My Apache Spark Path of Mastery
About three years ago I began my Path of Mastery for Apache Spark. Spark is the center of the universe of my specialty, Business Intelligence. It has the potential to integrate the major aspects of an enterprise system – databases, processes, data scientists and business folks.
White – The Very Beginning, the Noob
The first belt in most martial arts that use belts to signify progress is white. Standing at the beginning of the Path, you may not even have a reasonable idea of what you’re getting yourself into. But you at least have some idea for why you sought out the gateway to this Path:
Software Development: “I heard Spark is the thing to know.”
Judo: “I want to learn to defend myself.”
For all roads to mastery, there are prerequisites. This initial white belt phase of the Path of Mastery is for ensuring we meet these requirements.
For judo, this means you meet some level of fitness, you can fall properly, you know how to execute moves such that you prevent injury to yourself and others in the dojo, know how to properly warm up, you have some level of physical fitness.
For Spark, there’s the matter of setting up an environment to play in. There’s the matter of being clear about the value of Spark.
There’s also a matter of having at least somewhat of an understanding about databases and how we will interact with Spark. You need to feel at least comfortable with Python or Scala and SQL. If you don’t know any, you should probably take a sidetrack to another Path of Mastery to learn at least Python and SQL to an intermediate level.
Blue – Hello World
The journey to blue belt is the most tedious, thankless phase on the Path of Mastery. Many people give up before this point experiencing just tedium, seeing practically no progress. But you made it through the tedium, which means you’re well prepped to begin the real training.
For Spark learning, you’ve reached Blue when you’ve successfully executed your “Hello World” example. Your dev/learning environment is set up.
Standing at Blue looking towards Green, there is a cloud obscuring our view of what is yet to come. You don’t know what you don’t know – and at this point you don’t know much. However, you should know enough to decide whether continuing on this path is worthwhile.
At this stage of Blue, we must remember not to equate this early phase as being simple. In fact, Blue will take you from beginning topics through advanced topics several times.
For example, my Blue phase with Spark mostly consisted of going through four Spark courses, all starting from “Hello World” through a “make it real” project. Having a day job, it took at least a couple of months to get through each. Between Udemy, EdX, Youtube, etc, it was very easy to find fantastic courses.
You may ask, “If those courses were so ‘fantastic’, why did you need to go through four of them?” Learning is an iterative process. Think about having a trainer delivering a week-long course to your team, say on Dynamics 365. By Friday, you’re pretty good – you hit “Very Satisfied” on the survey. But if you don’t apply what you learned the coming Monday, by that next Friday, it’s vaporized out of your head.
It takes a few iterations for a skill to really etch into your brain. It’s not that different from carving a petroglyph into a rock canvas – one scratch isn’t going to do it.
Additionally, each instructor has a different take and teaching methodology. Seeing the same material from different angles creates deeper meaning. It reminds me of how our stereoscopic vision paints a literally deeper picture of what we’re seeing.
Instead, we’re used to learning through a sequence of topics, once, from one point of view. Each topic is checked off never to be seen again. We take a test at the end of the semester and that’s it. Most of it will be forgotten a few days after finals.
Multiple iterations of training on the same subject forms the more links with other subjects. It’s the quality of the relationships between things we know that forms intelligence. Those connections don’t happen after just one iteration of learning.
Towards the end of the Blue phase, you may even have a shot as passing a certification test. At the time of this writing, I have yet to take the Databricks Spark exam. Looking back at my Blue phase, I think I would have failed, but I’d probably come respectably close to passing.
It’s during the Blue phase that the bulk of people will quit. That’s because in this McGoogle world we’ve learned to seek instant gratification. The nature of the long period of little growth is a fundamental principle of Life on Earth. Therefore, it strongly suggests that the mindset of delayed gratification is a fundamental skill.
However, it’s OK to quit during this Blue phase if you’ve given this a mindfully sincere try. Your heart will know whether you’ve exercised the skill of delayed gratification enough to determine how important this is. At the very least, if your skill of delayed gratification has been exercised it was time well spent.
Some skills are critical enough where we almost have no choice but to endure this initial death march. For me, Spark is certainly is a skill for which enduring this phase and beyond is worthwhile.
Blue is the phase where “persistence” is the most important word. My most important advice is not to worry at all if it seems like you’re just banging your head against the wall. That’s what defines this Blue phase.
This is where faith and patience is key. Trust that the reward for your persistence is that it will all magically gel. It’s weird how that works. Trust that while sleeping, your 80 billion neurons are at working sorting it all out. One day you’ll be studying and realize something clicked. That’s when you’re at Green.
Siddhartha Gautama spent years seeking his Truth – suffering broken hearts, failures, and the end of many dead ends. Finally, one day he had enough and plopped under the Bodhi Tree – emaciated, completely broken … and awoke one morning looking at life from the other shore.
Green – Completed Bootcamp
At Green we’re at the point where we can start work on an actual enterprise-grade project at a junior level. This is where we “make it real”. Blue was about lectures and labs. Now you’re working on a live production system. In judo, at green you’re good enough to be worthy fodder for the brown and black belts and knowledgeable enough to fully benefit from their instruction.
As you look to the next level you’re heading towards, Brown, you see that the Path of Mastery takes a steep turn upwards. The persistence of the Blue phase pays off. The rate of Mastery and value accelerates. The pace of your Mastery picks up because you’ll be out in the real world now where you don’t have full control over the throttle.
We see too that the cloud of obscurity has lifted a bit. We can see that we’re standing at the bottom of a really steep slope. But the cloud of obscurity hasn’t lifted enough to see how high that steep mountain rises.
Although we’re ready to do some real work, we’re caught in the Catch-22 trap of needing experience to land a gig while having no real-world experience.
In my case, when I reached the green level with Spark, I was also brown or black with a few other adjacent technologies. For example, SQL Server and Azure Data Factory. In fact, these days, it’s pretty tough to be effective with just one skill.
If you can’t get onto a project to apply your new skill, there are a couple of options.
I’ve always enjoyed practicing my new skills on my own ideas I usually don’t get to explore at my day job. Although it’s a very effective way to hone your skills, there are aspects of working on a fully-staffed, fully-managed project that this garage band approach doesn’t duplicate.
Additionally, at this point, if there is a certification, this is the time to get it.
Brown – All Grown Up
At this point, you have completed a fairly hefty real-world project. At this stage of Brown, you’re a bonafide practitioner.
The focus of the Brown phase, at least for my pursuit of Spark mastery, is to get a few more projects under my belt. Preferably, these projects will be different from the ones worked on during the Green phase. The idea is to explore other use cases for Spark, deeply exploring other aspects, filling in the holes.
I feel at the time of this writing this is where I am – brown. However, I do have a few more projects under my belt covering a diverse array of domains. I think I’m about half way between brown and black. Part of the reason I don’t think I’ve reached the expert (black) level yet is because my work has spread out across a wider scope.
Looking upwards to black, I can see there’s still a cloud obscuring the visibility of what is at the top of the Spark world. But in my role of data architect/engineer, I feel I know Spark well enough at this point. At this time, I don’t think becoming a Spark Expert or Master is the best way to progress as a data architect/engineer.
Black – Expert
At Black, we’re just about there, pretty near the summit. We still have a little way to go, but the top is well within reach. We’re confident we can digest whatever awaits us further on the Path of Mastery.
We’ve gone through more than a few projects, diving deeply into a variety of domains. Our sphere of knowledge have no wide holes, but many little ones we’ll patch up on the way to Red. Almost as important, we also have a sense of the limitations of the product. At this point we could lead a very high-end, innovative project.
It’s important to remember that the slighter incline at this point in no way suggests the intensity of the work eases up. In fact, it’s not really a “good” thing. It means that gains to your mastery and value are tougher to come by.
At Black, there are no clouds to obscure the rest of the path. For the most part, we’re aware of most of what we don’t know. But we’re wise enough to keep our minds open.
The section of the Path to Mastery from Black to Red is a long road in itself. In judo there are eight degrees of black before hitting red at 9th dan.
This slight but long part of the path involves digging into some things a bit deeper. For example, for my case of Spark:
Dive into math to a deeper level than would normally be necessary for a software developer.
Learn more about some machine learning hyper-parameters.
Develop systems for troubleshooting, reference architectures, etc.
Keep up with the evolution of the subject area – not just Spark but the Cloud, AI.
That last point is very important. Things are always changing. As the expert, you’re now the “tech lead”. It’s your responsibility to ensure development follows the changing best practices. Similarly, in judo, black belts take on more teaching responsibility.
As a data architect/engineer, this is probably the farthest I should travel on this path. There is a huge world beyond Spark for an architect. That’s probably true for most fields.
The big consideration on the Path of Mastery is that at this level, the expended energy versus return on investment ratio isn’t very good.
A mentor of mine long ago taught me that I have a tendency to shoot for an A+ and it’s much better to be an A at many things. With all the effort I expended back then trying to be “the best programmer”, she could become competent at many adjacent skills. She would have what Scott Adams calls a “skill stack”. At the time I thought it was the stupidest thing I ever heard.
Red – Mastery
At Red, we are an “A+”, a Master. The subject is now ingrained in your DNA and you’ve become the thing you’ve been mastering. A good sign that you’ve reach the Red level is that you have very few peers.
In judo, red belts (9th, 10th dan) are of course quite elderly. It takes decades to reach these levels. These ranks aren’t awarded for winning tournaments, but for their contribution to the art. The equivalent of red belts in the Spark world could be the few major originators and continued contributors towards the evolution of Spark.
The Path of Mastery from this point heads downhill. But it’s not your mastery that diminishes. That’s yours forever. Remember? You’ve become that thing you mastered. However, the value to ourselves diminishes as we no longer grow. But more importantly, the value to society may diminish as the skill antiquates.
I don’t think I’ve ever reached Red in anything. So what am I doing talking about mastery when I myself don’t think I’ve ever achieved it? Maybe there’s no such thing as mastery for humans.
In all modesty, I think I’ve reached the mid-black dan levels at least a few times over my long 40-year software development career, my 60-years of life. Having taken at least a few skills to an expert level, I’ve been close enough to see what mastery looks like, what it takes to achieve, and what it means to get there. I’ve also had the privilege of knowing and observing many people who were very advanced experts – more advanced in their realms than I was at mine.
One problem I’ve faced is that my business of software development changes faster than most other fields, so it’s nearly impossible to spend enough on time on a subject to achieve such mastery.
Some would say that’s nonsense. “Coding is coding! If you know say, Java, you can pick up C# or Python or Ruby.” That’s true to an extent, as I’ve shared in my blog about my career reinvention. But each language has its own syntax, its own environment, its own strengths and weaknesses. There are countless idiosyncrasies that you need to stumble upon over the course of thousands of hours. Mastery exists in the entire tree of knowledge – the roots, trunk, and those very numerous high, little branches.
A changing world is faced by all people of all jobs. Mastery is mostly a target to shoot towards. Perhaps the definition of mastery should involve some element of elusiveness.
Maybe for humans, there is no mastery of skills. Our shtick as the human species is our ability to adapt to the ever-changing world. Lions and tigers and bears are indeed masters at being lions and tigers and bears. They don’t even need to worry about mastering anything because they are already that thing. But they’re not good at crossing oceans and flying into space and farming food and fixing our physical wounds.
In a world of constant change, the only thing we humans must master is the Path of Mastery itself. Mastering the Path of Mastery is the foundation of the Teachings of the Eternal Fishnu captured in the Three Zen Stories:
The Empty Cup – We must be open and sensitive to changes.
Is That So? – We must accept that change is forever.
The Man with the Bag – We travel the path, with focus on where we’re at.
In Part 2 we explore letting go of our cherished mastery when the time comes.
The enso represents the cycles of the Universe. Our lives are an enso. The cycle of learning new things is an enso. The food we eat, digest, then shit is an enso. Our friendships are an enso.
It is the phenomenon of cycles that upholds just enough order for the Universe to Be. Too much order and it’s just a rock. Too little order and it’s merely horribly boring chaos. With just the right amount and kind of order, the interacting patterns of Life emerge.
The end of each enso cycle must be a return to the beginning, so the cycle plays out again. The gap of an enso illustrates that the cycle may be over but the journey is not.
However, although the beginning and the return to the beginning look pretty much the same, they are always at least little bit different. As Heraclitus observed, we never step in the same river twice. That little difference means the beginning of the new cycle will be familiar but different. It will be familiar enough so can make use of things we learned before. We can also notice what things we no longer need and put them aside. Sometimes someone else on the journey may find it useful.
The Universe evolves, it’s in constant change. It evolves through endless links of cycles. Does an unchanging Universe even make sense?
Cycles are processes of phenomena coalescing from nothing then disintegrating into a soup of no thing. This soup of no thing is the seed from which the next cycle coalesces into phenomena. Our Life on Earth exists as we experience it because it evolves in such linked cycles.
The most wonderful enso is the pink donut above. It is an abundant enso. It’s really the enso for all our lives. But we kind of see the enso as just the superficial sprinkles, disregarding the bulk of the donut that is the blessing of this dynamic Universe we find ourselves in.
The course of each of our lives is an enso, but it’s in turn composed of little ensos and interactions with other ensos. For each of those ensos, qualities change over its course. For all sentient creatures of Earth, some parts of the course we deem pleasurable by our animal brains. But the bigger part of its course would be deemed unpleasurable by our animal brains, ranging from uncomfortable to seemingly unbearable.
The entire point of Zen is to transcend how the human animal brain sees only problems to be solved, down in the weeds here on Earth. That’s as opposed to the frame of what is really “out there” that our human minds don’t see. Think of the bacteria in our guts dying and beginning life by the billions every day. They quickly evolve in short, interacting, chains of birth and death as an intricate part of the bigger picture of a human life.
I sign off each post with “Faith and Patience to you”. But there’s a little bit more after that. It’s the mantra I say to myself when the green clover sprinkles of my pink donut enso are too sparse: “Faith and Patience keeps me calm so I can focus on the moment … then the next moment … then the next moment.”
Each of us, and every other living creature for that matter, is an amazing assemblage of atoms where every one of those atoms plays a specific part. All of life is precious beyond the ability of our human brains to adequately express. Meditate on that because that’s hard to remember when we see things from the perspective of normal human life.
The Eternal Fishnu has witnessed humanity heading in the right direction, the direction towards Oneness with the Universe, for hundreds of thousands of years. Especially the last couple hundred. And we still are!
My zen art as a software developer hones that model of the Universe safely housed in my headthrough the practice of modeling lesser domains than the Universe itself.
We are sentient objects somehow flourishing in the impermanence of a relentlessly changing world. Our brains are the model of that world which determines our interactions with the world. If the model doesn’t evolve along with the changing world, that disconnect results in suffering.
Likewise, a business flourishes in a business world that is in constant change. Without acceptance of that reality when building software, a business is destined for a life of dukkha. That is, there will be a gap between how it conducts business and how the world really is.
This blog may sound more like an IT whitepaper than a piece on Zen. Towards the last quarter, it can get a bit too techie. But this post is at the heart of what Zen from my software developer’s perspective is all about.
However, as it is with enlightenment on Bodhi Day, you will unlikely will walk away from this post armed with belt of silver bullets that will lead your software project to success. What I hope you walk away with is that this post is really a parable of the Eightfold Path, particularly the business equivalent of Awakened View.
Situation – Modern software projects are implemented in complex environments, but are managed as if they are merely complicated. So many projects ultimately fail. Even if they succeed, the win is under-whelming.
Background – The ability of sentient people to invent, design, and strategize is based on a model of the world we maintain in our brains. Our brain evolves along with the evolving world it’s modeling. Likewise, a business is an entity that must continuously strategize. Shouldn’t it have an evolving, integrated model too?
Assessment – For a software development project, if we turn our attention from merely satisfying requirements towards the ability for businesses to adapt to an ever-changing present, we’ll close the gap between what we build and reality.
Recommendation – Taking the time to thoroughly model the subject of the software being built, its domain, mitigates show-stopping misunderstandings. A methodology for building such a model is Domain Driven Design implemented with just the type and function definition system of a functional programming language – understandable by non-programmers with reasonable effort.
By the time you get towards the end of this post, you’ll see this is an abstract about a post on abstraction!
Building a Complex System in a Complicated Way
My biggest current challenge as a software developer is conveying to my co-workers that the complexity of even moderately-sized software projects are now beyond the capability of being managed solely through human effort. Today’s enterprises have so many intertwined moving parts and external relationships – people, machines, and other organizations – that even a small software project comes with so many external dependencies that it’s really much more involved than it appears.
During the days before the well-functioning networking of the Internet, the scope of software projects were well within the domain of a single business process, a department. The limitations of hardware and software tools back then naturally limited what could be accomplished. Additionally, a far less “global economy” limited what needed to be done.
The limited scope meant that software was still in the satisfyingly deterministic realm of complicated. There weren’t webs of dependencies between many objects with rules of their own that form complex systems. For example, dependencies between a number of systems on different hardware platforms, different software frameworks, policies of various companies, across clusters of servers, in different countries with different laws … on and on. Back then integration of software consisted of humans manually typing data from one system to another.
Typical software projects are now in well in the realm of complex. As an example of complicated versus complex, an F-22 fighter itself is a complicated object. It has a ton of really sophisticated parts, but it is still deterministic. The operation of an F-22 is complex, involving much more that the aircraft and the pilot. Complex software, or any other complex system, can’t be managed through tactics that work for merely complicated solutions.
Actually, large software projects have been beyond the paradigm of traditional top-down control by a small team of managers for a long time. It’s not like this issue hasn’t been addressed over the years. Around 2005 or so, Unified Modeling Language (UML), was one of the big over-hyped buzzwords of the time. Domain Driven Design was born in 2005 with the publication of Eric Evans’ book. There are the “user stories” of the Agile Methodology.
However, at of the time of this writing, most of my colleagues have heard of UML but don’t know much about it, maybe heard of user stories (despite much experience with Agile), and hardly any have even heard of Domain Driven Design. Yet, software projects at best succeed with under-whelming enthusiasm, and worse, years of really tough, expensive maintenance.
The challenge isn’t understanding the value of gathering requirements and documenting use cases. The challenge is to integrate these fragmented chunks of information into a coherent, integrated, fully-connected view that is understandable by all. Then to convince everyone that this obviously tedious task is worthwhile, not just some fluffy activity.
Alternatively, we could continue fighting complexity through tightened control. Which just makes things more rigid and paradoxically more complex. Rather than trying to conquer complexity in this complex world, we embrace complexity. That’s what Buddhism is all about. We embrace complexity by learning to adapt to an ever-changing present.
Automating Real World Processes
Software is about automating processes. That is, it drastically enhances what we currently do so we can focus on the more creative, more intellectually challenging aspects of work – the thing we humans currently do better than machines. Occasionally it can do things we never could do, even with all the time in the World. In either case, software augments our human activities.
The problem is that since the world is in constant change and software as it is today is incapable of self-evolving as how we humans are able to evolve by learning new skills. Or at least, software is incapable of self-evolving without making a big mess of things. Heard of “herding cats”? This is more like herding toddlers as big and mean as bears. Learning (evolving) is enough of a struggle as it is for we sentient humans – which is the main idea the skill of Zen – without the additional chore of manually evolving our software.
Yes, software too is cursed with dukkha. Meaning, the programming of the software doesn’t match reality. Except the non-sentient, non-evolving software doesn’t actually suffer pain from the turbulence it causes. It just tosses out an error message and passes the suffering to us.
Models versus Requirements
How can we help our software systems to evolve along with an evolving reality? What if we use our human intelligence to encode and link the large collection of narrow-scoped fragments of the world that we collect from lots of users into a single, connected, wider-scoped model? And these models should be readable to a all that are involved, human programmers, human non-programmers, and machine.
This post primarily address why we need to build a model of an enterprise in order to effectively engineer complex software undertakings. What is it about the way we currently approach large, modern software projects that result in under-whelming results or even failure?
A full explanation of the methodology of how we build this model is well beyond the scope of this blog. For the moment, I would like to point out that there is at least one viable methodology that has been in practice for over a decade – Domain Driven Design, which I previously mentioned.
It’s imperative to understand that models are not the same thing as requirements. Requirements are about what we need from the solution. The model is about how the domain works.
Requirements are relatively easy to gather. We just ask. However, we programmers don’t know what the business folks know and the business folks don’t know what we know. So requirements we’ve gathered can’t be all that good.
Now, it probably sounds very obvious that we need the requirements of what we’re building. How can we build something without knowing what it’s supposed to do? But this model thing probably sounds like another huge activity that has value but not enough to warrant the large investment of time and resources. “Requirements” always sounds so pragmatic, model sounds superfluous.
“Do we need that model? We can piece that together from the requirements.”
“It sounds really tedious. Do we really need to know the in-the-weeds details?”
“Sure it helps, but the users and test scripts will tell us if it’s not working.”
“We don’t have the time for ‘nice to haves’.”
The problem is that your understanding of what is done in a domain is a model you built in your head, and that version exists only in your head. Others on the team don’t have the same experiences as you, so the models in their heads won’t be like yours. Surely, you can communicate what you know to the others. But human-human communication isn’t really an exact science void of ambiguity.
The next two sections describe the model of the world in our brains and the resulting dukkha from the disconnect with the real world, followed by the counterpart for the “brain of a business”.
Our Brain’s Model of the World
Every thought we have and action we take is based on a model of the Universe that has been evolving in each of our brains since the appearance of our first neuron. This model is a network of rules, cause and effects, and goals we’ve been exposed to over the course of our lives. It empowers us with the ability to analyze a potential action in the safety and privacy of our heads before taking physically irreversible actions in the world.
The evolution of our brain’s model of the Universe is driven by rules encoded in our DNA, what we passively learn, what others train into our heads, and conclusions we make for ourselves (based on what we’ve learned). It’s the blueprint of how the world works and how we interact with it. However, as I’ve written before, this model is a pitifully inadequate model of the Universe.
There are two primary reasons the model in our heads is pitifully inadequate as a model of the Universe. First, we cannot possibly know everything. As Ringo said, “Only Everythin’ can know Everythin’.” The only adequate model of the Universe is the Universe. In the normal scope of our human animal lives, we’re not everything, so we don’t know everything, so we compromise – if we can’t know everything, we can at least know what seems important to us for the moment.
Therein lies the second reason; everything is constantly changing, so what’s important to us for the moment is a moving target. Therefore we learn; adapt by reflecting the present and mending the disconnects between what we think and what we’re experiencing. Most animals have some ability to learn but none like we humans can learn, at least on Earth.
However, unlike lower-sentience animals, learning for humans is about adapting to a perpetually changing world – in addition to building the skills we need for a role. Lower-sentience animals mostly adapt by passing on genes that seem to offer competitive value to their offspring. We humans can adapt our behavior and environment to suit present needs.
Dukkha arises from the disconnect between a change-resistant model of our world in our heads and the relentlessly changing world that is exceedingly bigger than any of us.
This model of the Universe we hold in our brains is so important that as obvious as it sounds, it’s worth the reminder that without it, the physical object we know as ourselves does not exist within Life on Earth anymore.
A Business Entity’s Model of its World
Businesses are very much like organisms. I’d argue that businesses are organisms because they are run by humans – where each human is the equivalent of a neuron in our brain. Except that a neuron isn’t nearly as opinionated.
Both individual humans and businesses are made up of thousands to many quadrillions of parts with a surprisingly large numbers of roles. For humans, it’s not just a matter of a couple hundred bones and a couple dozen organs. It’s trillions of cells and bacteria and thousands of different proteins.
For a large corporation, it’s not made up of a few dozen departments. It’s tens of thousands of employees, working at a few hundred locations, across thousands of unique roles, with thousands to millions of physical assets, millions of electronic assets. At least a human has a brain that organizes all of that in one place, where it’s well protected by a thick skull.
Both human animals and corporations compete in complex ecosystems against predators and competitors for limited resources (prey and customers). To succeed, humans and corporations must perform such that their whole is greater than the sum of their parts. For humans, that “whole” is sentience.
For business “organisms”, there isn’t the level of coherent sentience we humans enjoy, but there is a seed of sentience. We multi-celled organisms were once just a motley collection of cells, which over hundreds of millions of years coalesced into the coherently sentient beings we are today.
For corporations, success involves more luck than executives would care to admit, and a whole lotta dukkha. The knowledge of a corporation is fragmented across all employees, dozens to hundreds of software applications, millions of documents. The parable of the “Wise Men and the Elephant” is a ridiculous understatement. It’s more like “The Cells and the Human”.
What About A.I.?
Before continuing onto an exploration of the current state of software modeling, I need to point out that none of this post is about Artificial Intelligence.
Today’s A.I. can dig through massive databases for answers to posed questions with mostly rote algorithms taught to it by people. Today’s A.I. is far from capable of discovering linear algebra and calculus on its own. It’s far from organizing solutions for ambiguously defined problems. Strategic planning is still well within the realm of human brains.
Someday an A.I. or highly augmented people may render this blog moot. Shakyamuni Buddha’s audience of pure humans, will no longer exist, and his teaching will fall on deaf cyborg ears. At this point, what we need and can obtain is a model of the system that we can all look at (again, human and machine participants) and “observe” in a minimally ambiguous manner.
Current State of Software Modelling – Heap of Documents
As of today, whether or not a business possesses such a model, businesses somehow do plod along. All over the world, billions of people show up to work, work gets done, money is collected, money is paid, the doors open the next business day. So again, is this model thing just a “nice to have”?
Businesses do actually work without a model because there is a model. It exists in fragments distributed across all the employees. It’s often called “tribal knowledge”.
It’s long been fashionable to say, “Most of an enterprise’s data is stored on thousands of Excel spreadsheets on hundreds of desktops and laptops.” It’s probably true. But even worse, most of the knowledge of how things work in the enterprise is stored in peoples’ heads. It’s much harder to transfer that tribal knowledge brain to brain than to emailing a spreadsheet.
If a long-time employee leaves, the business takes a huge hit like the hole in a ship made by a torpedo. It’s usually possible to heal the business by piecing together fragments held in the heads of other people and assorted documents somewhere. Sometimes it can’t be reconstructed, so things need to be reinvented.
If we’re lucky, someone’s tribal knowledge is mostly redundant or easily re-constructed from a few readily identified sources. It would be great if someone, a small team, or something held the big picture at any given time. But unless the enterprise is a really small organization, no thing has that Big Picture.
It’s easy to counter that someone does indeed have the big picture – the CEO. But don’t mistake a 50,000 foot view of an enterprise as knowing everything. The CEO trades-off detail for scope. One way or another, no one person sees the entire elephant.
Requirements from Fragmented Sources
Before we can build anything we need to know what we’re building. Software, temple, Space Shuttle? Once we know what we’re building, an ERP system for example, we’re at least pointed in a direction and can now bring in experts to dive into requirements gathering.
Some teams are better at requirements gathering than others. Some teams dig deeper into the requirements than others. Some still jump straight into coding – often something no one yet asked for.
Requirements gathering is an iterative process with three stages.
The first is an overall idea of a problem we’re trying to solve. At this stage, we need enough of an idea of what we need so we know if it’s even possible, and if it’s really the right approach to solving our problem. For example, if I have a legal problem, I can’t expect to show up at the attorney’s office and expect to walk out the same day with a resolution. My reasonable expectation is the attorney will accept or not accept my case.
The second stage is where we dive deeply into the design and planning of what we’re building. We’re pretty sure this is the right approach. Continuing with the legal problem example, this is where my attorney has thoroughly researched the facts, formulated a strategy, and we have a court date.
The 3rd stage is really on-the-fly discovery as we actually get into the build. Ideally, there would be no 3rd stage, and we often over-confidently act as though there isn’t one. We delude ourselves into believing everything was fleshed out in the 2nd stage.
But complex systems are rife with unknown unknowns. Hopefully, whatever gotchas we discover in this 3rd stage are minor enough where we’re not hurled back to square one. To conclude the legal problem example, throughout a trial, things will come out that require altering the plan. However, if by some miracle a trial or project goes exactly as planned, we believe more in our planning skill than luck.
Software is a virtual thing. That’s a good thing and a bad thing. It’s good in that as an engineering project errors are much more forgiving than say for building a bridge. If software breaks, in most cases, a ticket is filed and someone fixes the bug. If a bridge collapses, people may die. And, re-building a bridge is much more expensive than rolling out a hot fix.
However, in the context in which software is built, potential problems aren’t as obvious as it would be for something built within the real physical world. The engineers and architects of bridges design to the model of physics. For a bridge, a physical thing, the requirements are more straight-forward, more in our face. The thoughts of our brains are just one step removed from the physical actions we execute in reality. But software is at least two steps removed from the reality of the physical world – jumping from programmer, to software, to the real world.
The Dukkha of Requirements
For the most part, when building software, requirements are gathered and organized by people who don’t actually do the work that we’re attempting to automate or augment. These are typically “white collar” folks such as managers of the people who do the work or business analysts who may not have ever lifted a shovel.
The dukkha of requirements is the disconnect between what the software does and how things actually are.
I need to label the two classes of workers from which requirements are gathered. As mentioned, there are the variety of “white collar” managers, business analysts, architects, and programmers. Managers can include those who manage the folks who actually do the work as well as “managers” of software projects such as the project manager, product owner, leads, etc. These folks are highly educated, creative, but at least one step removed from the “real” work.
Second, there are the ones who actually do the work the software is meant to automate or augment. They are the ones “in the weeds” who physically touch the details that are lost in the abstraction of requirements gathering and chains of translation. However, the closer to the weeds the more fragmented the knowledge.
The Concretes and the Abstracts
For the sake of brevity, I’ll refer to those who actually do the work as “Concretes” and those who don’t currently do the actual work being automated as “Abstracts”. I hope those terms don’t offend anyone. Since none of us knows everything, all of us at different times and places play the role of concretes or abstracts.
Considering the mix of abstract and concrete folks, there are at least three ways that requirements are gathered. All have their pros and cons.
The Way It Really Is
The concretes are the only ones who really know what goes on in the real world. The concretes are able to articulate their little piece to full fidelity. However, the problem still remains that the concretes can’t see the big picture. The concretes are the blind men feeling different parts of the elephant.
To clarify, concretes can be line workers “in the weeds” who only see a small part of the entire business. Or they could be the CEO with the 50,000 foot, Big Picture View – like being able to see the larger state-wide map, but not the map of her neighborhood.
The knowledge of the concretes is imperative for the design of software that reflects reality. For software to be in sync with the real world, we must know of the real world. However, a design authored purely by concretes very much risks disconnects between theory and reality due to their fragmented knowledge.
The Way We Think It Should Be
Concretes and abstracts will have opinions on how things should be, apart from how things currently are, because we must always yield to better ways. Even if they are in agreement, the fact remains that the work of the concretes is still manifested in a concrete world that currently isn’t doing things “the way it should be”.
If we design software to the way things should be, as opposed to the way things are, the real world of the concretes will need to evolve into that software reflecting the idealized way things should be. Of course, part of the software rollout will include plans for training the concretes to the new way as well as re-tooling the equipment and processes.
It does seem to make sense to build a software system towards the way we think things should be, along with a rollout plan to mold the real world to the idealized world. We are after all sentient humans gifted with the power to shape the world to our liking. However, writing software to an idealized world that doesn’t yet exist is a bold, maybe arrogant, all-in move. Do we assume there aren’t any unknown unknowns?
The Way We Think It Is
The dominant contributors to a software application’s design typically never actually did the work they are tasked with automating. But they probably have some notion of the nature of a domain. For example, programmers, architects, and project managers probably have a good idea of how healthcare, restaurants, or insurance companies work. However, aside from maybe a stint at McDonalds during high school, they unlikely were ever nurses or insurance agents.
Even though we engage subject matter experts (SME) culled from the ranks of those who actually do the work being automated, they typically are interviewed by the abstracts who mostly ask the questions, then pass those answers to the architects, who pass those answers to the programmers, who encode it in software.
So What Do We Do?
As a starting place for having our cake and eating it too, let’s start with this ridiculous unicorn fantasy: The people who actually do all the work, also happen to be crack software developers, who are infinitely patient. They get into a room for a couple of months and hammer out the perfect code.
Easy, huh? Such people do exist, but even Elon Musk can’t do everything. Back in “the day”, the 1980s, when software was merely complicated, not complex, I knew many professionals such as dentists and lawyers who learned how to program and founded successful software companies. Those days are somewhat gone. Things are very complex and low-hanging fruit is hard to find.
The Domain Model Pidgin
In these days where software is complex, it’s not possible to find people who can play all roles. So the roles must be distributed among a number of specialists.
Each of those specialized roles all have their own language. Bob speak nursing, Alice speaks architecture. But Paul speaks “Business” and sort of understands architecture and nursing, so Paul is the intermediary between Bob and Alice.
The problem is that communication risks being lost in translation from Bob to Paul to Alice – then back from Alice to Paul to Bob. The left side of the illustration above shows that communication from those who actually do the work takes a few hops before it’s encoded in software – to the Business Analysts to the Architects to the programmers.
But what if we could bridge those gaps with some sort of common language? Something we could all look at and understand to a workable level. Some sort of pidgin.
When people from different cultures are tossed together, for example Hawaii and New Orleans, they don’t have a way to communicate. They may initially just stare at each other. But soon enough a rudimentary language emerges that they all understand – a pidgin. The syntax is terrible, the vocabulary is sparse, but it’s enough where they get the workable gist of what each other is trying to say.
Over time, that pidgin evolves. A formal syntax emerges. New words are formed to fit new concepts unique to an emerging culture blended from that initial mishmash of people. That pidgin becomes a “creole” – a bonafide language. That language is the model, the expression, of a brand new culture. And it will continue to evolve along with the evolution that culture. Without that evolving model, there is no culture.
New State – Reflecting the Evolving Present
Businesses are all unique cultures. They have unique strategies, serving unique clients, and staffed with unique humans. Like the cultures of Hawaii and New Orleans, a model emerges that captures the culture and evolves along with it. However, as stated above, that model doesn’t exist in one place. Rather, it’s distributed in fragments among the employees and documents.
Is it possible to encode an evolving model? If not the entire thing, at least encode the domain of a software project?
The sophistication of such a model lies somewhere between a lifeless collection of project documents and genuine Artificial Intelligence. Unlike our real intelligence, this model will have practically no ability to self-learn.
No matter how we gather requirements, the software we develop will reflect a distorted image of reality to varying degrees. But that’s not the worst problem. The worst problem is the world is constantly changing. Similar to how house doesn’t clean itself up, software doesn’t evolve itself, we humans do that chore (for now) – and it’s a chore.
As the world changes around us and the software we use to automate our processes lags behind or even remains static (no new versions), the software limits our agility, our ability to compete. We find ourselves molding our real-world business processes to fit the software etched in silicon.
Remember, by “model”, don’t think of model airplanes, model trains, or even business models. Such models are just snapshots. Models are usually just shadows of the real thing – capturing the general shape, but masking away the details. What we’re looking for is a reflection of the real thing. The model we hold in our brains evolves to reflect our understanding of the world around us.
Similar to the brain reflecting our current understanding of the way the world works, this model reflects our current understanding of how the business works. The methodology that I feel best encapsulates this modeling is Domain Driven Design (DDD). DDD refers to such models simply as “domain models”.
I call this model a “Domain Reflection Model”, DRM, to better capture the notion that the model is meant to be a reflection the ever-evolving business.
Domain Driven Design
Domain Driven Design is a huge and deep topic. It’s also rather counter-intuitive in the software development world at the time of this writing. However, it’s actually very simple if you embrace the counter-intuitive aspects. With apologies to Sensei Eric Evan and other established DDD practitioners, here is my perhaps slightly off-kilter take on a few major points I feel are relevant to this post:
Thorough upfront vetting of a business’ processes illuminates unseen potholes – the unknown unknowns. Such an exercise will be a severe pain in many ways, but a well-lit road makes for better odds of getting home safely.
Sometimes it’s best to let apples be apples and let oranges be oranges. Instead of forcefully conforming one to the other, let them work as they are and handle any necessary translation as a separate translation component.
Communication between folks of different project roles are hampered due to confusion caused by the insistence of folks using the terminology of their respective fields. With DDD, people of all roles defer to the language of the concretes, the people who actually do the work. That vocabulary is at the finest level of detail since it’s the closest to what really happens.
A Few Major DDD Definitions
Domain – This is the subject of the software being developed. If we’re building software for servicing customers in a restaurant, the domain involves all of the processes from seating to billing along with all the things (waiter, customer, orders, etc) involved. However, be cognizant of the fact that servicing customers in a restaurant doesn’t exist in a vacuum. There are processes for things like procuring ingredients (no ingredients, no food to serve) and payroll (tips are part of the payroll process).
Domain Model – This is the documentation of the domain, the model, the prime object of this post. Whatever the method of documentation, the primary piece is the thorough mapping of process flows within the domain. It has three primary characteristics:
It’s a model that folks across all roles can “watch” and so agree upon what’s going on.
It’s a living structure that reflects the evolving present.
Aspects of this model can be projected into many fundamental things – such as documentation, code, visualizations …
We document the process in excruciating detail. For example, if we note that the waiter picks up an order from the kitchen counter are the results simply that the food will get to the table? The waiter could trip on the way, the waiter could find that it’s the wrong order, the customers could be gone. This is critical since this is real life. Each of those possibilities results in another process that the software must somehow handle.
Of course, we shouldn’t go overboard. Listing that a possibility is the police will storm into the restaurant is a bit too far. Those sort of overboard errors could fall into the typical catch-all “Unknown Error – See your Administrator” … haha.
Bounded Context – In software, we attempt to abstract concepts to remove the fuzzy details in order to simplify things. That’s good. But sometimes forcing a square peg into a round hole isn’t worth the trouble. Or at least, we can punt forcing them together to a later time. For example, in a restaurant, the process for serving a sit-down customer is different enough from drive-through where they should be handled separately.
Ubiquitous Language – This is the language within a bounded context. It’s the language the people who actually do the work use – not the jargon or the programmers, project manager, or that of another domain. We depend on the concretes, the people who actually do the work, to provide the knowledge as close as possible to reality. Therefore, we need to speak in their terms.
At this point, especially with phrases like “thorough upfront vetting of the processes”, it’s easy to imagine the reader horrified with visions of being mired in the quicksand of over-analysis – the so-called “analysis paralysis”. I don’t think this is the same thing.
Vetting of the processes is a learning activity while the kind of over-analysis that can mire you in quicksand is a decision-making process. The latter is especially true when choosing the lesser of two evils. We’re so afraid of making the wrong decision that we are caught in the limbo of analysis.
Analysis Paralysis is seductive because analysis usually seems more like “real work” than learning. It’s easy for people to assume you already know what they know about the business.
However, it’s not that people don’t already go through extensive discovery and mapping phases. The problem is that these discovery exercises, which can go on for months, is highly fragmented. Meetings are held with various parties, piles of information are gathered, people (usually abstracts) disappear for a few weeks or months, and then return from the mountains with The Plan.
Domain Driven Design Using Functional Programming
Development of a domain model itself may get all the humans of a project aligned with a common understanding of what is being built. However, what brings “Pinocchio” to life is to encode the domain model into something the computer pieces understand as well.
Of course, I’m not suggesting we provide “code as documentation”. Although computer program code does unambiguously describes the process, that’s a very silly notion. Even programmers have a hard enough time reading their own code a few weeks after last looking at it. So we need to do better.
One option is if we can build the domain model using the type and function definitions (only the definitions, not the gory code implementation) alone we can end up with what is genuine code. Of course, “code” is “understandable by computers, as well as programmers and architects.
But it can plausibly be readable by the “non-technical” executives, business analysts, and people who do the actual work as well! The type and function definitions of a programming language should be readily learnable by anyone who knows how to spell “Excel”.
It happens that the strongly-typed type system and function definition syntax of Microsoft’s F# is such a language. Thorough discussion of DDD and F# are well beyond the scope of this blog. But there are wonderful references listed below my signature. In particular, there is the fantastic work of Sensei Scott Wlaschin.
However, even though a model encoded using the F# type and function syntax should be plausibly readable by anyone involved in a software project for whatever role, we can easily express the model into whatever format works best for particular roles.
From the Domain Reflection Model encoded with F#, we can write simple code transcribing many views. For example, code to transcribe the DRM to a network view is very simple. For example, simple C# code that formats Cypher statements loading into a Neo4j graph database. Simple C# code could “normalize” the DRM into a relational database schema or Excel sheets.
The transcriber of the domain will be the architect, the person traditionally in between the programmers and everyone else. For the architect, the coding aspects of transcribing a model to F# types and definitions is child’s play.
The architect can hand over the DRM to the programmers as a genuine blueprint, DNA. For the case of F#, these are .NET classes with names matching the ubiquitous language into which programmers unambiguously fill in the implementation.
Now, I understand that the notion of DDD coupled with FP probably sounds insane. Both turn the current corresponding ways of thinking, upside down:
Spend a ton of time building a model we all can look at before even starting? It can sound very indulgent.
Very roughly speaking, the difference between the very familiar (at the time of this writing) object-oriented programming and functional programming is like focusing on the nodes of a network, then instead focusing more on the characteristics of the relationships.
Functional Programming should be more innately in tune with a Zen practitioner. Object-oriented centers around things. Functional programming centers around the transformations between things. There are no things, but their is constant change.
Data Governance and the Continued Evolution of the Domain Model
The Domain Reflection Model begins its life reflecting the business at the time of the project which gave birth to it. It needs to start somewhere. But if the DRM doesn’t evolve along with the changes of the business ecosystem it’s reflecting, it becomes like our brain stuck in ’82. All the business’ energy is spent in futile attempts to bring back those glory days instead of bravely flowing towards the future.
Therefore, a DRM should be under the authority of a Data Governance Office. It should be a first class citizen, right up there with Master Data Management, Security, Compliance, Data Quality, Business Rules, and Metadata Management. It should be governed by DRM Stewards.
Process Management and Project Management folks should work tightly with DRM Stewards to leverage the knowledge in the DRM and evolve the DRM to continue to reflect the current mechanisms of the business.
Like the data stewards of Master Data, the DRM should be maintained by “DRM Stewards” who update what is known of the processes.
Domain Driven Design and Business Intelligence
DDD is usually associated with software automating processes that can be well-defined within reason. Those are the the actual activities of a business such as manufacturing automobiles.
Business Intelligence is about a special business process. It’s about the very nebulous process of learning – specifically learning about how to optimize the processes or at least to evolve the processes to match the changing world.
Indeed, Business Intelligence is really supposed to be the “brain” of an enterprise. Not just the glorified reporting it seems to have become. Like the brain, it integrates information from all over into a single place where it can be centrally analyzed. I’ve also compared domain models to the human brain. Therefore, DDD has a naturally better fit in the learning-oriented realm of BI than simply the automation of business processes.
The DRM serves the effectiveness of BI by identifying the major drivers of a certain problem. In this Big Data era we store everything. For any given problem, most of that will be noise that we can tune out once we know what’s important.
As I mentioned at the beginning, this really is a Zen post that may seem more like an IT whitepaper. The primary message is that the notion of creating and maintaining a comprehensive model of an enterprise is the business equivalent of the model of the Universe in your brain. The disadvantages are the same – it’s a major pain in the ass to maintain. But the rewards are great – a coherent, evolving model of the world that minimizes the disconnect, thus minimizing the dukkha.
The model in our brain or the DRM will never be perfect. The world is in constant change. The point of Zen as well as the point of upkeep of a DRM is that the price of our Gift of Sentience is perpetual polishing towards perfect reflection.
The dorje cookie pictured above features an enso, the black circle in the middle. The enso represents our journey. They are usually depicted with a gap which means something along the lines of “not-yet-perfect” – that the journey still goes on.
I look at it from a subtly different point of view. The gap in the enso pictured above reminds me of the constant change in the World and that the model of the Universe in our heads will always be a little out of sync. That gap is the small price we pay for Our Gift of Sentience and all we accomplish that are miracles to non-sentient things.
Lastly, notice that the enso starts out as a rather straight line at the fat end, goes straight up along the left, and takes an abrupt turn at the upper-left. There’s something poetic about the smoothness of curve after that abrupt right turn. Is that kink the day you or your company realized the Oneness Nature of knowledge.
A Profoundly Sincere Happy New Year to You!
Faith and Patience to you,
Reverend Dukkha Hanamoku
When learning about Functional Programming, don’t get caught up in the mathy stuff – monads, functors, category theory – or the zealous FP dogma. For the purpose of encoding a domain model, just F#’s type and function definition system seems to be the easiest on the eyes for the purpose of encoding a domain model for all to read. For developers, the familiar C# is actually well-equipped for writing FP-style code. The FP capability of C# may constitute a “pure” functional language, but hardly any real-world software projects are.
If you wish to go hard-core with functional programming, you’ll figure out you need to learn category theory. But study Abstract Algebra before hitting Category Theory, especially if you have no idea what “abstract math” is.
The technical advice in this post is most useful for the architect role of a project since that role sits between everyone, and the architect is a “technical” person. Of course, programmers are all at least part architect too.
Domain Driven Design and Functional Programming References
I’m a big fan of Scott Wlaschin’s work combining DDD and Microsoft’s FP language, F#.
About a week from now is the 40th anniversary of the start of my career as a software developer. I received the call informing me of landing my first programming job on Sunday, October 14, 1979. I was fresh out of high school with nothing but a ton of spirit, a seeming aptitude for this weird thing called “computer programming” … and my willingness to work for minimum wage.
I discovered this aptitude for computer programming during my senior year of high school. I signed up for a course called “Computer Math”. Ironically, I needed two math credits during my senior year to graduate since I had a rough time until then. It’s ironic because if it wasn’t for my poor high school performance until my senior year, I wouldn’t have been forced to take this class that defined my career.
Most of the world was barely cognizant of the word, “computer”. What was this “Computer Math”? In the hindsight of 40 years passing, “Computer Math” was a naive title for the course. It was just an introduction to BASIC for we 1970s kids who barely started using calculators.
I pretty much took over the class from the poor teacher, out-of-his-element. I re-worked the tests, graded the tests, provided material for lectures, for example working with my trig teacher (the other math class I took) on how pi is calculated.
I even set up and ran a booth at a high school exhibition at Pearl Ridge. I knew this computer thing was special with all the people who seemed genuinely mesmerized by how we “talk” to these strange machines. Much of that day was monopolized by none other than the not yet internationally known Israel Kamakawiwo’ole. I was mostly demoing to him while others came by initially to meet him.
However, I may have had an aptitude for programming at the time of my first job interview, but what I quickly discovered is that what I learned during that senior year of high school was not even close to “industrial-strength” programming. It was analogous to the English comprehension required for “Go Dog Go” versus “War and Peace”. I had – and for many years still had – a long way to go.
I had no idea about what I was going to do on my graduation day. It’s hard to comprehend how the notion of college wasn’t on my radar upon graduation from high school. But it wasn’t. This was 1979 in a little plantation town on Oahu, along with a few disruptions not of my doing.
I quickly found a job at a hardware store making minimum wage. Any money was more than I was used to. It took just a couple of months to realize that this was a road to nowhere. I had to do something. I looked in the newspaper ads for something. And there it was, an ad for a “computer programmer”.
I excitedly called, too stupid to realize how in over my head I should be. I told my soon to be boss about what I did for that high school course called Computer Math. He invited me for an interview, which I believe was Friday, October 12, 1979. I only know that date because it was the day Fleetwood Mac’s Tusk album was released and the local rock station played it in its entirety. That was my tunes as I drove in.
I hardly remember anything else about the interview. I think I left feeling I had a shot. I sensed the talent pool in Hawaii was so small that a hungry high school kid stood a chance, especially at $2.05 per hour. As I mentioned, I got a call a couple days later on October 14.
I drove into the office the day after that fateful call, loaded my mom’s beat up Corolla with a four-foot high stack of manuals and disappeared with a promise to return in two weeks ready to hit the ground running.
Back then there wasn’t a gentle on-boarding process, no senior programmers expected to mentor you, no stackoverflow, no github, no vast riches of expert bloggers, YouTubers, no online course sites providing spoon-feeding, high-quality, dirt-cheap content on just about any topic you can imagine. What I got to set me on my way back in 1979 was that four-foot high stack of 3-ring binders written in the most terse, obtuse manner possible, all black and white. A foot each for the hardware specs, the operating system, the programming language, and the user’s guide for the basic accounting software from which we built specialized versions.
Honest to God, I don’t know how I did it … but I did it. Sheer will, the wondrous brain plasticity of a 17 year old, and too naive to know failure was the betting man’s pick. That’s Version 1.0 of my career. I snatched the opportunity and never looked back!
Versions 2.0, 3.0, 4.0 of my Career
I never did look back, but I fell down to the ground a few times over the years when the rug was pulled out from under me. There were many career-changing events along the way, tracking along with all the disruptions in the industry – “mini computers” and mainframes to networked PCs, to client/server, to the Web, to the dot-bomb, to the rise of business intelligence – to big data, the Cloud, and machine learning where I am now.
Each of those disruptions were intense chapters of reinvention, getting up, dusting myself off, and sprinting my way back towards the front-runners. Many of my colleagues were knocked to the ground along with me, but didn’t quite get up. Fortunately for them, the increasing complexity of software projects spawned roles beyond programmers and analysts, non-technical roles – project managers, testers, business analysts, product owners, etc. So many were able to rebound from the disruptions in different ways.
None of those roles suits the personality of a rogue. Most real programmers are rogues at heart. We’re renegade artists thriving in the act of bringing order to ambiguity and relishing in our conquest over it.
Wait a minute. Isn’t being rogue un-Buddhist, the anti-meaning of “go with the flow”? The dictionary definition of rogue is not flattering – the most amusing being Merriam-Webster’s #4 definition: a horse inclined to shirk or misbehave. To me, a well-behaved horse, while useful to the general audience, it isn’t very interesting.
The Universe we experience is all about the neat stuff that happens where opposing forces meet – where the rubber meets the road, where the shit hits the fan, the Yin and Yang. What is Buddhist is to know and accept that fact. Here, I use the word rogue to describe relentlessly seeking the truth of reality, no matter what delusion our brain and the brains of others paints for us.
Well, as bad and as those stumbles were to my career, none of those reinventions were nearly as intense as that first one. Nor is this current one.
Reinvention – Version 5.0 of my Career
With just a few more Bodhi Days to my “big six-oh”, my Earthly body is showing signs of wear and tear. But I still love what I do to for a “living” to sustain this Earthly body as well as the Earthly bodies of my loved ones. So in order to retrofit myself for another twenty years or so, I realized I need to take proactive steps sooner than later towards reinvention.
The downside of being a gainfully employed old-timer in an insanely difficult and evolving field as software is that you don’t ever have enough energy left at the end of the day to fully keep up. These days, for IT workers life is a non-ending series of two-week sprints in a field where you don’t know what you don’t know and the environment is made of lots moving parts and everything is a moving target. At the end of the day, you’re wrung dry.
Unlike professions where things don’t change much, those who are just starting out in software are on par with us old-timers. The constant flow of brand new things means that experience doesn’t matter quite as much. It’s odd to ponder that many colleagues are about the age of children I could have had (in their mid-thirties or so) who grew up only knowing a world with the Internet, and already a with a decade of experience after their PhD!
This Version 5.0 is a proactive reinvention, unlike most of the others that caught me with my pants down. I could still easily make a good living for a few years as a “Business Intelligence / Data Warehouse” architect without this reinvention, but I’m better off doing this a few years before sixty years old than closer to seventy.
I hope that the insights below offer inspiration and guidance to my fellow “old-timer” colleagues as well as to those just starting out.
Version 5.0 Career Strategy
A few weeks ago I published a blog titled, The Dukkha of Original Thinking, where I mentioned a system I have in place to keep me in within my lane on the path while my mind is preoccupied with the tasks at hand, the Now. The graphic just above illustrates that system.
First off, my real interest is Zen. If I could, I’d spend all of my time away in the desert, away from the hustle of modern life, pondering, observing. But there are great advantages to bustle of modern life, such as lots of food and modern medicine. And besides, every square inch of real estate is owned by somebody. So I require a way to earn a living and contribute value in order to partake in those wonderful things.
Fortunately, I happen to love what I do for a living. I’ve written that programming is My Zen Art. So my profession isn’t just a means to provide the Earthly bodies of my loved ones with the necessities of life. It also serves as an Earthly model helping me towards progression with my Zen studies.
Because my intent is to set me up for a long period of time, it isn’t good enough to simply learn about the new technologies. In fact, I’ve already been dabbling in these new technologies (beyond the current Cloud and Machine Learning stuff) for a few years.
Providing high value means I need to shift to the next gear, genuinely mastering the skills – beyond varying ranges of novice to advanced, and on to “Expert”. But because my brain is finite, there isn’t enough capacity to run my old mastered skills in parallel with these new ones. I’d need to clear some room (empty my cup) for these new skills by hitting the delete key on some of the older skills.
It wasn’t hard to see that raising my math skill to very high levels is key. To non-programmers, it may sound odd learn that a large percentage of programmers are not good at math. Some never learned it, some learned it only long enough to pass the tests in college. I’ve learned and forgotten higher maths several times over my career.
Software development is actually a highly mathematical field. Math and programming are both realms for computing algorithms but in very different ways. Unless a programmer works at NASA or SpaceX, the programmer doesn’t speak and think in math but in “code”. So as I learn to “read and write” calculus, unless I continue to study it every day, daily and intense coding quickly vaporizes it from my brain.
The vast majority of software developers work in IT shops where they mostly maintain and improve current systems or they work to further automate enterprise processes. Now that I work in an IT department, in the normal course of work, I rarely need to tap into even plain old algebra.
Unlike math, emphasis is placed somewhere other than building a solid foundation of truths upon which we can compose derivative truths. It’s more a matter of knowing how to make the necessary changes to code as quickly and cheaply as possible and without breaking anything else in production. Software are tools. Like all other tools, they need to provide a net value to the enterprise. Meaning, the best, most efficient, most elegant solution is rarely the best towards the goal of adding value.
Sometimes, there is hidden math in programming that just looks different. For example, instead of ∑ the rough equivalent in programming is “for loop”. Many of those intimidating Greek letters in math equations are simply functions or constants. Programmers using imperative languages such as C++ or Java (the vast majority of programmers) certainly understand constants and functions, but laid out as a clear logical series of steps with relatively clear variable names, not as a single cryptic equation. If programmers started out as functional programmers, it would probably be easier to understand.
Where heavy duty math is required, particularly machine learning, there are packaged functions written by a minority of developers who are math whizzes. It’s not just heavy duty math that’s encapsulated away in tidy modules. Most programmers don’t even really need to know the basic computer science algorithms and structures, which are also encapsulated in tidy modules written by CS whizzes.
But our relationship with computing machines long ago reached a point where we depend on them for more than calculating tips at a restaurant. As it is with the inability to read, the inability to fully comprehend those awful equations renders the latest and greatest ideas and discoveries inaccessible to you.
Further, the for most developers, the primary driving force isn’t the elegance of code, but return on investment. Software doesn’t need to be the best possible. In fact, sometimes software is purposefully written to be “simple” at the cost of performance so that it can be understood by more. A few seconds a day is easier to deal with than finding a highly skilled developer capable of comprehending the code then optimizing away those few seconds in less time than it will take her.
Anyone can run a linear regression model with a few lines in R or Python. For the vast majority of the time across all customers in the world, that’s good enough. But tougher cases require a deep understanding of math to approach and tune a problem.
Functional Programming and Abstract Algebra
Another looming factor in my field is the rise of Functional Programming. It is the paradigm required to reign in the exponentially growing complexity of the systems we programmers intend to “automate”. Functional programs look more like math than the currently “traditional” object-oriented paradigms.
Functional Programming, along with Category Theory (the math upon which it is founded) is a subject I’m glad I began delving into a few years ago. It’s been quite a struggle, a real shift in thinking. But it’s a profound shift in the right direction for software development. It brings up closer to the ideas of math, science, and yes disciplined engineering.
It’s tough making such a mental shift while I need to continue to keep earning a living using the older paradigms (primarily object-oriented programming). But I found a nice trick. While I try to keep up at least a “working level” of skill with functional languages (most F#), I find I can kill the math and FP birds with the one stone of Abstract Algebra.
In fact, a big hint: If the term “Category Theory” sounds completely foreign, take a step back even further to Abstract Algebra. Trying to learn Category Theory without first understanding Abstract Algebra is like trying to learn anything without understanding the purpose.
For even those well versed in math, Abstract Algebra is hard! The way I had to learn it is through the Richard Feynman method of going back to the beginning once you start to get lost. But it’s where Computation Theory meets back up with math and they are united. That’s powerful. It’s not something to give up on.
I’ve always had an interest in quantum mechanics and maintained at least a cursory awareness of what’s going on by devouring the pop articles of Scientific American, books by folks such as Michio Kaku, and now the wonderful YouTube channels such as Veritasium, Physics Girl, and PBS Space Time.
But I’ve always realized that unless you’re a working physicist, no matter how much you think you understand quantum mechanics, you probably don’t. It’s such a sophisticated skill that like law and medicine, it’s imperative to learn it through a rigorous formal process and genuine daily application. More so, quantum mechanics works at the infinitesimally small and indirectly at the mind-bogglingly large scale. Not that little sliver in between where our brains are optimized for dealing with things at the scale of normal life on Earth, the things we can see with our naked eyes.
It’s also a field where much of it is very difficult to directly test. Tests can be many layers of inference deep, the result of statistical probability, or just the purely virtual result of math. That is very unlike with software where practically every idea can in the end be objectively validated.
No, I’m nowhere near ready to apply for a job at CERN – not even ready to teach physics at my local high school. So is it just a guilty pleasure for me, just a hobby? Not quite.
There is the saying: We can’t see the forest for the trees. For Zen, I can say something similar: We can’t see the reality of the Universe for Life on Earth. The study of quantum mechanics (even at the pitifully noob level) helps tremendously cut through that lens of the brain.
For this latest reinvention, I initially thought of delving deeply into quantum mechanics since I’m concerned about being left behind when/if quantum computing goes mainstream – whatever that ends up actually being. But quantum computing isn’t yet a clear and present danger to my career.
What I did find, though, is that further understanding of quantum mechanics drives improvement in my understanding of both math and Zen. Quantum mechanics moves high maths from pure abstraction to something real (at least kind of real). Math is the language of physics.
The North 40
I’ve taken a couple of days off to celebrate and kick off another 40 years. Mrs. Hanamoku and I plan to watch the Breaking Bad movie, El Camino (released on Netflix 10/11/2019), along with our favorite pizza.
For anyone interested in meeting me, I’ll be at Eureka in Boise this coming Friday at 3pm. I’m having a celebratory shot of William Larue Weller – I think $75!
In software development, it seems like Version 5.0 is the peak of a product. It’s finally mature, finally getting it right. Further versions just add complexity. By Version 8 or 9, it’s a bloated monstrosity. Perhaps after this, your Reverend Dukkha Hanamoku will take up a completely new career.
I originally wanted to title this blog, “We Don’t Pay You to Think”, a phrase I often hear on movies and TV shows. It usually goes something like this:
A private approaching his sergeant: “Sergeant, I think …”
Sergeant immediately interrupts: “WE DON”T PAY YOU THINK, PRIVATE!”
The movie Sergeant makes a very valid point. They are paid to faithfully carry out their orders based on a carefully orchestrated plan. The beautiful execution of a plan, perfect or flawed, beats the hell out of thousands of independently-minded “armies of one” rushing onto the field like in an old medieval movie.
But the notion of beautifully executed plans only works well under simple, closed conditions. In fact, any fairly reliable machine is a physical manifestation of an executing plan; a watch, car, refrigerator.
We should obviously plan, incorporating what we’ve learned from the past. However, plans quickly unravel for endeavors beyond simple and closed systems. These are complicated or complex systems. For example, a large business enterprise and certainly the battlefield where our Sergeant depends on his orders followed as given.
I once again invoke the Zen Master, Mike Tyson’s, great quote: “Everybody has a plan until they get punched in the mouth.” In the end, plans only point us to the road to our destination. We will have yet to hurdle lots of things we never saw coming. Sometimes there isn’t any road at all, so the plan can only point us in the direction to our destination.
By “original thinking” I mean intellectual exploration of the unknown, as opposed to optimization and/or exploitation of the fairly well-known. Like Captain Cooke and Captain Kirk, I long to venture into the unknown. But instead of far off lands, for me it’s hurdling the challenges of understanding our consciousness through software development and Buddhism.
However, I need to be clear that “original” is a strange word that doesn’t really mean what we subconsciously think it means. Hardly any of our inventions or art are as original as we’d like to think. There’s hardly, if any, examples of purely original thought – or purely unoriginal thought.
Originality is a measure on a spectrum ranging from ideas that are unimaginable to our humans brains to a basic thought like “I’m hungry”, practically hard-wired in your brain. Examples of genuinely original thinking include … I can’t think of any … it’s beyond the computational confines of my 80 billion neurons.
Our brain is a device that spawned from the relatively closed system of “macro Life” on Earth (the interaction of things we can see with our naked human eyes). It’s incredibly hard to think of something outside the realm of Earthly life. So for this blog, let’s think of original thoughts as a clever association between one or more known concepts. For example, phone infrastructure plus cheap computers equals Internet.
“Original thinking”, doesn’t even mean being the first to have thought of something. In a world having hosted about 100 billion total people over the past 200K or so years, seven billion alive today, it’s extremely rare to have thought of anything, no matter how bizarre, that had not crossed the minds of at least a few hundred, if not millions of others.
Even if no one thought of your particular thought, it’s still based on our collective knowledge-base of thousands of years of history. As it is for practically every other computer programmer out there, lots of the code I’ve written over the past decade was born out of snippets code cut, pasted, and molded from StackOverflow.
In fact, The Eternal Fishnu told me that even the critical technologies for ancient humans, fire and agriculture, were re-invented many times in many different places. And they weren’t invented in a vacuum, but just part of a long series of little steps leading to a context where the association between flint and tinder wasn’t a far reach at all.
Conversely, what about at the other end of the spectrum, completely unoriginal thought? Life on Earth is so complex that what we humans face instant to instant is hardly (or never?) ever exactly the same. Remember Heraclitus’ teaching of never being able to step in the same river? Almost every action we take has some measure of originality, albeit a wee bit.
By “original thinker”, I’m not judging that as better than other sorts of thinkers. Being an original thinker is more of a preference. I prefer and enjoy original thinking. That doesn’t imply I’m even good at it. Because most of the time, I’m not good at it. Even for original thinkers, almost all thought through the daily grind is ordinary and unoriginal – walking to work, eating, attending scrum meetings.
However, this preference isn’t something I’m able to turn off. Believe me, I often wish I could. For whatever reasons – my genetics, my upbringing, my unique and random experiences – my brain computes an affinity for exploring the unknown, as opposed to exploiting the known. Unlike my colleague who once told me, “I used to look for shiny things too”, I can’t let go of that curiosity for what this Universe, or at least consciousness, sentience, is all about. Somehow he did … or at least he makes himself think he did.
Our society requires thinking preferences along the entire spectrum from original “shiny things” thoughts over to pragmatic thought. If we stand on a mountain peak, we can get to know every nook and cranny of that peak and hop around it like a Sherpa. But there are other peaks, higher with wider views. The original thinker is the one that figures out the tough journey of getting there without a map – she is the author of that map.
Well over a decade ago HP put out a classic commercial for EDS that gave us software folks the herding cats meme. For those who haven’t seen it, I don’t think I need to explain that commercial any further than asking you to imagine herding cats as compared to herding sheep or cattle.
What I think of as Computer Programmers are an odd sort. That is, the circa 1980s sort of Latter Day Hippy programmer. We’re not artists, nor are we scientists, nor are we engineers, certainly not academics. We are not corporate resources like a fleet of bulldozers or grid of cubicles. We are rugged individuals who love playing God in our little silicon universes, favoring the creation side of the job of being God.
Before the invention of computers gave us programmer types a venue to earn a living playing out our God fantasies, we were a combination of artist, scientist, and philosopher; inventors. Not just the ones you’ve learned about in history class, but the many who are lost to history who, awakened to the fascination of the unknown, still had to make a living as worker bees.
Such folks struggle in a team environment. Cross-human communication is much slower than intra-brain communication. Collaboration waters down that mucho cool idea we’re powering through. Of course, I say that tongue-in-cheek, but yeah, incubating original thought in just ourselves is our default mode.
However, many modern problems vastly exceed the capacity of one human brain. Interestingly, this is the same thing Big Data solved for computers. Databases used to grow at a rate where the larger ones could fit on the currently “largest single machine”. But the growth of the larger ones began to far exceed the growth of the currently largest single machine. So an architecture that recruits hundreds of 100 pound gorillas can scale much further than one 800 pound gorilla.
Can we overcome that default mode of rugged individualism towards the whole is greater than the sum of its parts magic that is teamwork? Can we attend the daily scrums with a pleasant smile, update our task hours daily to stay in the green, and gear our intellect fully into the sprint requirements? Yes, we can and have, even though we may absolutely hate it despite the mind games we’ve learned to play with ourselves. (“Yes, I love my job. Yes, I love my job.”)
It’s a very good thing that many people don’t like exploring the unknown. Instead they have an affinity towards fully exploring what is known. These are the A+ students. They optimize a domain, pushing the limits of that domain further out. In fact, Life on Earth never asked for original thinkers of our scope in the first place. All innovation has two fairly equal parts – the first 90% that takes half the time, and the last 10% that takes the other half of the time.
Armies of people are needed to manifest the thoughts of the original thinkers into this world. There are way enough people with way enough good ideas. Even for the most brilliant of original thinkers, much time will be spent in that army carrying out the ideas that somehow gained traction.
If you’re lucky, the thought being carried out fits nicely into your own goals. If it doesn’t, your pliable brain will mercifully “grow to love it” or at least endure it. For the rest, i.e. those who read my blogs, my guess is neither of those cases apply … hahaha! The good news is that the execution of someone’s great idea still has an entire journey of problems to be creatively solved before the manifestation of that vision is completed.
Teamwork and the Battle at the Species Level
It might help rugged original thinkers cope in a team environment to realize that the real shtick of humanity isn’t our individual brain power alone but that brain power combined with those attributes enabling us to work as a team. As sentient individual humans, it’s easy to forget that evolution isn’t just a competition among individuals. More compellingly is focusing on the competition between species.
We humans are not the strongest, fastest, most durable creatures. But our package of qualities enables us to gel as a team coalesced around a goal. Even if each team member has other things they’d rather be striving for, somehow the members put aside their personal desires.
Genuine teams opens up much more scope of possibilities than just a loose collection of guys with spears. If our hunter teams were just a bunch of guys with spears chasing an animal, all that animal species would need to do is get faster, bigger, meaner, fly away, and maybe develop thicker armor. Simple solutions for a simple predator.
Similarly, basketball wouldn’t be that much fun to watch if it was just a one-dimensional game of two sets of five guys catching a rebound, dribbling a bit, and shooting. Working as a team, they can get the ball closer to the net for a closer, higher-percentage shot. That’s a much more interesting game with much more scope for evolving.
Again, that goal or the approach taken towards that goal probably isn’t shared by everyone on the team. Maybe some guys on the basketball team favors a different style or one or more of those guys wants to be the scoring leader. If they suppress their individual desires, suspending their own needs for the “greater good”, what emerges is truly something where the whole is greater than the sum of its parts.
The notion of “greater good” can be tough for some of us rugged, individualist, original thinking nerds. But we do somehow put aside our own desires. As disjointed as society may seem at times, society wouldn’t work if we didn’t indeed coalesce as a species to a larger degree.
Lastly regarding teams, it’s important to point out that what sometimes may look like a team is not really a team, in the context of this blog anyway. Imagine a team working at a high-performing McDonalds. There are people adequately trained in their different roles. They must all effectively communicate. None of them can drop the ball.
Are they a team? Although they are certainly hard workers due the respect of any other workers, they are human cogs in a well-defined machine. When we refer to a team as a well-oiled machine, that’s not too far from the literal truth. As soon as a machine is able to fill in for a role, out goes the workers in that role in favor of that machine. And yes, that process started years ago.
A machine is an organization which for the most part not much unexpected happens. When the function of the organization of parts is well-defined, things are predictable and so the organization can be machine-like. In this case, for the most part, customers just want a quick and familiar meal. No surprises to either the McDonalds team or the customer. If anything unexpected happens, the only one “paid to think” is the manager. But even that manager probably has a special decision tree app on her phone of what to do for 99.9% of weird things that could happen.
Row, row, row your boat …
I’m still out there in the world banging away on a keyboard, working in an IT department, tucked away in a nondescript grid of cubicles. My current position certainly is not my dream job, not anything I would choose if I were an omnipotent being. But I’m not an omnipotent being, just a “piece of the Universe made alive”.
This is great. Original thinkers at least for iterative periods of time need to be directly connected to the real world. Go into the world and work for a while filling your mind with things new to you. Then spend some time composing an orchestration of novel relationships. If it’s of value, do the entrepreneurial thing. If not, go back out into the real world to refresh your mind. If you don’t do this iterative thing, the original thinker sinks into the realm of being impractically too academic or just plain crackpot.
Sometimes the transition between iterations can be a forest fire. Forest fires, literal or metaphorical, indeed have an invaluable purpose. I had been a gun-slinging Business Intelligence developer for almost 20 years. That’s maybe too long and a career forest fire took its inevitable hold. Enjoy the burn on your brain. As with the literal forest fire, a new trajectory is born.
The “team” I currently work on is indeed intended to be a team, not a herd of isolated developers. However, the comfort zone of those in charge are not familiar with software development tend to push towards the “sensible” non-team variety of human cogs in a well-defined machine. Software development is not a well-defined machine … not yet anyway. Software developers do indeed actually strive for the day we put ourselves out of business. However, every day is still Surprise Day and computers are still the dumbest computation devices in the room, still too tough and chaotic for what we’d call a well-oiled machine.
Although my desires often don’t jibe with the project plans on various dimensions – the schedule doesn’t exactly fit with mine, the subject matter isn’t very interesting to me – it’s best for the corporation, the team, and me if I offer the least friction possible. Time, energy, and other resources prevent adequately stating each “cat’s” case. So we settle on the “good enough” approach, take the blue pill, and carry on with no fear.
Life is but a dream …
I carry in my pocket an old gaming token from the Monte Carlo Resort in Las Vegas. I’m not into the Vegas thing, but many Hanamoku family reunions take place in Las Vegas – thee favorite destination for people from Hawaii. “Monte Carlo” is also the name of the most well-known simulation algorithm, a tie to my analytics past, and to my eventual return. But the symbolism runs much deeper.
What is interesting is that underlying the other-worldly atmosphere of a Vegas casino are a set of tight processes onto which the owners confidently place their faith. The casino architects set up processes with the scrupulous authority of mathematics on its side, as well as processes geared to keep gamblers genuinely entertained. Everyone is trained for beautiful execution of these processes.
With these processes in place, the workers at the casino can focus predominantly on execution, confident in their systems. This is like locking your doors and windows before going to bed so you can sleep peacefully without the worry of someone walking in and sneaking up on you in the middle of the night.
As a casino goes about its business, every now and then someone will walk out the door with a big jackpot. But it doesn’t bother them because the math of black jack, roulette and slot machines dictates that overall, in the long run, they will be comfortably ahead. In fact, a periodic big winner provides hope to the gambler even in the face of inevitable loss, bringing in even more customers. Growth is a good thing for a linearly-scalable system.
The Gift of Sentience
A central notion of Zen is to realize this heuristic: Pure acceptance of what comes your way is much less stressful than fighting it in favor of what your brain wants. However, strict adherence to that heuristic renders our gift of sentience meaningless. Our gift of sentience, our ability to purposefully design, is a magical power over evolution. The purpose of Zen is not to retreat back to the innocence and bliss of non-sentience but to smooth out the transition, which is a rough ride.
For me, practice of that pure acceptance means to set aside my own thoughts and desires in favor of the goal of my team. But as the days go by where I become more comfortable with my situation, I may be lulled into a cozy blanket of complacency that could be taken away at any time – i.e. I’m laid off like a Detroit auto worker from the 1980s with lots of skill that nobody needs anymore.
To sort of have my cake and eat it too – a dukkha-free day but avoiding the eventuality of getting caught with my pants down – I have a system in place to ensure my long-term marketability. I’ve identified a couple of key areas of knowledge and the highest level of abstraction that is practical. These couple of key areas take up only a few hours per week with minimal volume of knowledge.
For example, my current day job doesn’t include anything remotely requiring functional programming (FP), a skill I believe is a critical complement to object-oriented. I simply don’t have time and energy at the end of the day to keep up with the plethora of FP languages without spreading myself too thin and/or burning myself out. Instead, I study abstract algebra, the foundation of FP, a single topic that I actually enjoy. Instead of being a novice with a few FP languages, I became very good with a single topic from which I can easily ramp up to any FP language.
Similar to how it is for the workers at the casino, my system ensures that I can blissfully go about the “daily grind” with faith that my system is busily at work protecting my long-term marketability. I’m able to stay in the Now, without distractions of that future which isn’t yet here. The the end of a work day, having put aside my own “desires”, and fully accepting what is right in front of me, I lived a dukkha-free day.
But when I get home and empty my pockets, I notice the Monte Carlo token. It’s a reminder that what my brain computes hardly ever fully reflects reality. It’s merely a pitifully inadequate model of reality. Similarly, the Business Intelligence reports enabled by the systems I build are useful but still inadequate pictures of what’s going on in the physically real business ecosystem.
What is this process I have in place? What is this process intended to re-design what surrounds me towards what I envision, while at the same time living with 100% acceptance of what is right in front of me? I am working on a blog on that topic I will share some day soon. But it is one tailored for me that happens to fit the unique set of circumstances that is me. Following someone else’s process instead of taking full ownership of your own somewhat defeats the purpose of our gift of sentience.
Thanks to Donald Rumsfeld for making the very Zen, Philosophical concept of “unknown unknowns” rather mainstream. This notion reaches deeply in the field of analytics where my career focuses. I can’t ever say “there are unknown unknowns” in a presentation without someone attributing it to Donald Rumsfeld’s wisdom.
It’s easy to be cognizant of questions known to us for which we simply don’t know the answer. For example, when renting an apartment, I know to ask if there is a time after which loud music is prohibited. It’s something I learned to ask after once renting an apartment in a complex without such a rule. It’s a known unknown, which I resolve by asking the question.
However, I learned after renting another apartment that I didn’t know to ask if the parking space directly under my apartment was mine or someone else’s. I guess I assumed the space under my apartment was mine and so if I came home late, opening and closing the garage door would only disturb me.
Everything we ask today was once something we didn’t know to ask – an unknown unknown. In fact, I can’t recall any instance – renting an apartment, taking a job, buying a car – in which I wished I knew to ask a question about something that later surprised me. After experiencing some gotcha we suffer from that ignorance, we know to ask it the next time. It’s now a known unknown – a question we know to ask in the future.
Throughout our lives we collect these questions, these tips and tricks of life. If we’re smart, we can smugly go about life stating, “Fool me once, shame on you. Fool me twice, shame on me.” We grow into wise old folks who don’t fall for scams and rookie mistakes.
The problem is the rules are always changing. If we’re lucky an outdated rule is merely useless. Applying an outdated rule is often costly. That can be frustrating, tiresome, and in the jungle, it’s fatal. But knowing the rules are always changing is 99% of the solution!
Awareness of the Unknown Unknowns
I’ve often mentioned The Beginner’s Mind as the single most important aspect of Zen practice … if one must have a “one thing”, as Curly suggested in “City Slickers”. It’s #1 of the Three Zen Stories upon which the Eternal Fishnu’s teachings are based.
Couple the Beginner’s Mind with the knowledge known to even novice Zen practitioners that change is the only constant, you never walk in the same river twice, blah blah blah. With these two near-axioms in mind, in order to evolve with change, we must constantly learn the new rules. And to learn these new rules, we must make room for them by clearing out space by dumping out the old stuff.
The inability to cultivate a Beginner’s Mind leads to a brittleness that easily shatters with a well-placed strike. That well-placed strike can be in the form of a cat catching a mouse or someone losing their life savings to a scammer.
The Beginner’s Mind is the awareness of unknown unknowns. But an awareness of unknown unknowns doesn’t sound like the primary Zen advice of having a laser-like focus in the Now. It sounds suspiciously like the senselessness of worrying about the countless things that could happen, the bulk of which will not happen.
That’s one way to look at it. However, one could also look at a lack of awareness of unknown unknowns as un-Zen in the form of clinging to a past that no longer exists – i.e. applying rules you’ve already learned that lost its validity due to constant change. For the most part, being in the Now means to keep your mind focused on the present where things are actionable. It doesn’t mean to ignore the value of your unique experiences towards actions moving you in the direction of your path.
Our LCD McGoogle World
Awareness of unknown unknowns must be built into all planning. If you’re truly honest with yourself, do you recall any plan that was executed as planned? Even if successful, the plan succeeded due to heroics addressing the unknown unknowns. Rather, unknown unknowns are swept under the rug as we sign off on a plan, coerced into a belief it is bullet-proof and/or we knew that all along.
The way to build in an awareness of unknown unknowns into all planning is not just beyond the scope of this blog, but requires volumes of books. However, in a nutshell, it requires embracing the complexity of the world – in Zen terms, living the notion that change is constant.
In very secular and “tip and trick” terms, this means mastering the fundamentals. Fundamentals are relatively few core concepts underlying a wide range of application. Underlying the complexity of the Universe, or good software, or good practice of medicine and law, are layers of composition of simple rules.
The problem with mastery of fundamentals is that mastery of relatively few rules on the surface looks a ton harder to master than learning a lot of simple tips and tricks. The key is this: How many billions of tips and tricks do you need to learn to fully cover a knowledge domain – versus how hard it is to master a relatively few “tough” concepts from which you can answer almost anything?
As with any notion of foundations, it must be done correctly, thus requiring a long time to master. In these McGoogle days of immediate gratification (quick ROI), notions of true mastery of are considered indulgences.
Humility, Not Stupidity
Don’t mistake the Beginner’s Mind for stupidity. The Beginner’s Mind is genuine humility – the full acceptance that the only reality is the Universe, as opposed to the pitifully inadequate model of the world housed in our brains, as deficient as an aquarium is a model of the ocean.
However, it’s not a matter of dropping your experiences from your brain. Your experiences are data encoded in your brain. It’s a matter of dropping your beliefs from your mind, the rules you’ve computed from that data long ago. Only then can you re-compute rules based on that same data and new information now at hand. Your experiences are still of value today just as are the somewhat and arguably outdated works of Aristotle, Freud, or even the Buddha himself.
People unaware of the unknown unknowns are often insulted by others with the facetious phrase, “He knows everything.” That person to which the insult was thrown would unwittingly take it as a compliment. But that insult is hurled out of frustration to people unaware of unknown unknowns who are like huge boulders, a dam in the river – steadfast, unmovable, an obstacle to the flow of all else.
But don’t be hard on those people because we’re all guilty of it most of the time.
Years ago Mrs. Hanamoku took a few series of watercolor classes from the late Roland Roycraft. He was in his 80s at the time, still very much active. He was an excellent teacher and very open with teaching his wonderfully distinct style. Mrs. Hanamoku asked him about any concern he may have in divulging his hard-won techniques. He said, “I stay a mile ahead.”
That lesson made a big impact on both Mrs. Hanamoku and I in how we approach our careers. I spend a consistently significant percentage of my time “staying a mile ahead”. This is not for “competitive” reasons, but to stay in the light, out of the shadowy ruins of what is no longer there.
This is especially important in a field that changes so rapidly. However, “change” isn’t quite the right way to put it. New technologies (hardware and software) do come to market at a rapid pace – the buzzword flavors of the month. But for most enterprises out there, change is actually not really that fast.
New technologies quickly become the core of innovative startups incubating the industries of the future. But it can take years, sometimes decades, for established enterprises to adopt the new technologies to a noticeable extent. For example, neural networks and functional programming are older than my almost sixty years! We think change is faster than it really is because the media (journals, bloggers, online classes, professional conventions) place more emphasis the glamorous latest and greatest stuff.
Change is constantly around us, but it is variably fast and slow from one place to another.
I thought about this over this weekend, having completed my fourth week at a new job. It’s a job that has so far been limited to pretty much SQL – and that looks to be the case for the foreseeable future. There is indeed no room there for the fluency of the cutting-edge skills I’ve diligently and mindfully built over the past few years; particularly machine learning, functional programming, abstract algebra, Databricks (Spark/Pyspark).
All the other programmers at my new job are as good at SQL as I am! And they should be – they’ve been doing it long enough. Every tool has a limited scope, whether it’s SQL, watercolor, or a bulldozer. There’s only so much you can do with it, only so much expertise to build. That means, no matter how good you become, because such tools have limited scope, everyone is capable of eventually catching up to you.
If you took to SQL back in 1995 like a duck to water, back when its “declarative, set-based paradigm” was a bit mind-boggling to programmers used to procedural languages like COBOL, you had a golden skill. But by 2019 everyone will be as good as you are. The SQL language itself really hasn’t changed all that much in the past 25 years. The vast majority of SQL authored by business analysts, ETL developers, data scientists, and application programmers is the same stuff.
I mis-stepped my way back to 1995, caught in the past when SQL was a shiny thing. It’s reminiscent of life for me as a teenager, trapped in the past of a once booming pineapple industry of Hawaii, destined to work on the same plantations that attracted some of my first ancestors to Hawaii 80 years earlier. For them, it was a boon at that time, but for me, someone much too late to the party.
Almost 40 years ago, I found my way out of that ghost town of the once thriving pineapple industry of Hawaii. Three months out of high school the Universe presented an opportunity. I diligently fully assimilated, within a few weeks, a 4-foot stack of 3-ring binder manuals on the AlphaMicro system. My diligence and mindfulness transported me to a different place, with a different brand of boom time – a place where I can grow instead of spending precious life treading in the zero-sum games of an industrial ghost town.
This mis-step is perfectly OK. It isn’t the first nor will it be the last. Today, I have to again escape a metaphorical dying pineapple plantation town. Every day at lunch, I go downstairs from the 10th floor offices and sit under this nice tree, sort of meditating, unnoticed by the bustling crowds of a downtown. I meditate through a set of advanced books on functional programming, domain-driven design, and abstract algebra. Why? It’s the samurai ethic of diligently perfecting your skills so that when opportunity comes calling on you, you are ready for it.
To be certain, of course there are much worse things than falling behind in one’s career at an advanced age. There will always be someone with worse problems – and conversely, someone smarter, stronger, faster, more beautiful, more likable. The point of this post has nothing to do with winning. It has to do with being more aware and diligent than the frog in the proverbial pot moving slowly towards boiling – independent of what others are doing or where they are at.
Mind you, I’m not concerned about being the best – in this case being one of many interchangeable SQL programmer resources. Chasing such a thing is dukkha. Rather, I avoid the stasis of being a commodity. What do I mean by that?
We’ve seen those “cellular automata” computer programs where many things move around a screen, one thing eats another, others starve to death. Eventually, the movement on the screen slows down and stops in a static pattern. Every now and then, something has to shake it up to keep it going.
I sense that this mis-step is really a wonderful thing. As the theoretical 11th dan in Aikido is a white belt (a double-wide white belt), perhaps at almost sixty years of life and forty years of hard work, this deja vu sequel (SQL – get it – hahaha) is my 11th dan life test.
To help drive home the meaning of the Five Aggregates as I wrote about in Part 1, let’s try an exercise devising a high-level architecture for an Artificial Intelligence mapping to the Five Aggregates. In some ways it could be easier for us to relate to an Artificial Intelligence than our own intelligence since the parts (computers) were created by us. Using our brain to figure out how our brain works is way tougher than using our brain to understand how something that we created works.
The theme of this blog site is Zen/Buddhism from the point of view of a software developer; who finds much insight into ourselves by building software, which are models of things we do. It’s my Zen Art. That is, I research and build “A.I.” systems in the hope of gaining a better understanding into how our consciousness works – as opposed to the businessman’s purpose for A.I. being a dream of incredibly cheap, non-complaining, replicatable, re-programmable workers.
If you’re not a “computer person”, I still think someone with just a casual knowledge of software development could appreciate this high-level exercise of mapping of the Five Aggregates to a hypothetical A.I. architecture.
Before diving into this exercise I’d like to mention that because the frontier of A.I. is rapidly changing at the time of this writing, January 2019. Much of what I write here could be either obsolete, wrong, or very obvious a month from now. So I won’t burden you further by qualifying every other sentence with ” … at the time of this writing …”.
The term “A.I.” is often interchangeably and incorrectly used with “Machine Learning”. What makes it confusing is that whatever we call A.I. depends upon how you define “intelligence”. And there really isn’t a solid definition. For the sake of this blog, it may be easier to first consider an un-intelligent thing. An un-intelligent thing needs to be told every single detail about what to do. That includes practically all machines such as cars and computers.
It may sound strange to those who don’t know much about how computers work to call it un-intelligent? In workshops I’ve presented on “data mining” topics, there are a couple of things I often say that are obvious when you’re mindful about it, but overlooked as you go about in the normal life frame of mind:
What is the best thing about computers? They do exactly what you tell them to do. What is the worst thing about computers? They do exactly what you tell them to do.
What’s obvious to you may not be obvious to a computer. For example, consider the obvious observation that pregnant humans are female. It’s ridiculously obvious to us, but imagine if a computer figured that out all by itself.
It’s not necessarily that a computer itself is stupid. Rather, it’s the way we manually program them into these “intellectually brittle” dead ends. Instead, an intelligence must be self-adaptive in a very ambiguous world.
Currently, there are three major categories of machine learning: Supervised learning, unsupervised learning, and reinforcement learning. They are often presented as three kinds of tools, as if you were deciding between the purchase of a sedan, a pick-up truck, or a van. Rather, the three types of machine learning are three modes of learning, all required by an intelligence.
Before getting into a little analogy about the three modes of machine learning, I’d like to mention a fourth way a computer “learns” – programming. We may refer to propaganda and advertisements “programming” us, but that’s not correct. Rather, we are “trained” through those means. Programming a human would entail opening up our brain and manually wire up our neurons and manually adjust neurotransmitters.
Supervised Learning should really be called “Training”. Imagine you’re three years old sitting in your marine biologist mom’s office. You’re looking through a picture book of marine animals. You ask her about each picture, but since she’s busy, she only can glance over and answer with the general class of the animal; fish, mammal, crustacean, etc.
Eventually, you will figure out, find the patterns of what differentiates the mammals from the fishes. You’ll notice the whales and dolphins have tails pointing sideways and sharks have tails pointing up and down. The mammals also have fewer “fins”. If the picture has enough resolution you may notice fish have scales, mammals don’t. In the future, when you encounter such a creature you’ve never seen before, you’ll at least know whether it’s a mammal or fish.
Remember, this is training, not programming. To be programmed, your mom needs to get into your head and wire the neurons up and tune all sorts of other things in there.
Now, let’s say your mom is incredibly busy and just leaves the three year old you with the book. You’re on your own classifying the animals in the pictures. Without being given any labels for the animals, you may classify orcas and great whites together or blue whales and whale sharks together. Is that wrong? Orcas and great whites are predators of other big critters, blue wales and whale sharks are huge creatures that filter feed little critters.
The trade-off between supervised and unsupervised learning is that the former saves you lots of time learning what humanity already knows, whereas the latter allows you to be creative. Even if some knowledgeable adult were 100% available to teach you via supervised learning, you’re still being trained, not programmed.
Remember in the movie, City Slickers, where Norman the calf is walking just minutes after being born? No one taught the calf to walk. The calf awkwardly gets on his feet, stands very shaky, starts taking clumsy steps, going through cycles of trying, receiving feedback, adjusting neuron connections in his brain and muscles, trying again. After many fast cycles of this incremental training, the calf can run. That’s reinforcement learning.
The A.I’s Job
Even we mighty humans aren’t “intelligent” about everything. We each have our unique sets of skills, both seemingly natural and learned. So let’s give our A.I. a job. Let’s give this Artificial Intelligence the skill of a CEO of a corporation, an A.I. CEO. Let’s refer to this as an aCEO.
Taking advantage of what computers can do better than brains, the aCEO could be more effective than a human CEO if it could have a superior:
“Feelers” into all aspects across the corporation.
Feelers to life outside the walls of the corporation throughout its ecosystem.
A map of how all the parts relate, inside the enterprise as well as outside in the ecosystem.
Keep in mind too that it doesn’t need to look or act like a human CEO. It just needs to come up with superior plans and decisions and communicate and manage the executions effectively.
If we think of an enterprise as an organism, the CEO, real or artificial, is the brain organ of an enterprise. The entire enterprise is the “body”. The departments are like the other organs. Cash flow is like blood. What the company produces is their job, their livelihood. I guess that makes the human employees like our “gut bacteria”.
As with the human version of the Five Aggregates, the aCEO version is not a “series of five steps to consciousness”. It’s more the “five partsof sentience” where the whole is greater than the sum of the parts . So I present them in numerical order, but do jump back and forth to other parts. The illustration below provides a high-level view of how the five parts relate.
The 1st Aggregate – Integrated Data Model (Body/Form)
The body of the aCEO would of course imply the hardware, the computers. But body/form is much more than that. It’s information about things going on around us. In the Buddhism context, these are “conditioned” forms, the phenomenon that everything is resulting from everything else. The fact that things are moving and interacting means things are impermanent.
So the aCEO needs a way to obtain current data about things related to the business it runs. This current data could be current snapshots or just what has changed since the last time.
At this point, the data coming into the system from its surroundings is raw. To use an analogy based on the old “data mining” term, it’s like mud in a gold pan. But it’s mud from a place we think contains gold, not just randomly looking around. The 1st aggregate also filters out and cleanses most of the mud/crap before it gets to the Perception functions, the 2nd Aggregate. At the end of the 1st Aggregate, five pounds of mud is reduced to a couple ounces of “black sand”.
The 1st Aggregate further does some rough “data prep” on that cleansed data. This data prep organizes the cleansed data to a set of statistics about the data, the composition of values, and even looks for interesting “events” such as spikes in values, trending up or down. Still, this is just data.
This is pretty much where Data Warehouses are at. It encompasses all the processing and massaging of data from many different data sources. But it’s just data at this point – not much different from corn sitting in silos, grown and harvested with a combine. In general, this is where “Business Intelligence” system passes that integrated data to humans to make sense of with their superior Human Intelligence. Human analysts, data scientists, managers. “BI Developers” will author reports using tools such as SQL Server Reporting Services, analysts will visualize data with tools such as Tableau, PowerBI, or even good ol’ Excel. Data scientists could develop predictive models.
The 2nd Aggregate – Alerts (Sensation/Feeling)
Every machine or creature must operate within parameters conducive to its nature, whether it was designed by some intelligence or evolved over time. For example, without special suits (ex. SCUBA gear), humans thrive within strict parameters of atmospheric pressure, temperature, etc.
When it gets too cold, we put on more clothes. When we’re thirsty, we seek water. These are feelings, they are metrics. Your car dashboard is full of them. Going too fast, ease up on the gas pedal.
I make a living as a Business Intelligence consultant, which is all about maximizing performance of a business’ resources. A big part of that is deploying analytics tools conveying metrics to workers of all types – human employees in many roles as well as machines such as computer servers and manufacturing equipment. A tired cliche in my field is “what gets measured gets done”. We call these metrics “Performance Indicators”.
Metrics are determined by executives and “MBA types” and monitored throughout the execution of enterprises processes. Metrics aren’t just a business thing. Practically everything we do involves metrics, from our health (blood pressure, weight, temperature, cholesterol), to driving a car (speed, temperature, distance from other cars), to economic health (unemployment rate, GDP, stock market value).
An example of a business metric, in fact a key metric (called Key Performance Indicators – KPI) is Revenue Growth. What’s interesting about revenue growth is that no growth is usually bad, but too much growth can be bad too. Sometimes we need to limit growth to give the infrastructure time to catch up. Growth that’s too fast can overwhelm the infrastructure leading to mistakes, which lowers quality, which pisses off customers, which sends them away, which lowers revenue.
So a metric is more than a measurement, just one number. It’s more than simply stating Revenue growth is 10% over this time last year. Because too little or too much can be bad, a metric includes a component called a target, in this case, something like 5% growth, no less and no more.
Another component of a metric is a trend. Is the metric trending upwards, downwards, or steady? This is important to know because nothing stays still. Imagine your revenue is trending up and today it’s at goal, but it’s still trending up, which means tomorrow it won’t be where you want it.
The most interesting component of a metric is the status. Remember, too little or too much is bad. But we can live with a margin of error, say 2% on either side. So anything within 3% to 7% is good, from between 0% and less than 3% or greater than 7% and less than 10% is not good. But less than 0% or greater than 10% is bad. In general, the good, not good (warning), and bad are represented by green, yellow, or red icons.
The status of metrics are analogous to our feelings in the human 2nd Aggregate – pleasant, unpleasant, or neutral. The 2nd aggregate is interested in the status value of the metrics mostly so we know what needs attention – correction. In other words what metrics are in a bad (painful) status? In contrast, the target is the domain of the 5th Aggregate, the goals/objectives of the enterprise.
As with our human feelings, there are very many metrics (sometimes thousands) monitored in an enterprise. Some are very high level, such as Revenue Growth and Profit. But most are lower level, for example, the uptime of the reporting system, minimizing office break room expenses, and maximizing employee retention.
Before heading into the 3rd Aggregate, let’s review the process for a human. When pains are felt in the 2nd Aggregate, the 3rd Aggregate tries to recognize what’s going on, the web of goals/desires/beliefs of the 5th Aggregate engage the thoughts of the 4th Aggregate to make everything in the 5th Aggregate happy.
The 3rd Aggregate – Functions (Perceptions)
The 3rd Aggregate is mostly what we’re familiar with in regard to software systems of today. Software systems of today are machines automating some well-defined and tedious job at a really large scale.
If the job that software is automating was not well-defined, not highly predictable, we humans wouldn’t be able to encode the rules as software. But the real world outside of today’s somewhat totalitarian corporate culture is not highly predictable. That world outside is made up of countless independent things, at least seven billion of them with minds of their own. This is where Artificial Intelligence differs from conventional software. Intelligence deals with ambiguity and complexity.
Software today is made up of a designed (meaning “human-crafted”) web of designed functions. Functions take inputs, do some processing within the confines of a “black box”, and output an answer. Examples of computer functions range from something as simple as addition – takes in two numbers, adds them, and output the sum – to something as complicated as an image processor that takes in a picture, does a lot of processing, and outputs where it calculates what are faces, bananas, and cars.
It’s OK for now to let humans write logically tough functions. It beats the hell out of evolution which does things on its own time – like millions of years. But we still need the ability for a large percentage of the functions to self-organize, organize without human intervention. An A.I. consists of so many functions and require constant updating. It would be an insane nightmare for programmers to maintain that much, yet, that’s pretty much how software is currently maintained.
A general purpose programming language (for example, Java, C++, C#) is intended to encode a model of the world that is runnable by a computer. Because the world is so complex, programming languages must be very versatile. So programming by humans is tough. However, Nature found a way for the human brain to model the world. Our neurons wire according to what we’re exposed to at the time and place of our life.
So programming is hard. We try to mitigate that difficulty by creating specialized programming languages on top of specialized software. For example, SQL is a programming language limited to creating, reading, updating, and deleting rows from a table. Of course, there are libraries of packaged functions preventing “reinventing” of code that works across a wide array of applications.
Functions are machines, whether little or big, simple or complicated, natural or man-made, material or encoded. They all take in some inputs and output something. For example, a coffee machine that takes in water, ground coffee, and electricity, and outputs hot coffee. A single neuron is a function. It accumulates charge from many synapses on its dendrites, and at some very complicated point it “fires” a charge out on its axon.
Functions share a couple of very important characteristics. They shield we civilians from the intricacies (nice way of saying “yucky guts”) of how they really work. Imagine if we were required to know all there is to know about building and repairing a car and all its parts in order to drive one. The car’s guts are encapsulated in a tidy package for we civilians such that we just need to know five things; turn on/off, shift gears, gas, brake, steer.
Another important characteristic of functions is for a specific input always return the exact same output. Machines would be impractical to use or even useless to us if they weren’t predictable. Imagine what it would be like if the amount we stepped on the gas pedal fed gas into the engine in a linear manner as usual, but sometimes exponentially, sometimes according to the angle of the sun at the time of day.
That would be a dangerous car to take on the road! I’d be forced to drive very slowly because I never know when the function of the gas pedal will change. Therefore, I can’t go on the freeway because I’m forced to drive slowly. Do you see how any unpredictability drastically makes like tougher?
If a function does the exact same thing for the exact same input, complexity and risk are mitigated … even though complexity and risk can never in practice be eliminated, but at least on paper. Our example of the gas pedal only offered three different outputs and already I wouldn’t drive such a car.
All of our machines, including software 1, are composed of functions. Machines are a composition of parts, where each part does it’s thing and only its thing. Functions made of functions made of functions … However, the functions composing most of our software today do not have the quality of firing out the same output for the same input.
Note that a very interesting class of functions are those so-called “Machine Learning” (ML) models discussed towards the beginning of this blog. These are the product of “data scientists”, functions for guessing things such as how many people in a given population are at risk for becoming diabetic. Input whether you’re obese, over 45, high blood pressure, etc, and it guesses if you’re at risk for becoming diabetic.
They are often incorrectly referred to as Artificial Intelligence. Those functions are just machines. As is typical for machines, they beat the hell out of human effort as long as the rules don’t change.
So imagine again what it would be like if the gas pedal on a car didn’t react the same way for an amount the pedal is pushed down. Perhaps there is a rhyme or reason for its erratic behavior. What else is going on around us when the gas pedal responds to our foot pressure as usual versus when otherwise? This is the purpose for thought, the 4th Aggregate.
The 4th Aggregate – Relationships (Thoughts)
When our collection of functions in the 3rd Aggregate encounters something it isn’t designed for, it throws an exception an error message. Or if it’s a tangible machine, it may “crash and burn”. Usually, human programmers are alerted to these exceptions. They find the cause, use their human intelligence to formulate a solution, and implement it (modify and re-compose the functions).
As discussed towards the end of the 3rd Aggregate, to find a solution for a problem, humans search through webs of relationships – associations we’ve learned over our life. In the field of A.I. and neurology, there is the saying, what fires together, wires together. These relationships include:
Correlation – These are events that occur together to some degree. The simultaneous occurrence may be coincidence, due to another factor, or does indeed imply some sort of cause and effect.
Perceived cause and effect 2 – This is anything where we believe one thing will follow another because it has happened a lot.
Intended cause and effect – These are cause and effect of our human-engineered machines. We intend for gas to explode in cylinders, driving a transmission, that ultimately spins wheels.
For software, such relationships have traditionally been programmed by people. These are procedural relationships. However, in this Big Data age, relationships are wrested from a large history of facts by statistical algorithms.
Thoughts, whether human or machine, are about wonder, “what-if”. The human or machine then investigates what if by playing around with the web of relationships in the heads or in a graph database.
For example, in these pre-A.I. days, the process for optimizing something like low sales goes like this:
A sales manager experiencing low sales (pain in the 2nd Aggregate) would wonder what to look for in a customer most likely to purchase a luxury car now. That way, the sales personnel can maximize their sales performance (a goal of the 5th Aggregate) by spending more time with such people.
A data scientist will then run years of sales history and customer demographics through some sort of “machine learning” algorithm. That sales history and demographics data could be thought of as “sensed” in the 1st Aggregate.
The output of the algorithm are relationships between various customer attributes – gender, education level, age, income level, etc. This set of relationships is then deployed into production as a function (3rd Aggregate) of the enterprise, no different from anything we know to do as a human.
As it is for a human, the 4th Aggregate is the web of relationships as well as the process of finding possible solutions to a pain detected in the 2nd Aggregate. It’s not really that hard to trace a line of relationships when the dots are all connected. What is hard is finding a solution that minimizes negative side-effects. What are negative side-effects? Those are goals and desires that are hampered, the 5th Aggregate.
The 5th Aggregate – Metrics (Consciousness)
Like human creatures, corporate enterprises have goals, missions, and feel “pain”. Some of these corporate goals keep us from breaking laws and there are goals towards growth and higher profits. Goals not doing well register as pain. Metrics (nodes) statuses tie to functions of the 3rd Aggregate – think of these as pain and joy receptors. The web of these goals dictates the “values” or character of the system.
Businesses are founded upon strategies. Strategies are a web of “cause and effect”, a theory that if we perform some action, there will be a chain of effects leading to the satisfaction of our goals. For example, in the 2nd Aggregate, I mentioned Revenue Growth as a key metric. However, Revenue Growth itself isn’t a good thing. We could increase revenue easily by slashing prices, increasing revenue, but lowering profit – and increasing volume, which increases expenses, which also lowers profit.
The illustration below shows a simple strategy map for a doctor’s office. If the practice adds a doctor, that leads to more billable hours, which contributes to higher utilization which leads to higher revenue, which contributes to higher profit.
The strategy map above illustrates the trade-offs taken for some action represented by the blue boxes. If it weren’t for the fact that more doctors cost more (in many more ways than just salary), we could simply add as many doctors as possible to make as much money as possible. But there’s always trade-offs.
The complex world in which the aCEO must thrive goes way beyond such simple trade-offs. There are cascading webs of cause and effect that make almost everything hard. For example, the task of something as universal as hiring is so complicated that every business has a specialized Human Resources department. Do candidates have a long list of required skill, required education, are they trustworthy, will they get along with everyone, will they be assertive when necessary, cooperative otherwise?
A more realistic strategy map would be a big unwieldy mess, impractical as a visualization for our human brain. Fortunately, handling such a web of relationships is something computers can do better than us!
Each item is an issue because it interferes with some goal or another. Such magical employees meeting all requirements are rare and so delicate trade-offs (compromises) are made that will hopefully make all goals reasonably “happy”.
Because things are always changing in a business ecosystem, the aCEO will always be “buzzing” like a human brain dealing with constant change. A consciousness isn’t some static picture, but an in-motion process. It’s a process that goes something like this:
Things in the world change (1st Aggregate) …
Which could result in pains (bad KPI statuses in the 2nd Aggregate) …
As well as invalid results from functions in the 3rd Aggregate …
Which results in goals in the 5th Aggregate trying to minimize pain, re-balance itself …
And the 5th Aggregate employs the web of cause and effect in the 4th Aggregate to find candidate solutions, and test them out.
Exploring for a set of measured trade-offs that will eventually satisfy all goals, a solution to a minimizing pain, is actually the relatively easy part. Much tougher is organizing the map of relationships in the 4th and 5th Aggregates, connecting the dots. “What” is usually an easier question to answer than “How”.
The biggest breakthrough in software will not be further improvements in accuracy, speed, and volumes of data processed with well-defined tasks. It will be the ability to re-program itself, self-organize the connected dots.
From a Buddhist perspective, software today is dukkha, incapable of adapting itself to constant change. Software today is mostly rather “hand-wired” organizations of functions – meticulously hand-coded by highly-skilled human programmers. And that’s starting to change.
1 It’s not exactly correct to say that software is composed of functions since what appear to be functions (at the time of this writing) can return different results since it depends upon circumstances outside its inputs and/or the functions often do not handle exceptions (inputs it wasn’t designed to take in) well enough. Those two inadequacies account for much of the reason software has so many bugs. There is a movement towards “functional programming” that enforces at least the first issue, towards the goal of writing software that is less complex.
2 Any time I utter the phrase “cause and effect”, someone reminds me that “correlation does not imply causation”. I don’t think anyone actually does think correlation implies causation. It doesn’t make sense to do something for an effect unless you believe there would be the desired effect. Whatever decisions that are made by people of sound mind are based upon what they perceiveto be cause and effect relationship.
Today is January 13, 2019, the 8th Day of the 12th moon for 2018. It’s Lunar Bodhi Day!
Here is a little post I hope helps you meditate on a very important aspect of Enlightenment on this Bodhi Day: What are the Five Aggregates?
The Whole is Greater than the Aggregation of its Parts
Here’s what the “The Five Aggregates” are if you’re taking an online practice test and just need a fast answer: In Buddhist literature, it’s a collection of things/concepts that comprise our sentience – body, feelings, perception, thoughts, and consciousness.
The Five Aggregates plays a big part in the core lesson of the Heart Sutra. The most succinct way to state the lesson is:
The Universe is One inseparable thing, so stop acting like it’s a bunch of independent pieces.
Roughly, the argument of that lesson can be broken down like this:
Everything is always changing …
So any thing we think we see is just a temporary assemblage of other things …
So none of the things we use our brains to recognize really exists anymore …
Which renders as invalid any cause and effect relationships that we’ve learned about between those things which no longer exists …
Therefore, it doesn’t make sense suffering through our relentless and eventually futile attempts to coerce the Universe to what our minds compute based on that outdated information …
Rather, turn it around and learn how to better read and respond to the Universe.
However, point #6 above doesn’t mean we humans should become completely thoughtless, spineless, gooey, creatures. As many say in Buddhism, “there is a middle way“.
But even if we can intellectually understand that lesson, at our human realm of sentience it’s really difficult as our actual experience is more like that of a goldfish in a bowl. We’re trapped in our little cubicle of space with a whole lot going on outside of those glass walls, as well as a whole lot more we cannot ever see.
Let’s start this journey with Ralph Waldo Emerson’s quote, “Man is a piece of the Universe made alive.”
In this post I will cover the Five Aggregates one at a time, but remember that the whole that’s made up of those five aggregates is our Sentience.
An “aggregate” is the product of an assemblage of parts. That product could be a homogeneous collection of customer service reps, a stack of one dollar bills, or inventory of some mass-produced commodity. For such a collection of things, each member is interchangeable. These are things we count and measure in volume, for example, 50 cartons of milk or one-thousand dollars.
More interestingly, an aggregate could be a heterogeneous collection of parts. For example, an automobile, a surgical team, a hamburger, the Earth’s environment, or the Beatles. Such assemblages of parts into a whole are very different things from simply a “bag o’ parts”. Think of the music from each of the Beatles members’ solo work. All are very different nor as good as the music they created as a band.
Therefore, we say that the whole is greater than the sum of its parts. That’s not usually true for homogeneous aggregates where the whole is simply the sum of its parts – ten customer service reps is just ten customer service reps.
Each of the Five Aggregates is a heterogeneous aggregate. We’ll cover each shortly, but for now, let’s take the example of the 1st Aggregate, which includes our senses – sight, sound, taste, smell, touch. They are all very different kinds of information at our human level of existence. For example, the picture painted in our heads through the photons hitting our retinas are an aggregate of three kinds of rods (red, green, blue), a bunch of cones, the adjustable aperture of our pupils, etc.
As just mentioned above, our Sentience is an aggregate of the Five Aggregates. In fact, it doesn’t take much pondering to realize that practically everything we can think of is a hierarchical aggregate of aggregates – from the uber aggregate of the Universe itself, to the Earth’s ecosystem, to machines we design and build, to sub-atomic particles.
For example, a car is a hierarchical aggregation of aggregations. One of those parts of a car is the engine which is itself an aggregation of parts. One of those parts of the engine is the distributor, which is an aggregation of parts, …
We often call an aggregation “the Big Picture”. Do we see the forest or do we see the trees? It’s easy for us to lose sight of the big picture because we’re trained to specialize – break down the big picture into smaller pieces so each of us can become an expert in our assigned bite-sized piece. As specialized experts, we can optimize the hell out of each of those pieces, then assemble all the pieces back together into a theoretically better machine.
Overview of the Heart Sutra
This blog could be considered part of a set centered around the The Heart Sutra along with these three others:
If you haven’t read or don’t want to read those three blogs before continuing, the section below, Symbolic Thinking, sort of recaps the material covered in No Thing Exists and Everything Forms Every Thing. As for the notion of “The Other Shore”, think of it simply as the side of the Bay where the sun shines brighter and there’s a prettier view.
From Thich Nhat Hanh’s English translation of the Heart Sutra, The Five Aggregates are mentioned a couple of times such as in this excerpt:
That is why in Emptiness, Body, Feelings, Perceptions, Mental Formations and Consciousness are not separate self entities.
Deeper understanding of the Heart Sutra equates to peace of mind. The Heart Sutra is a succinct encapsulation of Buddhism which one can read within about two minutes, three tops. However, as “spot-on” as the Heart Sutra may be, what it encapsulates is ineffable without decades of experience mindfully soaking in its deepest wisdom.
For example, I know what Business Intelligence is – at least I hope I do, as that’s my job. But I can’t effectively pass on the full depth of that knowledge to someone, even through a big book. To fully understand that knowledge space, it takes years of pressure-filled work, exploration of thousands of dead ends (which usually comes in handy some time in the future), an endless succession of unknown unknowns, terrible mistakes incentivizing you to find betters ways, and the resourcefulness skill built from having the technological rug constantly pulled out from under you every other year.
Likewise, progress towards a full understanding of the Buddha’s teaching (enlightenment) goes well beyond reading the Heart Sutra over and over again or analyzing each word to death. Think of it more as a periodic test of your understanding gained through your direct life experience and guidance from those who offer advice from their own experience. Hopefully, this post and the other three I mentioned above is one of those sources of guidance.
Our capabilities provided through these Five Aggregates – senses, feelings, perceptions, thoughts, and consciousness – enable we flesh and blood creatures to “preserve” our form longer (live longer) in this indescribably dynamic reality in which we are manifested (Life on Earth). In our human case, “long enough” means to live long enough to mature enough to raise children. The Five Aggregates are our “smarts”, which enable us to some extent to outsmart the soul-less laws of physics – at least for a while.
If things didn’t change, we wouldn’t need senses, thoughts, consciousness. As Ringo said, “I think because things change.” We may recognize things such as our parents, our car, or our chair at work, but what we think we recognize is never exactly as it was before. “No man ever steps in the same river twice”, as Heraclitus observed. We could be seeing someone in disguise, they could be high on drugs, angry, therefore “different”.
Sentience for humans is centered around our awareness or ourselves being aware – self-awareness. That is, I’m aware of something I define as “myself”. My sentience involves my body, sensations about my body and other things out there, my primal emotion about what I’m sensing, my perceptions of what I’m sensing, my thoughts, and the ability to reflect upon those thoughts – the Five Aggregates.
We’re aware of our mortality, in fact, the mortality of other creatures as well. But that awareness is merely a by-product of the real value sentience afforded to us. That real value is that we can manipulate things in our environment. Our human forms are machines that can recognize things, form a map in our heads of how those things relate to each other, imagine what the rearrangement of these things will look like, and to a good extent mold the physical world to that vision.
Sentience isn’t a uniquely human thing. It’s a continuum onto which creatures of Earth fall at some point. Even an amoebae is an intricate assemblage of molecules resulting in the skill to conduct a species-perpetuating life; birth, eat, develop, evade being eaten, reproduce. There’s really no man-made machine as I write this that can do that with absolutely no human intervention.
Sentience isn’t just “thinking”. Most if not all animals can “think” to some extent. We humans are just much better at it than other animals. It also depends on how one defines “think”. For the purposes of this blog, “think” means that when presented with a decision to make, you’re capable of coming up with a novel answer that’s more effective than what you’d get plugging numbers into some rote formula.
And, although we humans are “sentient”, our sentience isn’t the epitome of sentience. Perhaps we’re at the top of one kind of sentience. But as our human brand of sentience may stand at the very top of a hill, there are other types of sentience throughout the Universe on peaks much higher than ours.
The Eternal Fishnu tells me that symbolic thinking is what the story of Adam and Eve is about. Adam and Eve were at first one with the Universe, until they made a choice to exercise their own will by going against God’s instructions not to eat the apple. The ability to exercise their own will is founded in symbolic thinking.
We are trained from the instant we’re born to recognize things – usually starting with Mama, Dada, then on to various toys, foods, etc. We’re taught the alphabet, a set of symbols, before we’re taught to recognize words (a symbol for a thing) before we’re taught to read a book (a symbol for a story).
The coffee mug sitting on my desk as I type is a thing. If I drop it on my foot, will I not feel pain? If I bonked my teeth with it, will I not require a trip to the dentist? The mug is a thing in the context of the “normal world”, that realm where you need to go to work to pay the bills and have no time to ponder enlightenment.
However, in the realm of your Dukkha, enlightenment, the Heart Sutra, and Buddhism, a “thing” is a symbol in your brain representing something we’ve sensed. Those things and the relationships between them form a “soft” model of the Universe from which we make all of our decisions.
Those things in our brains are organized into a map of relationships we’ve observed between those things, we’re able to perform experiments in our brain before taking physically irreversible actions. However, none of those symbols in your head, any of those “things”, really exist anymore – in the Buddhist context.
Those experiments we run in our head are the secret sauce to humanity’s ability to manipulate our world … as well as the source of our Dukkha. We take the continuous, fully inter-dependent processes and break them down into discrete things and discrete steps. If we think of any machine we humans have made, such as a watch (the mechanical sort), we can see all of the interacting gears, springs, and dial hands. We can understand how the cascading cause and effect from spring to gear, to gear, to dial works.
The Five Aggregates
The Five Aggregates comprises the high-level architecture of our human sentience – all that we feel and think, the foundation of decisions we make. In the context of the Heart Sutra, the Five Aggregates are discussed to methodically lay out the wisdom that all we think about is based on what is usually a highly flawed reading and interpretation of what’s really out there.
There are two over-arching points to keep in mind as you read through this section on The Five Aggregates. The first, as just mentioned, is that our reading and perception of what is around us is very prone to mistakes. Our symbolic thinking breaks down the oneness of the Universe into discrete things and discrete steps. That way, we can move around the pieces on the game board in our minds to predict what will happen. Thinking that this is an adequate model of the real thing is like thinking we can butcher a cow, reassemble the steaks, soup bones, organs and hide, and it will again be a live cow.
It’s all about building, utilizing, and maintaining a model of the Universe encoded in our neurons. However, this model is never nearly as good as the real thing, the Universe. In fact it’s pitifully inadequate.
Second, we really don’t have much control over anything going on around us, even though we want to think we do. We have some control over what is “close” to us. That is, what is physically nearby, not far off in the future, or what we have strong ties to, such as our possessions, our responsibilities at work, our close friends and family, etc. But that level of control, already not very well-founded, rapidly diminishes the farther out in time and space we go.
More interestingly, we don’t even have as much control over our own selves as we want to think. We have little control over how we feel about an event, what we think about it, and often what we decide to do about it. We don’t have much control over rather hard-wired instincts. What we often think of as our ability to control ourselves is our ability to suppress our expression of negative feelings that will get us into trouble. That’s not the same as genuinely expunging a negative feeling out of our mind.
Therefore, the point of this blog is to convince you of the futility of your desire to control things. Then you can settle into the world, at peace.
The 1st Aggregate – Body and Form
The Body and Form Aggregate is an aggregation of all that there is outside of ourselves and our sensory organs to detect what is out there. Those sensory organs sensing – sight, hearing, smell, taste, and touch – are the windows into our brain.
But practically everything we sense is flawed. Or at least information we sense will quickly become obsolete because everything is always changing. Right out of the gate, our sentience is prone towards huge opportunities for misinterpretation of the world.
At this point, it’s easy to think of the phrase, garbage in, garbage out. However, that’s not quite right. Our eyes see real photons. Our ears pick up real sound waves. Our noses and tongues detect real molecules. There’s no “garbage” being sensed – those photons, sound waves and molecules are right here, right now. Where there is garbage is in your brain where there are conclusions you come to founded upon outdated and limited information.
Imperfect Information and Information Overload
The philosopher, Bishop George Berkeley’s, “koan” of the tree falling in the forest (and no one was there to hear it, does it make a sound?) fantastically reminds us that although we don’t hear or see everything, that doesn’t mean it’s not there. What we sense with our senses is such a small part of the whole story that there’s an inherent degree of speculation to everything. There is so much going on in the Universe that it’s impossible for our brains to sense everything.
Just about all information we process in our brains is based on imperfect information – we don’t and can’t know everything. But none the less, we still must make our decisions based on what little we really know – the real world won’t stop to give us all the time we need to make a decision.
Although it’s easy to accept that we don’t and can’t know everything, it may be even tougher to accept that we shouldn’t know too much. Even if our brains were much larger, capable of sensing much more than it currently can, that information still needs to be processed. “Process” means connect the dots about what all is happening and make decisions fast enough to matter. This point is very loosely analogous to having 80 billion neurons in our head, but that the real information is in the quadrillion connections between them, connecting those neuron “dots”.
Sensing more information than our brain can effectively process is information overload. Imagine your eight bosses in a room overwhelming you with all they need and all the reasons why. Your brain will be so scattered that you wouldn’t be able to help even one of them as well as you can. That’s a minor illustration of information overload.
Nature worked out the problem of the immense volume of information in several ways. One is the natural tuning of our senses, limiting them towards the goal of optimized fitness for competing in the game of Life on Earth. Everything we’re geared to notice is a lean and mean package of sense capability, honed through thousands of generations of competition – light, sound, smell. All so the better to notice our predators or our prey smartly and quickly enough to matter – day or night, hot or cold, in the forest or on the savanna.
Pretty cool, huh? But it comes with the price of the atrophy of whatever doesn’t matter mostof the time. Our eyes have cells (rods) that detect red, blue, and green, but not yellow. Yellow rods would be redundant because we can also detect yellow as a combination of red and green. So if we hypothetically used to have yellow rods, if a human were to be born without them, it wouldn’t matter since there is a workaround. In fact it would positively affect that unfortunate yellow-rod-less human, as there’s room for more red, green, and blue rods.
Similarly, we don’t have receptors up our nose for every chemical – which means those left out chemicals have no smell, and so as far as our brain is concerned, it doesn’t exist. It can seem detrimental that we can’t smell everything, but if we could, that would again overload us with information. Now, if we really needed to smell more things, we could have evolved long snouts like dogs to expand the number of olfactory receptors.
So we don’t see, hear, smell, or taste everything – just what happens to have been the bag o’ powers most beneficial for our ancestors at the time and place of their lives, going back tens of thousands of generations. We’ve naturally evolved the bag o’ powers that recognizes the most important things – even though every now and then that trade-off of a wider breath of powers towards the goal of mitigating information overload fails us. But it works well enough for enough of us to mature enough to raise enough children. The heuristic of evolution is a “good enough”, not necessarily the “best”.
Further, much of what we sense undergoes a lot of massaging before it hits our thinking brain, like highlighting a picture for better presentation. For example, our eyes take in our three dimensional world on our two-dimensional retinas and uses algorithms in our brain to simulate three-dimensions. It even creates the illusion that we’re seeing a big view of what is in front of our eyes, even though only a small percentage of it is in focus at any instant.
Predator and Prey
If Life on Earth were a board game, it would be named “Predator and Prey”.
Our senses are mostly geared to sense change – i.e. the activity of our predators or our prey. Change is a good word in Buddhism, right? Ironically, at our human scale of existence, time frames of seconds and at the scale of creatures we would normally consider predators and prey, you often will not notice change – you won’t see your predators or prey move. Why? Because a primary tactic is to stay really still and/or pretend you’re something else so your predators walk right past you and/or your prey walks closer.
At our “normal” scale of existence things may seem unchanged, but that doesn’t mean things are not changing. Although we may be aware of how things work at smaller or larger scales, we live at the scale of human existence. Are we usually cognizant of the billions of bacteria on our skin? Or the electron buzzing in the atoms? Even at the scale of an insect, “normal” is different. Insects can carry balls of water as big as them.
The bottom line is that our senses don’t see everything, and that’s by design. The net of what nature evolved in humans to sense and ignore, at the end of the day, results in nothing less than our position as the apex animal of Earth.
The Sixth Sense
Guess what? We are psychic. Not like Miss Cleo, but we do have the ability to read the minds of other people. We have empathy because we need the ability to read other minds – predict what is in their heads, so that we can effectively work with them or not be fooled by them. This is another sense along with smell, sight, hearing. Big parts of our brain are dedicated towards recognizing very subtle things in faces, such as the minutest of eye twitch or smile. Without a word or a sound, we’ re pretty good at telling if someone didn’t like something we said, if they are happy, angry, we’re still on the same page, maybe even lying to us.
It’s not “psychic” as in fortune tellers, seeing into your future. There’s nothing magic, it is mind-reading, an innate skill developed in our human brains out of necessity we call Theory of Mind. It’s just reading body language, which we all have as a natural gift and can improve with training.
We wouldn’t be able to communicate anything more than the simplest of information if we weren’t able to read peoples’ minds to some extent. That “unspoken” communication in peoples’ expressions and body language sets swaths of context.
It would be too cumbersome to craft completely unambiguous messages to be of practical use while we’re working together. It’s very valuable for to me just see the look of fear or anger in your face and know I’d better run now and ask questions later. No need for you to write an essay first.
But of course, we’re still often wrong about what we sense people are thinking. We may see someone is angry or in a receptive mood, but incorrectly guess why, which can lead to wrong conclusions.
Further, they could be great actors faking a behavior. Lying is something every animal is capable of. For lower animals, it’s called camouflage – prey hiding from predators and predators hiding from prey to ambush them. We humans are just able to take lying to another level.
The 2nd Aggregate – Feelings
Think of feelings as our most primitive and simple of thoughts. For example, when we were simple one-celled creatures swimming around a billion years ago, we felt water too hot or too cold and moved away from it. These are the sorts of mechanisms that are simple enough that we can imagine them evolving without some sort of “intelligent design”. I’ve described these simple “thoughts” in a series of blogs starting with The Root of Dukkha – Part 1 – Envy.
The Buddha mentions that feelings can be pleasant, unpleasant, and neutral. In our human logic, we tend to seek pleasant feelings and avoid unpleasant feelings. However, the actions of seeking and avoiding are the cause of stress. That’s because in our human realm of existence, which is little more complex than that of an amoeba, seeking pleasure and avoiding pain isn’t as easy as binary choices of yes/no, up/down, or fight/flee.
The only widely effective solution in a complex world is to be like the Zen master, Hakuin, 100% accepting of what Is. Or, at the very least, learn to be happy with what you have, and know things can always be worse – believe me, it’s very true. I know that’s boring, cliched advice, but it is a necessary start that does lead to 100% acceptance like that of Hakuin.
This doesn’t mean we ignore our feelings. But we have no control over them, so ignoring them is too a kind of avoidance. Besides, our feelings are often wise advisers nature honed for us over countless generations. We keep faith and patience that they will pass through us knowing they are temporary.
However, although the ability to take comfort in knowing everything is temporary is a vast improvement, it really means you still don’t get it. It means you’re still judging things as good and bad as opposed to seeing the Universe as One, where all the wonders out there are due to opposing Yin and Yang forces.
The 3rd Aggregate – Perception
The 3rd Aggregate is the aggregate of the things we “know”, things we’ve learned, which we use to assess what’s happening now or could happen later. That knowledge is applied to information we’re sensing from the 1st Aggregate, our senses. We smell turkey roasting, we see people we know coming over to our house, and we assess that the Thanksgiving festivities are at hand. This is what’s happening now, right around us.
The 2nd and 3rd Aggregate differ in that the former is an aggregate of fairly hard-wired instincts, whereas the latter is more like software, the things we’ve learned and can modify as we learn more and our environment changes. Both are similar in that they process what we’re sensing and outputs actions we should take.
However, the problem is the world is constantly changing, as described in the 1st Aggregate. We sense only a very small fraction of what is going on, and it’s filled with deception due to the game of predator and prey. But we do need to make decisions before life makes a decision for us. We can’t all just sit still doing nothing. So we settle on something that seems plausible enough.
That means our brains often make mistakes.
I don’t mean to sound so negative. It only sounds terrible to humans because we’re aware of our mortality. Remember, we are spawned from a world where pretty much every creature makes a living devouring another creature. That’s how Life on Earth works, that’s why Life on Earth is over three billion years old, immortal for all practical purposes.
Note too that our perceptions of things dictates what we focus on. So going back to the 1st Aggregate of body/form, we direct our senses to focus on certain things, ignoring everything else. That’s how magicians fool us and fish get hooked. But we’re better than that. Aren’t we? Yes, we can think … we can reason.
The 4th Aggregate – Thoughts
Imagine having the intelligence of a fish. For that level of intelligence, if it looks like a worm, smells like a worm, and moves like a worm, it’s a worm. Or is the lure of a clever human fisherman?
It’s here in the 4th Aggregate (and the 5th Aggregate) where humans are much more developed than fish, reptiles, and plants. Thoughts and Consciousness are the keys for our highly adaptable skill. We’re harder to fool by our non-human predators or prey, we can manipulate our environment towards something we envision, and we can learn new skills.
Thoughts are what takes our minds to the past so we can better predict the future. But it comes with the cost of diluting our attention towards what’s happening right now. If what’s happening right now is that Thanksgiving festivities are underway, we may have thoughts of eventual family bickering based on past experience, regrets of weeks of dieting being wiped out, the mess that must be cleaned up.
It’s our unparalleled prowess in the 4th Aggregate, thoughts, which separates us from the other animals. By “unparalleled”, I mean that to be fair to other animals, our thought capability isn’t exactly exclusive to humans, since most have some level of thought capability. We’re just alone on the far end of the spectrum, an outlier of intelligence on Earth.
The 4th Aggregate is our “smarts”. It causes us to question what the 3rd Aggregate recognizes. It causes us to be on guard because we’re aware of the risk of something bad happening. It motivates us to action based towards what we predict to be something good. It develops novel solutions where the 3rd aggregate fails to find a fit tactic to employ.
It’s not quite as easy to say that perceptions are a picture of what we’re seeing now versus thoughts are what we make of it, what else could be going on, and what we should do about it. Thoughts and perceptions go back and forth, with thoughts coloring, distorting what we’re perceiving. There are dozens of types of biases applied to what we are perceiving.
We don’t record our memories as we would film. We reconstruct much of what we try to recall, and it’s always at risk for being pulled together in a slightly different way.
Thoughts come into play in perception when what we’re seeing doesn’t make sense. Thoughts direct us to look for something else that play provide a clue or something that may prove a hypothesis.
Thoughts possibly can be massively parallel, responding to stimuli, but putting two and two together, trying to recognize something not quite right.
Thoughts predict the future by projecting from the past. Perceptions and Thoughts are logical computations. They aren’t always right. Sometimes events are recalled, assembled, and are a little off. Occasional Error is a critical component of genes and memes.
The 5th Aggregate – Consciousness
I don’t know what consciousness is. There are good ideas out there, but all leaving gaping, unsatisfying holes. Understanding of consciousness is a nut really smart people have been trying to crack for thousands of years. For now, let’s go with re-purposing U.S. Supreme Court Justice Potter Stewart’s famous phrase on pornography, “I don’t know what consciousness is, but I know it when I see it.”
Materially, consciousness is the aggregation of all our goals and desires. These include our responses to our primal emotions (2nd Aggregate), addictions, hopes and dreams, habits, and even our sacred beliefs. They are encoded in a yet undecipherable code distributed all over the web of neurons and synapses of our brain. But that’s not our consciousness any more than being a human means having human DNA.
That material description of consciousness is as dead as a photo of a Giant Sequoia or Bryce Canyon versus being there in person – with walking around it, smelling the smells, sharing moments of awe with fellow visitors, the tactile feel of the heights of the tree and walls. Even a video or art by the best artists ever falls short of fully capturing the conscious experience.
There is a big hint in the paragraph above. When we’re really there at Sequoia National Forest and Bryce Canyon, our senses are fully engaged, dynamically reacting to incomparably richer information than we could get from a photo or video. But “richer information” doesn’t mean higher resolution of what we’re seeing (5 megapixels vs 20 megapixels), or simply adding smell and touch to the experience.
The richer data stimulates more things in our head, and they are all connected in that web of neurons, synapses, and all the other stuff (fluids, neurotransmitters, “glial cells”) crammed into our skull. All our goals and desires compete with each other. For example, our addiction to junk food versus our desire to be healthy versus our limited time to shop for healthy food versus the convenience of junk food.
Our consciousness is made up of this tangled web of competing goals and desires. But consciousness is the dynamic process of trying to make all our goals and desires happy. Consciousness is a verb.
The Aggregate of the Five Aggregates is Us
So how does this knowledge of the Five Aggregates help us to be at peace? We see from much of the discussion above that the Five Aggregates are highly integrated. That means changes to one of the Aggregates affects them all.
We can conclude a few actionable items that will help us:
The fewer goals and desires that we have, the easier it is to make all of them happy, and so the less “turmoil” it makes for your brain.
What we think we sense and what we make of it is most likely incorrect to varying degrees. Therefore, we’re better off not clinging to what our experience tells us about what we’re sensing. Always have an open mind.
We have little control over the Universe and that our mostly futile attempts to control it is the source of our own suffering.
Our minds are not something we can see in a static blueprint. Our minds are a dynamic process as it is for the entire Universe. But we spend the bulk of our energy trying to stop the world, rather than using our sentience just to provide an occasional nudge to something caught in an endless loop.
See You on the Other Shore
There is a big trade-off taken for the prowess of our thoughts, the 4th Aggregate. Caught in the storm of the turmoil of thoughts, we suffer because we’re mercilessly sloshed around by years of our hoarded crap trying to predict uncountable possible outcomes.
If we didn’t have thoughts, we could be as content as the other animals just being what they are – perfect cows, perfect ants, perfect fish, perfect trees – unhindered by what has happened and what could happened. Cows, ants, fish, and trees perceive and react. With our advanced level for thought, life isn’t nearly that simple.
Zen and Buddhism are the skills we utilize on board our boat (our mind and body) to the other shore, just as we would utilize sailing skills on a literal boat. That journey starts from the shore of content animals perceiving and reacting, One with the Universe – to the other Shore, where surprisingly, we’re also One with the Universe! So what’s the point of getting to the other shore?! In between is a journey across a treacherous ocean where we’re at an Ugly Stage, half-way between the shores.
On the treacherous ocean we suffer. We’re aware of our mortality. We’re cut off from the “solid” ground from which we emerged and all it offers. It’s a journey all humans are on because our elders pulled us onto the boat. They intend to take us to a better place, but the vast majority don’t make it all the way to that other shore. Caught in the middle of the ocean, caught between animal and fully awaken sentience, they suffer until the end.
Our ability to think didn’t develop in us for the lofty human goals and desires we strive for. It developed to better protect ourselves from lions and tigers and bears, as well as to better hunt rabbits and seals and deer. But it swept us off the safe shore of just being another animal to another shore.
What awaits us at the other shore? Well, almost magically, we’ll have our cake and you can eat it too! That is, we’ll be sentient creatures capable of design and manipulation, without all the suffering we experience due to clinging. If we see that everything is connected, we’re all one, not fragmented, we don’t cling to the past, we don’t cling to beliefs, we’ll be a force of the Universe. We’ll again be One with the Universe, without our Dukkha and the effects of our Dukkha on everything around us.
The Eternal Fishnu is not just that blue, rubbery, fish-like figure with a yellow cap and a red backpack. The blue rubber figure is just an Earthly manifestation, an icon, of The Eternal Fishnu for my Earthly, visually-oriented and symbolically-thinking brain.
The actual Eternal Fishnu is a phenomenon of the Universe like gravity and time, an ethereal phenomenon gracefully nudging us from the friction-filled bumpy ride of dukkha to the smooth sailing of Buddhahood. He is a phenomenon similar to the gentle agitation over time on a chaotic bucket of mud, settling it into orderly layers of gold at the bottom and progressively lighter things moving towards the top. The teachings of Fishnu gently agitate our brains to re-wire, smoothing out our arduous transition from animal to intelligent designers.
A Human’s Bodhi
On the other hand, as Ringo put it, I’m a buddha trapped in a man’s Bodhi. I’m not an ethereal phenomenon as is Fishnu. Meaning, although I’m awake, seeing through the illusions computed by my mind by cultivating a perpetual beginner’s mind and 100% acceptance of what Is, my sentience at least for now is intimately intertwined in a human animal. My Bodhi still lives the life of a man of today with joys and the challenges that are just part of being a member of society and to a wider extent, a creature of Earth.
What is a “man’s Bodhi”? Or rather, a “human’s Bodhi”? It is waking up to the reality of what is out there, as opposed to living the life of the compelling delusions of our brain – that pitifully inadequate model of the Universe we rely on to figure out what’s out there. Our brain allows us to outsmart physics by enabling us to imagine outcomes, all in the safety of our heads, before committing to real, physically-irreversible actions.
Imagination is playing with the model of the Universe we’ve built in our heads, those symbols, things, and how they relate to each other in a big web of cause and effect. But there is much disconnect between the model of the Universe we’ve built in our brains and the actual Universe. That model in our heads is outdated because:
Everything is always changing. Impermanence means our database of information in our brains must constantly be updated. Is it? Can it?
We don’t know everything – we haven’t experienced much and the Universe, well, it’s a big place. The reasoning behind the reincarnation thing is that it would take thousands of lifetimes, from thousands of roles, to fully experience everything.
Our brains do not require actual energy to imagine something like tipping over a car. We don’t know exactly how much energy it takes to move a boulder until we actually do it. But we can imagine some approximation that’s probably not exactly right. It’s what I call “cartoon physics”.
We’ve heard it said that there are many paths to enlightenment. In fact, we can almost say that it’s a given that everyone’s path will be different because everyone’s brain is wired differently. In addition, though, there are also many paths to sentience. Human sentience is just one kind of sentience. So different kinds of sentience requires a different Dharma – the teachings of a Buddha.
However, whatever our kind of sentience, human, Klingon, or plant, the destination is the same. That destination is a fully in-sync connection to the Universe. For humans, we suffer because we chose to believe the illusion of our brains that we are independent of the One Universe.
The circumstances from which all of life on Earth evolved over these last three billion years, involving interactions between countless creatures and physical conditions, lead to our brand of sentience for which Siddhartha Gautama’s Four Noble Truths make sense. Siddhartha Gautama is the Teacher of humanity during our “era”.
If we were to create an artificial intelligence and it clunked along with a kind of dukkha of its own, repairing that dukkha probably wouldn’t be carried out by teaching it the Four Noble Truths. It will require the “dharma” of the programming prowess of the developers of the A.I. (Coincidentally, as I took a little break during this writing, I see a friend on Facebook posted a meme; a baby’s face with the caption, “I refuse to take a nap … Is that resisting a rest?”)
The Era of Fishnu Buddha
Actually, The Eternal Fishnu is crashing Siddhartha Gautama’s party. The Eternal Fishnu’s “party” was roughly what we call the Devonian period, about 400-300 million years ago, when to put it simply, “fish” moved onto land. Why? There really wasn’t any “reason”, there aren’t “reasons” where evolution is concerned. But that vertebrate architecture of those fishes, that pliable exterior over a sturdy frame, lead to whole classes of new creatures, the tetrapods, which includes us.
Those new classes of creatures were only possible outside the friction and buoyancy of water to the lighter friction of air and the stability of ground. The new set of problems the tetrapods faced would force new innovations in creature design.
Were the fish sentient? Probably not as we humans experience sentience – nothing like the anthropomorphized version of fishes where Ariel, Sebastian, and Flounder lived, under the sea. But fish are life, where every molecule is intricately placed. All life has an intelligence of its own. It must. It filled this planet with wonders for three billion years! That intelligence probably wouldn’t get much out of the Four Noble Truths.
What was the dukkha of those fishes for which Fishnu appeared? Well … being a fish out of water for one. Why did those first fish that popped their heads out of the water continue to venture more and more out of its protection? Maybe they found a concentration of food at the shores, like scum accumulates at the sides of a bathtub. Maybe then the tide would go out leaving a bunch stranded there. Maybe most dried up, but some were sweatier or had thicker skin and were able to survive the low tide exposing them.
From our human perspective, it’s hard to understand why there would need to be a Buddha for fish. Again, Evolution never has a “plan”; there was never a plan for vertebrate fish to venture onto land. But once the process got under way, a Buddha helped them through that ugly stage of the process to the other shore. Looking towards the future, Maitreya, the Buddha of the future will only appear when the Four Noble Truths no longer makes any sense.
The Life of a Buddha Trapped in a Human’s Bodhi
So far, the writing on this blog site mostly speaks to the first two of the three Zen Stories that comprise the foundation of The Eternal Fishnu’s teaching: the beginner’s mind (The Empty Cup) and 100% acceptance of what is (Is that so?). However, I haven’t written as much about the third story, Picking Up the Bag.
The first two stories are about awakening to your Buddhahood and the third is on staying awake. One can’t fully appreciate the third story without fully digesting the first two. In a nutshell, the story of picking up the bag is that of an enlightened person demonstrating that enlightenment comes by simply dropping your burdens … just like that … and what comes after is picking it back up and continuing along your way.
So what’s the point? What changed? He’s still some schmuck carrying around a heavy bag. Well, nothing changed, but everything changed. When you realize your brain is a lousy model of what is really out there and you stop relying on it, you are now free to merrily, merrily, merrily go about your business as a member of society and creature of Earth – like in the nursery rhyme. Except it’s not life that is “but the dream”. Life is very real. It’s those beliefs of your brain where dreams lie – whether the dreams we have when asleep or the dreams we have wide awake.
Let’s look at a more pragmatic example from “normal” life where you drop your burdens, then pick them back up. Say that at work you know vast improvements must be made to a critical process; say this is an overhaul of an ETL system so severely scarred with years of hacky patches (a very common scenario) that it breaks every day. And it’s getting worse, to the point where some day you may not be able to fix it quickly.
But your plate is already overflowing with work for which you are directly accountable. Even if you did somehow muster the energy to tackle those improvements along with your “real” work, you’re afraid of stepping on the toes of others. You may be rocking the boat for folks not wanting the boat rocked with a few years to retirement.
Many will be under the delusion that nothing is wrong, that you’re a paranoid Chicken Little. Very often, if something hasn’t yet happened, people often think it couldn’t happen. From their perspective, nothing is wrong because, because they don’t see your heroics, your 60 hour weeks patching the dam all over the place. From their perspective, if it ain’t broke, why fix it?
(Before continuing, I do want to note that worrying about some awful future seems to contradict being in the now – we’re not supposed to worry about things that haven’t happened and may not even happen. The advice is more so to not let that future which may not happen complicate our efforts and drain our energy now.)
In any case it has to be done and you’ve decided you’re going to pursue it, taking all the hits, tripping over yourself and others with hesitations. As it is with treating cancer, the company really has no choice but to put everything on hold and take its metaphorical chemo or radiation therapy.
What if, however, the CEO fully comprehends the gravity of the situation and is brave enough to act on it? The CEO is willing to put her neck on the line against the whiny shareholders and gives you carte blanche to do whatever you need to do, providing funding, protecting you from retaliation. With or without the CEO’s permission, you’re going to do it, but you’re now 100% free with the CEO’s permission. At that moment, not next week or tomorrow, you become that mission.
So what changed? The CEO gave you permission to do what you must. But that’s the human, corporate life. Real life is much better. The Universe doesn’t need to give you permission to live like a Buddha in this life because it never said you had to live with dukkha in the first place. It’s our choice to cling to or be defined by the past, be terrified by imagined futures, and wait for good luck to save us.
For life in general, you’re Free! You need not fear anyone, any embarrassment, and pain. Drop your burden … just do it. No leashes, no buttons to push. Empty your mind of all that crap you’ve been hoarding, and accept what Is around you, right here, right now.
That bag we drop then pick back up holds the three billion year history of Life on Earth, every single event. It’s inseparable from our bodies, the vehicle in which we travel this world, more inseparable than any other organ. But it’s not a burden, no more than your car is a burden to you on the freeway. The burden our insistence on relying solely on our imperfect brain.
Feel the freedom. Then pick it up and embrace life like a person with nothing to lose and nothing to gain! This is what is meant by, “When you become nothing, you become Everything.” When you have no constraints, you are all things. Like a sub-atomic particle no longer observed, it goes from a single point to a field.
You are still a human animal, but you can now continue along your way with the genuine fearlessness you were born with.
A Full Life
Do you remember this scene from Groundhog Day?
Phil: What would you do if you were stuck in one place, and every day was exactly the same, and nothing that you did mattered?
One of the guys at the bar: That sums it up for me.
We watch the movie, “Groundhog Day”, and think it novel to relive the same day over and over. But with the mindless way most of us lives our lives, is it different from “Groundhog Day”? Our days go by like a blur because we’re hardly fully present. It’s like driving down a boulevard past countless strip malls. Unless we get out and spend time at each, they all look exactly the same from a inside a car at 40 mph. We see our days from a high level of I went to work, did stuff, came home, had dinner, went to bed.
Either your mind can give in to the Universe and let it take it on its wonderful ride in this amusement park called Earth, or you can be codependent on the Universe, being happy only when and if it decides to appease you.
My Teacher’s Enlightenment
Lastly, this blog is dedicated to the one-year anniversary of the Enlightenment of His Holiness, the Rubber Ducky Buddha of Joliet, on this date last year, November 3, 2017.
From The Rubber Ducky Buddha of Joliet today (my translation from His native “Mack mack mack …”):
“Humans are designers. That’s the human shtick. You only get to use that shtick to design your way out of problems. It’s those problematic times when humans can be humans, as I can be a duck when in water.”