The Limits of Automation

Too much time is wasted because of the assumption that methods already in existence will solve problems for which they were not designed; too many hypotheses and systems of thought in philosophy and elsewhere are based on the bizarre view that we, at this point in history, are in possession of the basic forms of understanding needed to comprehend absolutely anything. — Thomas Nagel, The View from Nowhere.

One way to think about the challenges we’re facing in our day-to-day and professional lives is to classify them based on our knowledge of our knowledge.

Put another, less mind-bending way: do we know how to solve it? And if so, do we know how we’re doing it? Can we explain everything in such exact detail so that even a machine can imitate our process, or are there blanks that have to be filled in by experience? This neatly categorizes any problem to one of four kinds.

The Known Knowns are the things we know how to solve and exactly how we’re solving them. These problems are reducible to their individual constituent parts and can be automated as factories, recipes, and algorithms. The solutions are straightforward, even if sometimes finicky and exacting. They lend themselves to mnemonical, atomic, modularized forms. They are what we hope to arrive at, so that we can give the keys to a well-oiled machine and continue with the more interesting parts of our lives.

The Known Unknowns are the things we know will forever be out of reach. These are the inherent limits, difficulties, and impossibilities of physical, mathematical, and computational reality — the Bekenstein bound, Gödel’s incompleteness theorems, and computational complexity separations are just some of the results that perpetually obstruct our way. We can only accept them and work around them.

The Unknown Knowns, or I know it when I see it, are the things we do know how to solve, but aren’t exactly sure how we’re doing so. These are problems of both high-level subjective and cultivated taste and low-level sensorimotor skills. (Hence the dangerous apple-bite economical polarization in which automation forces the comfortable center into becoming either a cog or a maverick).

There’s taste: picking the right song for the mood, the right word for a sentence, the right color for a painting. And sense: reading facial expressions, navigating in a packed crowd, deciding if a pause is a break for thought or a space for response. The doors leading to the engine room of the mechanics involved seem forever barred to us. We can only observe and probe how we handle these problems indirectly and from the outside.

Trying to outline these problems we see our pen taking many turns. There’s a lot of ifs and buts and not necessarilies and sometimes, and always there’s those pesky special cases, a slight resemblance of sorts, and tiny critical details preventing any kind of elegance from forming. These problems refuse to submit to a digestible explanation, a clear story, a napkin calculation, an equation we can do in our heads, or a categorization that can be easily shelved and retrieved. They are not succinct nor simple to describe.

Up until recently the unknown knowns were the prerogative of humans, laborers and curators alike. Machine Learning has made it possible for computers to carve their own niche here using optimization of universal function approximators.

With enough labeled examples and computational capacity there’s no such task we cannot automate an imitation of: collect a list of what’s this and what’s that, sketch some dividing lines at random to start, and gradually move the lines around — squeezing, scaling, rotating, smoothing, and bending — progressively getting a better approximation, a tighter shape around what we already know to be true. Our hope by the end of this process is to have a shape that hugs our existing knowledge in such a way that allows the right kind of as-yet-unseen examples to comfortably fit, while excluding the rest.

Can this process of automation based only on assumption and past experience capture everything we don’t already know? Can it shed light on unseen variations, hidden explanations, and rare combinations? Can it create or discover anything new? The answer is a resounding, uh, maybe, sometimes.

Lucky for us, many of the situations we encounter are similar to those we’ve already experienced. Whatever difference there is does not completely alter the situation and can usually be resolved by playing around with the same known ingredients. We can interpolate from the past, even extrapolate, and usually get things right when the foundations are stable and unchanging.

But, as the usual disclaimer goes: past performance is no guarantee of future results. Fortunately or not, our reality is an ever-growing ever-changing non-ergodic uncontained self-referential mess. There’s no guarantee that anything — and certainly not everything — will remain the same. In life, reactive predictions can only get us so far. The landscape is always evolving: following along means being a step behind. The story of the future cannot be told today.

The Unknown Unknowns are the black swans, the things we don’t even know that we don’t know. The things we can’t even give an example of. By the sheer act of thinking about them we give them form and make them imaginable, known, and expected. There is no specific constructive instance to point at. The specifics of the unknown can only be talked about subjectively and rationalized retrospectively. Like capturing a wild animal, any instance you find is already in the process of being tamed.

No matter how much we advance, there is always a constant stream of startlingly fresh unknown unknowns to find. And even though most will forever remain in darkness and we’ll never be aware of their existence — let alone figure them out — there is a smaller infinity always at the cusp, ripe for discovery. In time, we’ll get to know them, get to have a feeling of what is and what isn’t about them, and, with considerable effort, articulate the why and automate the how.

As we better our grasp, the specific instances we encounter move toward the knowable end of the continuum, but their class can remain an everlasting source of new experiences. There will always be new clichés. The memes and neologisms that make you feel old and confused will not stop. Always new ways for people to express themselves: new stories to tell, new art to inspire, new music to touch. Though our worldview is now clear on some laws and theorems that stumped the brightest of minds for centuries, we will always have more mathematical truths and proofs to discover.

We can imagine the growth of knowledge as a sphere slowly pushing outwards in all directions. At its core is the hard-fought solidified knowledge we’ve assimilated to such a degree we can wield it as we please. The mantle is more fluid: techniques are given names, processes are formed, rules of thumb get enshrined — all on their way to being set in stone. Further up, there’s wilderness at the surface. Here no rules apply, for if we find a law that describes a phenomena we could quickly drag it below. That is our goal: To cast the unknown in malleable forms.

The unknown unknowns are hard to find but can be easy to understand. We can hardly imagine how unobvious these things were, how much skill and perseverance were involved in figuring them out. It took generations of brilliant effort to come up with the things we now expect high-schoolers to master. We ask ourselves how we could not foresee the now oh-so-obvious market bubble or the second-order effects of a foreign policy yet we forget that only in the eighteenth century, about a hundred years after calculus was first formally described, did the concept of a sandwich enter Europe.

So far, the unknown unknowns have only been tackled successfully by evolutionary forces and the human mind, using insight and serendipity. The separation, where our methods of automation fail, and only mind and life succeed, is at the frontier of the new. Here, the unexpected is a recurring theme, and there’s no hope in imitation, whether it mimics a static pose or a vector of motion. Following a path is not charting a course. The data is not the data-generating process. Imitation might be good enough, until it isn’t.

Back on the comfortable side of the line, where the known is on its way to being controlled and automated, there is little point to flair, invention, and exploration, as all you need once found is to follow the recorded steps. Over time, for any specific thing, automation always wins. And that’s okay. That’s what we want. If you already know perfectly well what you’re doing, what’s the point of doing it? There’s plenty else to discover.

As Popper has pointed out, the future course of human affairs depends on the future growth of knowledge. And we cannot predict what specific knowledge will be created in the future — because if we could, we should by definition already possess that knowledge in the present. — David Deutsch, The Fabric of Reality.