My word for the elusive aspect of human thought still lacking in synthetic imitations is “slippability”. Human thoughts have a way of slipping easily along certain conceptual dimensions into other thoughts, and resisting such slippage along other dimensions. A given idea has slightly different slippabilities — predispositions to slip — in each different human mind that it comes to live in.
Once the finished work exists, scholars looking at it may seize upon certain qualities of it that lend themselves easily to being parametrized. Anyone can do statistics on a work of art once it is there for the scrutiny, but the ease of doing so can obscure the fact that no one could have said, a priori, what kinds of mathematical observables would turn out to be relevant to the capturing of stylistic aspects of the as-yet-unseen work of art.
but no matter how you phrase it, the possession of this ability to break out of loops of all sorts seems the antithesis of the mechanical. Or, to put it the other way around, the essence of the mechanical seems to be in its lack of novelty and its repetitiveness, in its trappedness in some kind of precisely delimited space.
I don’t know about the consistency of the genius of Bach, but I did work with the great American architect Louis Kahn (1901—1974) and suppose that it must have been somewhat the same with Bach. That is, Kahn, out of moral, spiritual, and philosophical considerations, formulated ways he would and ways he would not do a thing in architecture. Students came to know many of his ways, and some of the best could imitate him rather well (though not perfectly). But as Kahn himself developed, he constantly brought in new principles that brought new transformations to his work; and he even occasionally discarded an old rule. Consequently, he was always several steps ahead of his imitators who knew what was but couldn’t imagine what will be. So it is that computer-generated ‘original’ Bach is an interesting exercise. But it isn’t Bach — that unwritten work that Bach never got to, the day after he died.
— William Huff, quoted in Metamagical Themas by Douglas Hofstadter
When things get complicated enough, you’re forced to change your level of description. To some extent that’s already happening, which is why we use words such as “want”, “think”, “try”, and “hope,” to describe chess programs and other attempts at mechanical thought. Dennett calls that kind of level switch by the observer “adopting the intentional stance.” The really interesting things in AI will only begin to happen, I’d guess, when the program itself adopts the intentional stance toward itself!
So if there is a mathematical theorem telling us that no program whatsoever will be a perfect self-watcher, able to catch itself in any conceivable kind of infinite regress, well, that is simply a statement that perfect intelligence is unreachable — something that ought to please us rather than dismay us, since it would be rather horrible and disappointing if someone came up with some finite program after a while, and could legitimately announce, “Well, folks, here it is at last: the end-all of intelligence, a perfectly intelligent program”.
This is an important idea: the test of whether a concept has really come into its own, the test of its genuine mental existence, is its retrievability by that process of unconscious recall. That’s what lets you know that it has been firmly planted in the soil of your mind. It is not whether that concept appears to be “atomic”, in the sense that you have a single word to express it by. That is far too superficial.
Here is an example to illustrate why. A friend told me recently that the Encyclopaedia Britannica’s first edition (1768-71) consisted of three volumes. Volume I: “A-B”; Volume II: “C-L”, and then Volume III: the rest of the alphabet. In that edition, 511 pages were devoted to topics beginning with ‘A’, while the last volume had 753 pages altogether! (I guess that in those days there weren’t yet many interesting things around that began with letters between ‘M’ and ‘Z’.) Hearing this amusing fact instantaneously triggered the retrieval of the memory, implanted in me years and years ago under totally unremembered circumstances, of how records used to be made, back in the days when there was no magnetic tape and the master disk was actually cut during the live performance. The performers would be playing along and all of a sudden the recording engineer would notice that there wasn’t much room left on the plate, so the performers would be given a signal to hurry up, and as a result, the tempo would be faster and faster the further toward the center the needle came. I think it is obvious why the one triggered retrieval of the other. And yet — is it obvious?
On the surface, these two concepts are completely unrelated. One concerns printed matter, books, the alphabet, and so on, while the other concerns plastic disks, sounds, performers, recording techniques, and so on. However, at some deeper conceptual level, these really are the same idea. There is just one idea here, and this idea I call a conceptual skeleton. Try to verbalize it. It’s certainly not just one word. It will take you a while. And when you do come up with a phrase, chances are it will be awkward and stilted — and still not quite right!
So much for the “constructive”. We finally come to the prospective, also known as the productive. Myhill’s characterization of it is this: “A prospective character is one which we cannot either recognize or create by a series of reasoned but in general unpredictable acts”. Thus it is neither effective nor constructive. It eludes production by any finite set of rules. However — and this is important — it can be approximated to a higher and higher degree of accuracy by a series of bigger and better sets of generative rules. Such rules tell you (or a machine) how to churn out members of this prospective category. In mathematical logic, works by Tarski and Gödel establish that truth has this open-ended prospective character. This means that you can produce all sorts of examples of truths — unlimitedly many — but no set of rules is ever sufficient to characterize them all. The prospective character eludes capture in any finite set.
As his prime example outside of mathematical logic of this quality, Myhill suggests beauty. As he puts it:
Not only can we not guarantee to recognize it [beauty] when we encounter it, but also there exists no formula or attitude, such as that in which the romantics believed, which can be counted upon, even in a hypothetical infinitely protracted lifetime, to create all the beauty that there is.
Thus beauty admits of a succession of ever-better approximations, but is never fully attainable. Beauty and irrationality are often linked. Is it coincidental that the first example of such a notion of something approximable but never attainable in a finite process is called an “irrational” number?
[…]
If we see the aim of art as the production of all possible objects of beauty (which is doubtless an oversimplification, but let us adopt that view nonetheless), then each individual artist contributes objects in a particular style. That style is a product of the artist’s heredity and formation, and becomes a hallmark. To the extent of having an individual style, any artist is sphexish — trapped within invisible, intangible, but inescapable boundaries of mental space. But that is nothing to lament. Artists in groups form movements or schools or periods, and what limits one artist need not limit another. Thus, by the fact that its boundaries are wider, a school is less sphexish — more conscious — than any of its members.
But even the collective movement of a school of art has its limits, shows its finitude, after a period of time. It begins to wind down, to lose fertility, to stagnate. And a new school begins to form. What no individual can make out clearly is perhaps seen collectively, on the level of a society. Thus art progresses towards an ever wider vision of beauty — a “prospective” vision of beauty — by a series of repeated “diagonalizations”: processes of recognizing and breaking out of ruts. As I like to put it, this is the process of jootsing (jumping out of the system) to ever wider worlds.
This endless jootsing is a process whose totality (so says Gödel) cannot be formalized, either in a computer or in any finite brain or set of brains. Thus one need not fear that the mechanization of creativity, if ever it comes about, will mark the end of art. Quite the contrary: It is a day to look forward to, for on that day our eyes will open — as will those of computers, to be sure — onto whole new worlds of beauty.
Each of us — even the Mozarts among us — exhibits a “cognitive style” that in essence defines the ruts we are permanently caught in.
Far from being a tragic flaw, this is what makes us interesting to each other. If we limit ourselves to thinking about music, for instance, each composer exhibits a “cognitive style” in that domain — a musical style. Do we take it as a sign of weakness that Mozart did not have the power to break out of his “Mozart rut” and anticipate the patterns of Chopin? And is it because he lacked spark that Chopin could not see his way to inventing the subtle harmonic ploys of Maurice Ravel? And from the fact that in “Bolero” Ravel does not carry the idea of pseudo-sphexish music to the intoxicating extreme that Steve Reich has, should we conclude that Ravel was less than magical?
On the contrary. We celebrate individual styles, rather than seeing them negatively, as proofs of inner limits. What in fact is curious is that those people who are able to put on or take off styles in the manner of a chameleon seem to have no style of their own and are simply saloon performers, amusing imitators. We accord greatness to those people whose “limitations”, if that is how you want to look at it, are the most apparent, the most blatant. If you are familiar with his style, you can recognize music by Maurice Ravel any time. He is powerful because he is so recognizable, because he is trapped in that inimitable “Ravel rut”. Even if Mozart had jumped that far out of his Mozart system, he still would have been trapped inside the Ravel system. You simply can’t jump infinitely far!
The point is that Mozart and Ravel, and you and I, are all highly antisphexish, but not perfectly so, and it is at that fuzzy boundary where we can no longer quite maintain the self-watching to a high degree of reliability that our own individual styles, characters, begin to emerge to the world.
When a computer’s operating system begins thrashing (i.e., bogging down in its timesharing performance) at around 35 users, do you go find the systems programmer and say “Hey, go raise the thrashing-number in memory from 35 to 60, okay?”? No, you don’t. It wouldn’t make any sense. This particular value of 35 is not stored in some local spot in the computer’s memory where it can be easily accessed and modified. In that way, it is very different from, say, a student’s grade in a university’s administrative data base, or a letter in a word in an article you’re writing on your home computer. That number 35 emerges dynamically from a host of strategic decisions made by the designers of the operating system and the computer’s hardware, and so on. It is not available for twiddling. There is no “thrashing-threshold dial” to crank on an operating system, unfortunately.
Why should there be a “short-term-memory-size” dial on an intelligence? Why should 7 be a magic number built into the system explicitly from the start? If the size of short-term memory really were explicitly stored in our genes, then surely it would take only a simple mutation to reset the “dial” at 8 or 9 or 50, so that intelligence would evolve at ever-increasing rates. I doubt that AI people think that this is even remotely close to the truth; and yet they sometimes act as if it made sense to assume it is a close approximation to the truth.
In cases of judgement, whether it be of one musical composer over another, one potential title or subtitle for a book over another, or whatever, the top level pretty much has to wait for decisions to percolate up from the bottom level. The masses down below are where the decision really gets made, in a time of brooding and rumination. Then the top level may struggle to articulate the seething activity down below, but those verbalized reasons it comes up with are always a posteriori. Words alone are never rich enough to explain the subtlety of a difficult choice. Reasons may sound plausible but they are never the essence of a decision. The verbalized reason is just the tip of an iceberg.