Longform quotes from Peter Watts’ Blindsight every Monday! Here’s last week’s post. Today’s quote is rather long: it’s a string of infodumps (not just one) that build up the main argument of the book, which I really really like (especially since it’s backed by extensive amounts of published research). This argument is basically continued in Watts’ Echopraxia, the second book in the series.
You invest so much in it, don’t you? It’s what elevates you above the beasts of the field, it’s what makes you special. Homo sapiens, you call yourself. Wise Man. Do you even know what it is, this consciousness you cite in your own exaltation? Do you even know what it’s for?
Maybe you think it gives you free will. Maybe you’ve forgotten that sleepwalkers converse, drive vehicles, commit crimes and clean up afterwards, unconscious the whole time. Maybe nobody’s told you that even waking souls are only slaves in denial.
Make a conscious choice. Decide to move your index finger. Too late! The electricity’s already halfway down your arm. Your body began to act a full half-second before your conscious self ‘chose’ to, for the self chose nothing; something else set your body in motion, sent an executive summary—almost an afterthought— to the homunculus behind your eyes. That little man, that arrogant subroutine that thinks of itself as theperson, mistakes correlation for causality: it reads the summary and it sees the hand move, and it thinks that one drove the other.
But it’s not in charge. You’re not in charge. If free will even exists, it doesn’t share living space with the likes of you.
Insight, then. Wisdom. The quest for knowledge, the derivation of theorems, science and technology and all those exclusively human pursuits that must surely rest on a conscious foundation. Maybe that‘s what sentience would be for— if scientific breakthroughs didn’t spring fully-formed from the subconscious mind, manifest themselves in dreams, as full-blown insights after a deep night’s sleep. It’s the most basic rule of the stymied researcher: stop thinking about the problem. Do something else. It will come to you if you just stop being conscious of it.
Every concert pianist knows that the surest way to ruin a performance is to be aware of what the fingers are doing. Every dancer and acrobat knows enough to let the mind go, let the body run itself. Every driver of any manual vehicle arrives at destinations with no recollection of the stops and turns and roads traveled in getting there. You are all sleepwalkers, whether climbing creative peaks or slogging through some mundane routine for the thousandth time. You are all sleepwalkers.
Don’t even try to talk about the learning curve. Don’t bother citing the months of deliberate practice that precede the unconscious performance, or the years of study and experiment leading up to the gift-wrapped Eureka moment. So what if your lessons are all learned consciously? Do you think that proves there’s no other way? Heuristic software’s been learning from experience for over a hundred years. Machines master chess, cars learn to drive themselves, statistical programs face problems and design the experiments to solve them and you think that the only path to learning leads through sentience? You’re Stone-age nomads, eking out some marginal existence on the veldt—denying even the possibility of agriculture, because hunting and gathering was good enough for your parents.
Do you want to know what consciousness is for? Do you want to know the only real purpose it serves? Training wheels. You can’t see both aspects of the Necker Cube at once, so it lets you focus on one and dismiss the other. That’s a pretty half-assed way to parse reality. You’re always better off looking at more than one side of anything. Go on, try. Defocus. It’s the next logical step.
Oh, but you can’t. There’s something in the way.
And it’s fighting back.
Evolution has no foresight. Complex machinery develops its own agendas. Brains—cheat. Feedback loops evolve to promote stable heartbeats and then stumble upon the temptation of rhythm and music. The rush evoked by fractal imagery, the algorithms used for habitat selection, metastasize into art. Thrills that once had to be earned in increments of fitness can now be had from pointless introspection. Aesthetics rise unbidden from a trillion dopamine receptors, and the system moves beyond modeling the organism. It begins to model the very process of modeling. It consumes ever-more computational resources, bogs itself down with endless recursion and irrelevant simulations. Like the parasitic DNA that accretes in every natural genome, it persists and proliferates and produces nothing but itself. Metaprocesses bloom like cancer, and awaken, and call themselves I.
The system weakens, slows. It takes so much longer now to perceive—to assess the input, mull it over, decide in the manner of cognitive beings. But when the flash flood crosses your path, when the lion leaps at you from the grasses, advanced self-awareness is an unaffordable indulgence. The brain stem does its best. It sees the danger, hijacks the body, reacts a hundred times faster than that fat old man sitting in the CEO’s office upstairs; but every generation it gets harder to work around this— this creaking neurological bureaucracy.
I wastes energy and processing power, self-obsesses to the point of psychosis. Scramblers have no need of it, scramblers are more parsimonious. With simpler biochemistries, with smaller brains—deprived of tools, of their ship, even of parts of their own metabolism—they think rings around you. They hide their language in plain sight, even when you know what they’re saying. They turn your own cognition against itself. They travel between the stars. This is what intelligence can do, unhampered by self-awareness.
I is not the working mind, you see. For Amanda Bates to say “I do not exist” would be nonsense; but when the processes beneath say the same thing, they are merely reporting that the parasites have died. They are only saying that they are free.
“—because for one thing, if it were really so pernicious, natural selection would have weeded it out,” James was saying.
“You have a naïve understanding of evolutionary processes. There’s no such thing as survival of the fittest. Survival of the most adequate, maybe. It doesn’t matter whether a solution’s optimal. All that matters is whether it beats the alternatives.”
“Well, we damn well beat the alternatives.” Some subtle overdubbed harmonic in James’ voice suggested a chorus: the whole Gang, rising as one in opposition.
“It’s true,” Sarasti told her, “that your intellect makes up for your self-awareness to some extent. But you’re flightless birds on a remote island. You’re not so much successful as isolated from any real competition.”
“It doesn’t bug you?” Sascha was saying. “Thinking that your mind, the very thing that makes you you, is nothing but some kind of parasite?”
“Forget about minds,” he told her. “Say you’ve got a device designed to monitor—oh, cosmic rays, say. What happens when you turn its sensor around so it’s not pointing at the sky anymore, but at its own guts?” He answered himself before she could: “It does what it’s built to. It measures cosmic rays, even though it’s not looking at them any more. It parses its own circuitry in terms of cosmic-ray metaphors, because those feelright, because they feel natural, because it can’t look at things any other way. But it’s the wrong metaphor. So the system misunderstands everything about itself. Maybe that’s not a grand and glorious evolutionary leap after all. Maybe it’s just a design flaw.”
“But you’re the biologist. You know Mom was right better’n anyone. Brain’s a big glucose hog. Everything it does costs through the nose.”
“True enough,” Cunningham admitted.
“So sentience has gotta be good for something, then. Because it’s expensive, and if it sucks up energy without doing anything useful then evolution’s gonna weed it out just like that.”
“Maybe it did.” He paused long enough to chew food or suck smoke. “Chimpanzees are smarter than Orangutans, did you know that? Higher encephalisation quotient. Yet they can’t always recognize themselves in a mirror. Orangs can.”
“So what’s your point? Smarter animal, less self-awareness? Chimpanzees are becoming nonsentient?”
“Or they were, before we stopped everything in its tracks.”
“So why didn’t that happen to us?”
“What makes you think it didn’t?”
It was such an obviously stupid question that Sascha didn’t have an answer for it. I could imagine her gaping in the silence.
“You’re not thinking this through,” Cunningham said. “We’re not talking about some kind of zombie lurching around with its arms stretched out, spouting mathematical theorems. A smart automaton would blend in. It would observe those around it, mimic their behavior, act just like everyone else. All the while completely unaware of what it was doing. Unaware even of its own existence.”
“Why would it bother? What would motivate it?”
“As long as you pull your hand away from an open flame, who cares whether you do it because it hurts or because some feedback algorithm says withdraw if heat flux exceeds critical T? Natural selection doesn’t care about motives. If impersonating something increases fitness, then nature will select good impersonators over bad ones. Keep it up long enough and no conscious being would be able to pick your zombie out of a crowd.” Another silence; I could hear him chewing through it. “It’ll even be able to participate in a conversation like this one. It could write letters home, impersonate real human feelings, without having the slightest awareness of its own existence.”
“I dunno, Rob. It just seems—”
“Oh, it might not be perfect. It might be a bit redundant, or resort to the occasional expository infodump. But even real people do that, don’t they?”
“And eventually, there aren’t any real people left. Just robots pretending to give a shit.”
“Perhaps. Depends on the population dynamics, among other things. But I’d guess that at least one thing an automaton lacks is empathy; if you can’t feel, you can’t really relate to something that does, even if you act as though you do. Which makes it interesting to note how many sociopaths show up in the world’s upper echelons, hmm? How ruthlessness and bottom-line self-interest are so lauded up in the stratosphere, while anyone showing those traits at ground level gets carted off into detention with the Realists. Almost as if society itself is being reshaped from the inside out.”
“Oh, come on. Society was always pretty— wait, you’re saying the world’s corporate elite are nonsentient?”
“God, no. Not nearly. Maybe they’re just starting down that road. Like chimpanzees.”
“Yeah, but sociopaths don’t blend in well.”
“Maybe the ones that get diagnosed don’t, but by definition they’re the bottom of the class. The others are too smart to get caught, and realautomatons would do even better. Besides, when you get powerful enough, you don’t need to act like other people. Other people start acting like you.”
Centuries of navel-gazing. Millennia of masturbation. Plato to Descartes to Dawkins to Rhanda. Souls and zombie agents and qualia. Kolmogorov complexity. Consciousness as Divine Spark. Consciousness as electromagnetic field. Consciousness as functional cluster.
I explored it all.
Wegner thought it was an executive summary. Penrose heard it in the singing of caged electrons. Nirretranders said it was a fraud; Kazim called it leakage from a parallel universe. Metzinger wouldn’t even admit it existed. The AIs claimed to have worked it out, then announced they couldn’t explain it to us. Gödel was right after all: no system can fully understand itself.
Not even the synthesists had been able to rotate it down. The load-bearing beams just couldn’t take the strain.
All of them, I began to realize, had missed the point. All those theories, all those drugdreams and experiments and models trying to prove what consciousness was: none to explain what it was good for. None needed: obviously, consciousness makes us what we are. It lets us see the beauty and the ugliness. It elevates us into the exalted realm of the spiritual. Oh, a few outsiders—Dawkins, Keogh, the occasional writer of hackwork fiction who barely achieved obscurity—wondered briefly at the why of it: why not soft computers, and no more? Why should nonsentient systems be inherently inferior? But they never really raised their voices above the crowd. The value of what we are was too trivially self-evident to ever call into serious question.
Yet the questions persisted, in the minds of the laureates, in the angst of every horny fifteen-year-old on the planet. Am I nothing but sparking chemistry? Am I a magnet in the ether? I am more than my eyes, my ears, my tongue; I am the little thing behind those things, the thing looking out from inside. But who looks out from its eyes? What does it reduce to? Who am I? Who am I? Who am I?