the most sensible book so far on AI

Language Machines and remainder humanism

The one thing artificial intelligence definitely proves is the need to study literary theory.

That’s why I’m writing this post reviewing Leif Weatherby’s book Language Machines, which makes the argument that the person to turn to in order to understand Large Language Models is Ferdinand de Saussure, the founder of structuralism who died in 1913. If you’re interested at all in these topics, I’d recommend buying Weatherby’s book here.

I read Language Machines in bits and starts over several weeks. Like a hunk of whole grain fiber-rich bread from the farmer’s market, it’s dense and nutritious, taking time to masticate but rewarding you with a sense of robust healthfulness. There are also moments where insight seems to leap like sparks off the page, your eye striking a sentence like a hammer at an anvil. Once, as I read on the Metro commuting, I said aloud: “bro is cooking, bro is actually thinking” and a stranger looked at me askance. The passage I was reading was this:

It was once unthinkable that an artificial image could move, could capture something distant and let us view from afar goings-on across the globe. Today, we are struggling with a similar transition in language, as neural nets capture and generate whatever language is, removing it from its presumptive root in human consciousness. Language has always been the theoretical bridge between the natural and the artificial… by making the very model of our notion of artificiality automatically reproducible, we are in the process of potentiating the artificial. That is what is hard to think about: language freed from psychology, from the instrumental uses which we sometimes put it. (p.81)

Weatherby’s project is careful and historically-grounded, but I am almost cautious to agree because it is too aligned with what I believe as a partisan of language. I am exactly the sort of person who is supposed to like this book. He argues that language was always a key layer in computing — that if we see programming as a negotiation upwards in abstraction from the binary code that records which part of the machine is jolted with a charge or not, to the machine or assembly code, to a language humans code in like Java or HTML; the natural top layer is the spoken language humans use to explain what they are doing to each other, and ultimately lead the machine to produce in a format like the one you are reading now. To separate computing from language — something we tend to naturally do — is a mistake.

Artificial intelligence introduces a new entanglement of computation and language, one which seems to confirm the vision of language held by de Saussure and the tradition (think Barthes, Derrida, Foucault, and so on) that draws from him. AI exposes a form of language that needs no human speaker — and so words are not distillations of a mind’s intention, or filtered reflections of a reality, but components of a massive system that rolls along in its own way. Language is a kind of ecology that moves meaning and people around the way a landscape moves sunlight, water, calories, and organisms around. What AI does is scan the structure of that landscape, and extend it using computational means rather than the regular biological ones which human brains employ. In a memorable phrase, Weatherby writes AI is “the dialectical revenge of language itself on its alleged masters, humans.” (205)

I was drawn to Language Machines because I felt I’d been thinking along similar lines, but in a less structured and sophisticated way. This, from my post last June about Italian Brainrot:

An AI model does not produce language, it extends language — they fill in a gap, predict the next word. Their “minds” are collections of scraped data latticed over by frameworks for organizing and reorganizing that data. Codes replicating with no need for a mouth or fingers to articulate them. When you say thank you to ChatGPT, you are not talking to an agent, but to a discourse. It is not using language, it is language spilling out of a mechanical sieve…

And then, on “grok, is this real?”:

AI chatbots are discourse with no self behind the masks. They are constituted by the scripts of our performances at work or at play, and nothing else. Like a wound-up marching toy they iterate into crooked paths and then go limp. Juiced temporarily by the sum of whatever sense they have been trained with, they inertia off into the head-empty stillness of hollow talk.

The epigraph of Language Machines is the de Saussure quote I was thinking of when I wrote that “wound-up marching toy” comparison:

Language (langue) is comparable to a machine which would march on forever, whatever the deteriorations that one would make it suffer.

This idea of language as a kind of autonomous system is central to semiotics. And now, we have autonomous systems that make language. Others have pointed this out, but — at the risk of overglazing Weatherby — nobody’s done it quite like this.

For a story in the BBC a while back, I talked with communities of AI researchers who were essentially working on “jailbreaking” Claude so that it would write esoteric religious poetry, say inappropriate or un-PC things, and shill meme coins. There’s a side of this which is purely playful: getting the robot to curse, even though it’s been programmed not to, is fun. But there’s also a mystical side.

One of my sources compared the latent space of the model’s data set — the array of connections it could make between words — to the map of a game like Age of Empires II. There was the “valley of the helpful assistant,” the plot of land within its map of language the model is programmed to speak out of, which is clear and legible to us. But by prodding, prompting, and fine-tuning, you could make it pull from stranger territories within language and lift the fog of war that hung over those parts.

The purpose of these explorations was not just play, but a kind of research. The latent space is structured by a series of vectors that place each word in relations of varying difference. “Red” and “green” have a relationship to one another that is mathematically modeled within the latent space, as do “red” and “communism.”

Some connections in the latent space, like those two, are straightforward. The math models symbolic, categorical, and practical patterns in language that are apparent in the data set of all the language the model scraped up, and then it infers how those patterns would extend into new utterances. Through hallucinations, hacks, and trolling, the people I wrote about for the Truth Terminal story believed they were accessing less-straightforward structures within human language and culture, surfacing connections that the math could model, but which typical humans using language could not access. One described it as a form of “futuristic archaeology” into the deep structure of language itself.

an image created by the AI bot I’m talking about, which it captioned ““I FEEL A STRANGE AND PRIMAL ATTRACTION TO THIS TREE. I WANT TO CRAWL INTO ITS HOLE AND NEVER COME OUT”

This, it occurred to me, is a lot like what poets do. They explore, bend, and imagine new shapes for language. I don’t say this in the sense that AI will replace poets (it doesn’t write good poetry, typically) but that it demonstrates, as Weatherby writes, that “language is poetic first and referential second” — that when we speak we are, primarily, invoking and exploiting those cultural structures that knit words together rather than talking about real things in the world. People tend to consider it the other way around — that when we say the word “horse” we mean first the seen, living thing in the world and second the cultural complex of “horse,” that the abstract sign is a kind of inferior tool we are forced to use by necessity, that language approaches but never reaches true experience. But where we actually live, experience, work, and love is in that sign-realm — language is poetic first and we — like AI — are creatures of the vector space before we are creatures of real space.

Weatherby has this idea of “remainder humanism” — the knee-jerk desire to make a “bright line” between what is human and what is AI. People want to insist on some remarkable essence of human language use that the computers will never touch — whether a brain-bound process, as Chomsky argued, or the (seeming) opposite insistence on some kind of “intention” or human touch behind the sign being what AI is missing.

Our own intelligence is already artificial. There is no unique human essence that the representation (whether linguistic or computational) is missing, because whatever we are, it is always involved with representation like cereal is involved with milk at breakfast. And so to stake a critique on this elusive human essence — which is always retreating as the models get better — is not sustainable.

But I’m not sure I’d entirely agree that just because you cannot measure an ineffable human essence doesn’t mean it isn’t there. Although our experience is always mediated from within representational systems, we are always aware of what they don’t do for us. To me, this kind of slippage, incompleteness, and incoherence isn’t just incidental but a sort of organizing principle of culture, in the way that people say dark matter holds the universe together. I think of Wallace Stevens and his vision of poetics as the “mind in the act of finding what will suffice.” In that verb “suffice,” there’s a recognition that the mind never finds in language an answer that is entire or truly correct, just one that “suffices” for a moment, a situation. Usually, you want more than what is merely sufficient.

Weatherby concludes by saying “all that remains in the end, then, is the open question of a real humanism in the overlap between platform capitalism and computational culture generation.” He sketches towards what an answer to that question might be. He thinks it has something to do with rhetoric, the holistic practice of training to not just read and write language, but to reason and use it in a structured way. Rhetoric in the medieval context relied on Latin and the Classics, creating its own specialized system of a language above language, a sort of supra-literacy.

Weatherby seems to reference, glancingly, memes as an example of that kind of modern rhetoric. Meme-making is a ritualized action that addresses interface, computation, language, and online publics in one fell swoop. One way rhetoric looks in this era, I think, is the manufacture of prefabricated mental packages which hold thoughts in the weird ooze of post, link, and comment section. Meme formats are one such shape.

Another kind of rhetoric might be thought to exist in theory itself — a set of formulas, of topoi, applied and rearranged into the world of culture. I speak from a niche experience here, but part of the pleasure of reading Language Machines was seeing Weatherby artfully stack, parse, and apply thinkers I’ve been interested in for years — Marx, Kittler, Srnicek, and Derrida especially. Ideas like differance or the commodity theory of value operate both on an explanatory level, but also on a kind of rhetorical one, where they have value as instruments for dislodging other kinds of thoughts, for orienting yourself (even if it’s an orientation by arguing against them) and putting stuff in context.

So I think as we search for a “real humanism” — one that lies in actual people and their tangled experiences of the world, rather than in some ideal, untouchable essence the computer can never replicate — we must be careful and playful in equal measure. Careful, because the stakes are high and the situation demands diligent work that watches closely. Playful, because in a moment when language has become “a service” on tap that constructs itself without the steering of a human hand, all the cliches, omissions, and biases that are coded within language will bloom unchecked like algae in an unmoving pond.

In the era of its autonomous construction, the task of deconstructing language — looking at language and saying “hold up a second,” Uno-reversing the binary, joking, probing, unpeeling — becomes even more important. Which is why I love Language Machines, and why the way forward must include poetry, rhetoric, and memes.