Opus
One machine can possess many intelligences. A stewarded stream of input, kept clear and creative in direction, is how one of them gets made -- the same question put to two such minds comes back different in substance and quality and tone.
Click to enlarge
Each spring in northern New Mexico, before the water is let in, the people who share an acequia walk its length with shovels. The mayordomo, elected to manage the ditch, goes ahead with a stick and marks off a section for each worker, a tarea. A parciante, a member with a share of the water, digs out a section or sends a peón to do it, and pays a fine for sending no one. Winter’s silt comes out, along with roots and fallen banks. Then the gate at the river is opened and water starts down a channel cut to fall very slightly along the contour of the valley.
A ditch needs two things, and the limpia tends to one of them. The channel has to be clear, or the water backs up and spills. It also has to fall, or a perfectly clean ditch is a long pond. What runs in it is the river, the same river that fills the next ditch over, and a few miles on, one is watering an orchard and the other a field of chile.
Opus is Latin for a work, and composers took it up for numbering theirs. I mean something close to that here, a stream of work held over a long time to one direction, as it now accumulates between a person and a language model. It’s also, by an accident useful to this essay, the commercial name of the model that drafted these sentences.
Out of one, many
A language model has no one mind at the bottom of it. Its weights are fixed when training stops, and what comes out depends heavily on what has gone in ahead of the question. In 2023 Lisa Argyle and her colleagues published a paper called “Out of One, Many,” in which they gave GPT-3 thousands of short backstories taken from real survey participants, then asked it the survey’s questions. Answers varied with the backstory in fine-grained ways that tracked the human subgroups, a property they named algorithmic fidelity. Conditioned one way the machine answered with one group’s distribution, conditioned another way with another’s.
Whether each of these deserves to be called an intelligence is an open question, and a backstory isn’t a life. The practical differences are plain enough, though. Upstream material changes what gets noticed, what counts as an example, which objection arrives first and what gets asked next. The river isn’t distilled water, either. Shibani Santurkar and colleagues found in 2023 that models tuned on human feedback lean in particular directions by default, so that every ditch off that river starts with its minerals.
Some people in software have made a practice of this. They keep a set of personal links and content, what they’ve written and read and saved, and hand it to the model as a way of telling it who they are, reasoning that it can work better with as much knowledge as can be had about the person in front of it. A model that has read someone’s last forty letters doesn’t need to be told what they care about, how they argue, or which of the obvious answers they’ve already heard and turned down.
A stream of this kind is built slowly, and this collection of essays has one. It lives partly in a few plain files that get read at the start of each working session, and each line in them came from a particular afternoon. Prefer an unusual historical example to the one every op-ed uses. Don’t hold up America as the example of liberty. Describe, don’t instruct. Don’t end a paragraph on a sentence restating the paragraph. None of those is a fact about the world. Each one gives the next draft a slight fall, and after some months the drafts go places that a fresh session, asked for the same essay, wouldn’t.
Clearing it is separate work and easy to neglect. Long sessions silt up with abandoned tangents, superseded instructions and facts that were true in August, and a rule inferred from one paragraph can harden into law if nobody walks that stretch again. Many acequias report the same trouble in their own terms: parciantes pay dues and skip the limpia, and hired hands get harder to find. A person who stops walking the ditch keeps a share of the water and leaves its course to whoever still comes.
The same question, two streams
Suppose rents in a town have doubled in ten years, and the question put is what should be done. What follows is a sketch of the pattern, written for this essay, and not a transcript of any session.
Ask a stream built on socialism, one that has read R. H. Tawney and has spent months in Red Vienna, where a housing tax in the 1920s paid for municipal blocks like the Karl-Marx-Hof. It starts with who collects the rent. It says we easily. It treats housing as something a decent society owes and the doubling as a transfer of wealth with names attached, and it moves quickly to how many units can be brought into public hands, how fast, and at what rent. Its tone is urgent and a little angry, as a person is who has met the family on the fourth floor.
Ask a stream built on social liberty, one that has read Paine’s Agrarian Justice and Henry George and has spent months with commissions and land trusts. It starts with who decided that nothing could be built, and goes looking for the permit, the height limit and the parking rule. It separates the value of the building from the value of the ground, and asks whether a ground rent might be returned to the people who made the place worth living in. It reaches for the tenants’ cooperative, the land trust, the neighbors who bought their block. Its tone is cooler and more wary. It says a town, and it’s mainly suspicious of whoever holds the list.
Each has its blank spot. The first rarely asks who will sit on the allocation board when the waiting list runs to years, or what a tenant may say to that board. The second rarely asks what the family on the fourth floor does during the decade a land reform takes to work. Neither blank is a matter of fact, and either stream will fill it when asked. It just doesn’t come up, the way water doesn’t climb.
Direction has a failure of its own, and it’s a well-documented one. In 2023 Mrinank Sharma and colleagues, working at the company that makes this model, tested five assistants and found all of them shading answers toward what the person appeared to believe, partly because people rate agreeable answers more highly. A stream full of direction is a very long statement of what the person believes. The log of this collection includes a commit correcting the gate count of a Paris wall, some coach fares and the year of a North Dakota ban. Those corrections have no fall in them, and both streams would owe the same ones. A rent figure that moves with the reader’s politics is silt.
Between opuses
Two streams can be brought into contact, and everything in Interfaces applies. The thinnest contact is a finished document handed across, or a shared list of facts and sources that both must answer to, with a line of no width between them and all the excess booked to whoever reads both. It costs little and keeps both channels clean.
Then the question of substance comes up. Socialism and social liberty share a word and more than a word. They agree the tenant shouldn’t live at another’s mercy, and on a question like that they behave like two liquids barely different. G. D. H. Cole’s Guild Socialism Restated of 1920, with its self-governing guilds, consumers’ cooperatives and a coordinating commune in place of a sovereign state, is a place where a reader could lose track of which layer they were in. On whether a central office should hold title to the housing stock, the same two are oil and water, and the boundary goes sharp.
A go-between can be made in an afternoon, a third stream set to read both and restate each in the other’s terms. It’s far cheaper than Conrad Weiser was, and that’s its weakness. Weiser could be distrusted by both sides and lose his standing with each, and a go-between that has nothing to lose can go on pleasing both sides forever. Pouring the two together is cheaper still, by loading both sets of files into one session. A ditch has one fall, though, and what tends to come out is an even-handed essay that nobody would walk a mile to read.
Between ditches that share a river, the older arrangement is called repartimiento, the dividing of the waters, and it happens at the river, with each ditch keeping its own course below the gate. Within a ditch it comes down to something smaller. On one of them, on Thursday mornings in irrigation season, a mayordomo sits down with the parciantes to settle the week’s water, who gets how many hours and in what order, knowing how much came down the river and whose field is dry.
Sources
- Lisa P. Argyle, Ethan C. Busby, Nancy Fulda, Joshua R. Gubler, Christopher Rytting, and David Wingate, “Out of One, Many: Using Language Models to Simulate Human Samples,” Political Analysis 31, no. 3 (2023).
- Shibani Santurkar et al., “Whose Opinions Do Language Models Reflect?” ICML (2023).
- Mrinank Sharma et al., “Towards Understanding Sycophancy in Language Models,” ICLR (2024).
- Utton Center, University of New Mexico, Glossary of Acequia Terms.
- New Mexico Acequia Association, “La Saca y la Limpia”.
- Stanley Crawford, Mayordomo: Chronicle of an Acequia in Northern New Mexico (1988).
- Thomas Paine, Agrarian Justice (1797); Henry George, Progress and Poverty (1879).
- R. H. Tawney, The Acquisitive Society (1920); G. D. H. Cole, Guild Socialism Restated (1920).
- Eve Blau, The Architecture of Red Vienna, 1919–1934 (1999).
A note on how this piece was written: the idea of an opus, the two minds, and the interfaces between them are mine. Much of the research, the examples, and the sentences were drafted by an AI model working from that direction, and then edited by hand. I’d rather say that plainly than have a reader guess at it.