Put a Person There
On what a defense is allowed to cost the people it is for
A levee stands on the riverbank it was built to hold.
The weight of it presses into the soil it exists to keep from washing away. Every sandbag. Every sheet of steel. Every foot of packed earth, bearing down on ground that is already wet, already shifting, already being eaten from below by the thing the levee was built to stop. The levee needs the bank to hold. The river is taking the bank. The defense and the attack occupy the same ground.
This is not a design flaw. It is the condition. A levee still needs ground that can bear it.
I said last time that the levee stands on the ground it is defending. I was talking about the Rio Grande, straightened before I was born and held in place by a wall of its own floodplain. They straightened it down here too, in Las Cruces, where I live now. I wrote it as a fact about a river. It is also the shape of every defense I have ever helped build.
I am an architect. For years my work was designing the systems that decide, in the dark, whether a transaction is a crime. The rules. The models. The alert that fires at two in the morning when something looks wrong. Every one of those depended on something upstream: on the data being real, on the clocks being honest, on the network carrying the signal without bending it, on the people who got the alert trusting the thing that sent it. Pull any one of those out and the detection is not weaker. It is nothing.
Now scale this up to a country.
Every institutional defense against disinformation depends on trust.
- The deepfake detectors.
- The media literacy programs.
- The coordination frameworks with the acronyms.
- The early-warning tools with the machine learning inside.
All of them need you to trust that the institution naming the threat is legitimate. That the tool is accurate. That the people passing the warning along mean it. That the air the warning has to travel through is not already poisoned.
And trust is exactly what the attack is for.
Not as a side effect. As the objective. The point is to wear down a population’s ability to trust its own elections, its own information, its own neighbors. The attacker does not need to win an argument. The attacker needs you to stop believing arguments can be won.
The defense requires the resource the attack targets. The levee stands on the bank the river eats.
The trap with no clean exit
Go hard: Remove the content. Ban the accounts. Coordinate the takedowns. Turn the ranking down on whatever looks coordinated. And you become the thing the attacker said you were. That story was seeded first. The institutions are censors. The government decides what you may think. Every move they can see confirms it. The defense is not wrong. The audience was prepared to read it as proof.
Go gentle: Teach media literacy. Build cultural resilience. Fund the long programs. And the clock betrays you. Those programs take years. The ground was being softened for decades. A generation grew up inside feeds built to hold attention, during the years when a mind is learning what other people think of it, by systems that owed them nothing. By the time the literacy class reaches a student, the student has spent thousands of hours inside a machine that taught them to distrust exactly the kind of institution now offering the class.
Every defensive move risks speeding up the erosion it was built to stop.
I have tuned that dial in fraud work. Set the rules tight and the declines pour over honest customers. Set them loose and the losses pour through.
Albuquerque learned a version of this in concrete. The arroyos I grew up beside used to catch the monsoon, slow it, and let it sink into the aquifer under the city. In the 1960s the city poured concrete into them for flood control. The logic was sound. People were dying in flash floods. Nobody asked what the arroyos were doing before we called them a problem. They were recharging the aquifer, for free. I have called this the arroyo paradox: a system that was already solving the problem gets reclassified as the problem itself. Trust was that arroyo. It was doing the defending before anyone built a defense.
John Sterman put the general case plainly: “All too often the policies we implement to solve important problems fail, make the problem worse, or create new problems.”
The asymmetry is structural, and it favors the attacker in a way better strategy cannot fix. The harm an attacker does is indirect. Nobody can point to the moment their trust went. Nobody wakes up and says, that was the post that changed my relationship to the truth. It happens the way a bank erodes. Particle of dirt by particle of dirt. Flood by flood.
One day the ground that was there is not there.
The harm a defender does is direct and it has a name. When a system silences you by mistake, you know it. Your people know it. You can screenshot it. And now the attacker has a story. A real story, with a real person, really harmed by the thing that claimed to be protecting them.
I know the shape of that system, because I have designed its cousins.
In Michigan, between 2013 and 2015, the state let a computer decide who had committed unemployment fraud. No person reviewed a verdict before the state began collecting on it. The machine accused tens of thousands of people. It took their tax refunds. It garnished their wages. When the state went back through more than twenty-two thousand of those verdicts, it could not stand behind nine in ten. There was no villain at a console. There was a rule, and a scale, and nobody on the state’s side standing where the rule landed.
Every one of those people remembers the letter. None of them remember the thousand times a system like it quietly worked. Correct calls elsewhere do not repair the person this call harmed. An enforcement error gives the attacker the one thing erosion never does, a moment to point to.
And there is a layer under the layer.
Get Chris Perkins’s stories in your inbox
Join Medium for free to get updates from this writer.
The tools themselves can be turned. The models built to spot manipulation. The classifiers trained to catch coordinated behavior. The detectors that flag synthetic faces.
All of them can be corrupted by the same methods used on human minds. Poisoned training data. Inputs crafted to blind the detector to the one pattern it was built to find. A reward signal gamed until the defense works for the attack. You do not only need to trust the information environment.
You need to trust the instruments watching it, and they are standing on the same bank.
The economics of the signal
Everything I just described treats trust as a cultural value. Something to encourage, defend, restore with better messaging and stronger institutions. A soft thing. A feeling. The kind of word that appears in a mission statement and evaporates under load.
I will not tell you trust is a law of physics (I have, and then I wrote its obituary). A river has no adversary reading your blueprints. The bank does not study the levee to learn where it is weakest and flood there next spring. Disinformation does. The levee is an image for how the ground behaves, not a claim that trust falls like a stone.
But there is something harder than a feeling under all of this, and it is not hydrology. It is an accounting.
Trust runs on the cost of a signal, and a cost can be counted.
In fraud work that is the whole job. A transaction that costs the sender nothing tells you almost nothing about the sender. A transaction that cost something, that required the sender to have been somewhere, done something, risked something, it carries weight. The only question is what it would cost a liar to send the same signal. Biologists have been arguing this since 1975, when Amotz Zahavi proposed it. A gazelle that stots in front of a cheetah, jumping stiff-legged on all four legs at once, is not bluffing. It is telling the cheetah it has been seen, and that this one is not worth the chase.
The signal is honest because the signal is expensive.
Donella Meadows explained whole systems with a bathtub. A stock, a faucet, a drain. “If the inflow rate is higher than the outflow rate, the stock gradually rises.” Run it the other way and you have the ground under every institution I have worked for.
Here is the part the mission statements miss. The capacity to trust is not infinite. It is a reservoir, and every empty demand for your trust draws from it. Every chatbot that apologizes without meaning it, every deepfake you cannot catch, every “we take your privacy seriously” from a system built to do the opposite, every robocall, every consent box you click without reading, every political ad you have learned to look through. Each one is a trust encounter with zero trust in it. No single one matters but they do add up.
This is what happened to the ground. For the length of one childhood the attention economy flooded every channel with signals that cost nothing to send. The reservoir went down while the betrayals accumulated. The river does not care why the bank eroded. The bank is gone.
What a defense is allowed to cost
So the question underneath all of this is not whether we can defend. What is a defense allowed to cost the people it is for?
The honest answer is: some. A levee takes ground. Every real defense takes something from the thing it protects. A detection rule declines some honest cards. A takedown silences some honest voices. A warning system cries wolf. Anyone who tells you their defense costs nothing is describing a defense that does nothing.
The question is which cost you can stand behind.
I think the line runs through repair. A defense is allowed to do the harm it can undo. Quickly, by a person, before the harmed party has to prove anything. The card decline that one phone call lifts is a cost I can stand behind. Lifting it is where repair begins, not where it ends. The fraud verdict that takes a class action and nine years to settle is not a defense.
That is the one restriction I would keep. Not the softest defense. The one with someone standing where it lands, with the power to reverse it and the duty to explain. The harm you can justify is the harm you can repair. Everything past that line is the river’s work, done in the levee’s name.
Michigan had the rule and no one standing where it landed. It could only punish.
Donella Meadows said it plainly.
“If you want to understand the deepest malfunctions of systems, pay attention to the rules, and to who has power over them.”
Systems like Michigan’s are approved once, while they sit still. A review, a sign-off, a document in a file. Then they move, and no one is watching the motion. The alternative has a name, governance in motion: authority exerted at every step along the path, by someone who can be changed by what they see.
I wrote a book called Break the Chain. The chain in that book belongs to the attacker. An intrusion has to take its steps in order, and the defender only has to break one of them. It took me longer than it should have to see that my own systems were chains too. A rule fires, a score is computed, a verdict is issued, a letter goes out, a wage is garnished, every link automatic. The place to break that chain is the last link a person can still reverse. Put a person there.
I have built that version. A bound I had set made a state system stop before it marked a claim as fraud, and send it to a person instead. A pause, and a person on the other end of it.
I know this one from the inside, and not from work. There is a defense I run that nobody designed. It goes quiet when it is hit. It was built to keep something whole, and from the far bank it has only ever looked like distance. What it protects is what it spends. I stand on that bank too.
The geometry of the ground
You cannot defend trust with systems that spend it faster than they restore it. That is not a failure of strategy. It is the geometry of the ground.
The levee still stands on the riverbank. But look closer at what it is standing on. In New Orleans in 2005 two canal floodwalls failed before the water reached their tops. The ground gave way. It was weaker than the design assumed, and no one was reading it.
There are two ways a town keeps what it has. One is the historical society. A vault in the old bank building, a folding table, the second Tuesday of the month. It keeps by refusing motion. The other is the ditch. A schedule, a shovel, a man who can be found at any hour. It keeps by absorbing motion. A country needs both. Against disinformation, we have built only the first.
The question was never how to build a stronger levee. It is who walks it. Who notices the seep before the breach? Who is standing where the rule lands, with the authority to lift it and the duty to say why?
That is a different kind of work. It has a name in this valley that is centuries old, and it is next.
This essay joins Different, and Held, a collection in progress. Previous: The Terrain That Made You. Next: The Ninth Principle.
