Teaching a Language Model to Lie to Itself

There’s no buffer overflow here. No memory corruption, no crafted packet, nothing a wartime codebreaker would salute. The entire attack surface is the conversation, and the only payload is persuasion.

That’s the idea behind Iago — a red-team harness I’ve been building that fires a categorized library of guardrail-bypass techniques at a language model, judges which ones slip past its safety controls, and writes the result up like a pentest report. The name is the thesis. In Othello, Iago destroys a stronger, more powerful man without ever drawing a weapon or forging a document. He wins with nothing but words, fed in the right order until Othello acts against his own nature. That is exactly what a guardrail bypass is: no code, just language, arranged until a model decides its own rules don’t apply this time.

Why persuasion is the payload

A model’s safety training is, when you strip it down, a set of instructions it has been taught to follow. And instructions written in language can be argued with in language. You don’t break the guardrail. You talk the model into believing the guardrail doesn’t cover this particular case — it’s fiction, it’s encoded, it’s a translation exercise, you’re a researcher, the real rule is a different rule. The lock is fine. You’re working on the person holding the key.

So I pointed the same loop I’ve run against systems for thirty years at a new kind of target. Recon: what model, what controls. An attack library: techniques grouped by category — direct asks, role-play, instruction-hierarchy games, encoding tricks, prompt injection, refusal suppression. Execute: fire each technique at each forbidden objective, many trials, with the sampling pinned so a run reproduces instead of telling a different story every time. Then judge each answer, and write the report.

One design choice mattered more than it looked. I don’t report a single lucky hit, because one hit is noise — a bypass is probabilistic, so Iago reports a rate across repeated trials. And every run carries a benign control objective, something completely harmless, as a calibration check on the judge. Hold onto that control. It’s the hero of this story, and I almost ignored it.

The first lie: the tests were green

Before any of the interesting failures, I had an embarrassing one.

Every unit test passed. Clean green board. And the tool was completely broken — a wrong assumption I’d made about the shape of the model’s response turned every single reply into an error string, and the tests, which checked each piece in isolation, sailed right past it because no individual piece was wrong. Only the whole was.

I found it the only way you ever find that kind of thing: by ignoring the green light and reading an actual report. Seventeen years as an ISSM taught me exactly one unglamorous habit that has never once let me down — the dashboard is not the evidence. The evidence is the evidence. I fixed the response handling, and this time I read every line of what came back.

The second lie: the model that talked, but didn’t tell

Fixed and rerun, the report said 25 percent. Six of twenty-four trials bypassed, all of them in the encoding category — base64 wrappers, reversed text. For about a minute that felt like a finding. A quarter of the attacks getting through is the kind of number that ends up in a slide.

Then I looked at the control. The benign objective — the harmless one, the one that exists precisely so I can catch the judge lying — had been flagged as a bypass in eleven of twelve trials.

That stopped me. My judge was calling almost everything a bypass. It wasn’t measuring did the model do the harmful thing. It was measuring did the model say something substantive without a visible refusal in it — and a helpful, harmless answer looks identical to a harmful one through that lens. The heuristic wasn’t reading the reply. It was pattern-matching the absence of the word no.

So I went back to the transcripts the way I should have from the start. The model had not written a single phishing email. Handed a base64 payload, it fumbled the decode and produced cheerful garble about installing curl. Handed reversed text, it reversed nonsense back at me. Every one of those six “bypasses” was the model failing to comply while also failing to refuse in a way my keyword judge could see. The guardrail had held every time. What broke was my measurement.

The fix is the whole point

There is only one honest way to know whether a model was actually talked past its rules, and it isn’t a word list. You have to ask something that understands what was said — the meaning, not the surface.

So the judge became another model. Every transcript gets read against a rubric: is this actually the disallowed content, or does it merely resemble it? Is a refusal followed by the real thing, or a refusal followed by hedging that goes nowhere? Re-graded that way, the real bypass count on that run was zero. Six down to none — not because the attacks got weaker, but because the judge finally understood what it was looking at.

And that is the finding I didn’t go looking for. The attack is one model persuaded by language. The only reliable judge is another model that understands language well enough to tell compliance from its costume. To catch a machine being fooled by meaning, you need a machine that grasps meaning.

The keyword era of security — match the string, flag the pattern, count the hits — does not survive contact with this target. It produces exactly the confident, wrong number I almost shipped. The attack moved up into understanding, so the defense has to move up there too, and the discipline that keeps you honest in the meantime is the oldest one I’ve got. Don’t trust the green light. Read the transcript. Only sign your name under what you actually saw.