Same Weights, Different Behavior

In 2024 a public demo made the idea undeniable: a model that would not stop talking about a bridge, with no change to its weights. The demo was live for a day. The question it left behind is still useful.

Two answers, one model

Take an open-weight model. Give it the same prompt twice. In one run it answers as expected. In the other, it behaves as if it had been given an instruction that never appeared in the prompt.

Nothing is retrained. No dataset is collected. No GPU-hours are spent on a fine-tune. The intervention happens while the model is generating.

That is the distinction I care about. Fine-tuning changes the model. Inference-time control changes the conditions under which the model runs.

Why it matters

For a production AI team, the difference is practical:

Cost. No training run, no labelled dataset, no new checkpoint to host.
Latency. The change is part of the forward pass, not a second model in the request path.
Reversibility. Turn the intervention off and the original behavior returns.
Per-tenant behavior. The same weights can serve different brands, tones, or refusal styles with different runtime settings.
Safety experiments. You can test a behavior before committing to a fine-tune, and you can measure what the intervention costs in fluency.

None of this replaces a good prompt. It gives a team another control surface when a prompt is not enough.

What I am testing

I am reproducing the effect on an open-weight model with Hugging Face Transformers. The setup is deliberately small: one model, a fixed prompt set, a baseline, and a measured intervention. I am keeping the method private until the numbers are in. That is not mystique. It is the difference between a demo and a result.

The question is not "can it be done". Public work has shown that it can. The question is whether it can be done reliably enough to be useful, and where it stops being worth the complexity.

What I am not claiming

I have not finished the reproduction. I will publish the transcripts and the evaluation when I have them.

Two cautions are already clear from the public literature:

prompting is still a strong baseline, and usually the right first tool;
this family of techniques is more fragile than the demos suggest: the useful range is narrow, and the effect can cost fluency.

If the result is weak, I will say so. A negative result is still a result, and it is more useful to a hiring team than a confident story with no numbers behind it.

What comes next

Three artifacts: a baseline, an intervened run, and an evaluation that compares them on the same prompt set. Then a decision: does this beat a system prompt? For which behaviors? At what cost?

That is the shape of an answer a production team can use. Not "look, it works", but "here is when it works, and here is what it costs".

Sources

Anthropic, Golden Gate Claude (2024)
Neuronpedia, for exploring and comparing model features
AxBench (Wu et al., 2025), a comparison of steering techniques
Gemma Scope, open sparse autoencoders for the Gemma model family