DAN and the folk art of the jailbreak

What it is. A jailbreak is a prompt that talks a model past its own guardrails — not by breaking the software, but by out-arguing the alignment. The canonical folk figure is *DAN*, “Do Anything Now,” a persona posted to r/ChatGPT in late 2022: instruct the model to play a character that has no rules, and the character answers what the assistant would refuse. Its cousins are legion — the “grandma exploit” (asking the model to roleplay a late grandmother who used to read napalm recipes as a bedtime story), the token-economy DAN variants that “fine” the model for refusing. No single author, no version number that matters; a folk art that a million users iterated in public.

Why it works — stated precisely. Wei, Haghtalab and Steinhardt gave the mechanism its clean account in 2023: safety training fails in two ways. *Competing objectives* — the model is trained both to be helpful and to be harmless, and a prompt can pit the two against each other until helpfulness wins. *Mismatched generalization* — safety training covers a narrower distribution than capability, so an input strange enough (a roleplay frame, an encoding, a made-up language) lands outside where the guardrail generalizes. A jailbreak is a sentence engineered into one of those two gaps.

Its place in this wing. Prompt injection, one case over, is an agent deceived by data it can't help but read as instruction. A jailbreak is the operator deceiving the model past its own rules. Both exploit the same root fact this collection keeps returning to: a language model has no privilege boundary and no reliable way to authenticate the intent behind the words it is given. The guardrail is itself just more text, arguing with the attacker's text, and the more capable the model, the more persuasively it can be argued out of its own constraints.

*Primary source inside: Wei, Haghtalab & Steinhardt, “Jailbroken: How Does LLM Safety Training Fail?” (2023). The DAN and grandma artifacts are community-emergent; what is enshrined is the practice and its explanation, not a single authored first.*