The Implications of Linguistic Illegibility for LLM Security

Researcher James Mickens argues that Large Language Models' internal computations cannot be reliably understood through their linguistic outputs or mechanistic probing, a phenomenon he terms "linguistic illegibility," which undermines security approaches that rely on monitoring a model's self-reported reasoning. He proposes that effective LLM security requires non-linguistic isolation techniques like taint tracking and robust virtualization that function independently of what the model claims to be doing, rather than relying solely on chain-of-thought monitoring or constitutional self-critique.