# Neuralese: what happens before the answer

Neuralese is a useful name for learned numerical representations that carry information inside or between neural networks. It is not a discovered language with words and grammar.

## One term, two technical uses

In 2017, Jacob Andreas, Anca Dragan, and Dan Klein used "Neuralese" for real-valued messages learned by agents that coordinate through a differentiable communication channel. Their translation method treats a vector and a sentence as equivalent when they induce the same belief about the world in a listener. [Translating Neuralese](https://aclanthology.org/P17-1022/)

Recent writing also uses the term for intermediate computation that stays in a model's continuous hidden states. In research papers, "latent reasoning" or "continuous chain of thought" is usually more precise. [Reasoning Beyond Language](https://arxiv.org/abs/2505.16782)

Calling it a language is a metaphor, not a measurement.

## How continuous latent reasoning works

1. The model converts input tokens into vectors.
2. Standard chain of thought decodes intermediate steps into tokens. Latent methods keep one or more steps as hidden states.
3. Coconut uses the last hidden state as a "continuous thought" and feeds it back as the next input embedding.
4. The model returns to language mode and produces a readable answer.

The Coconut authors write: "We feed it back to the LLM as the subsequent input embedding directly in the continuous space." [Training Large Language Models to Reason in a Continuous Latent Space](https://arxiv.org/abs/2412.06769)

## What the evidence says

The 2017 translation method preserved the semantics and pragmatics of agent messages well enough that agents using a translation layer suffered little reward loss.

Coconut outperformed written chain of thought on selected logical planning tasks that require backtracking and used fewer thinking tokens. The authors describe hidden states that retain several possible next steps, which resembles breadth-first search.

The case is not settled. Later experiments found Coconut sensitive to training choices. Reinforcement-learned latent reasoning still trailed language-space chain of thought on mathematical reasoning. [Reinforcement Learning for Latent-Space Thinking in LLMs](https://arxiv.org/abs/2512.11816)

There is no public evidence that hidden states have a fixed dictionary, that every model uses the same code, or that current frontier systems run a deep private monologue.

## Interpretability and safety

Readable reasoning can help. OpenAI found that monitoring written chains of thought can reveal reward hacking that is harder to catch from actions and final answers alone. [Detecting misbehavior in frontier reasoning models](https://openai.com/index/chain-of-thought-monitoring/)

Readable does not mean faithful. Anthropic found that written reasoning can omit factors that changed a model's answer. [Measuring Faithfulness in Chain-of-Thought Reasoning](https://www.anthropic.com/research/measuring-faithfulness-in-chain-of-thought-reasoning)

If useful intermediate work never becomes language, text-based monitors lose access to it. Researchers would need activation probes, causal interventions, behavioral evaluations, or other controls. None currently yields a complete account. [Evaluating chain-of-thought monitorability](https://openai.com/index/evaluating-chain-of-thought-monitorability/)

Neuralese may remove a token bottleneck. It may also remove the part of reasoning we know how to inspect.

## FAQ

### What is Neuralese?
In AI research, Neuralese originally meant learned vector messages exchanged by artificial agents. The term is now also used, less formally, for reasoning that remains in a model's continuous hidden states instead of being decoded into words.

### Is Neuralese a real language?
Not in the ordinary linguistic sense. It has no shared vocabulary, stable grammar, or model-independent dictionary.

### Do large language models think in Neuralese?
All neural language models compute with hidden numerical states. Calling those states thought or language is an interpretation. Some research systems explicitly reuse hidden states for several reasoning steps, but that does not prove that deployed models use a coherent secret language.

### What is continuous chain of thought?
It keeps intermediate reasoning in latent vectors instead of decoding every step into a token.

### How does Coconut work?
Coconut alternates between language mode and latent mode. In latent mode it feeds the model's last hidden state back as the next input embedding.

### Is latent reasoning better than written chain of thought?
Sometimes, on selected tasks. There is no general result that makes latent reasoning a replacement for written chain of thought.

### Can Neuralese be translated into English?
The 2017 work translated agent messages by matching their effect on a listener's beliefs, not by assigning a word to every vector. Hidden-state interpretation remains open.

### Why does Neuralese matter for AI safety?
Reasoning that stays latent removes a signal used by chain-of-thought monitors. Written chains of thought are already imperfect, so neither form is a complete audit trail.

## Sources

- [Translating Neuralese](https://aclanthology.org/P17-1022/), Andreas, Dragan, and Klein, 2017.
- [Training Large Language Models to Reason in a Continuous Latent Space](https://arxiv.org/abs/2412.06769), Hao et al., 2024.
- [Reasoning Beyond Language](https://arxiv.org/abs/2505.16782), Chen et al., 2025.
- [Reinforcement Learning for Latent-Space Thinking in LLMs](https://arxiv.org/abs/2512.11816), Özeren and Aßenmacher, 2025.
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://www.anthropic.com/research/measuring-faithfulness-in-chain-of-thought-reasoning), Anthropic, 2023.
- [Detecting misbehavior in frontier reasoning models](https://openai.com/index/chain-of-thought-monitoring/), OpenAI, 2025.
- [Evaluating chain-of-thought monitorability](https://openai.com/index/evaluating-chain-of-thought-monitorability/), OpenAI, 2025.
- [AI 2027, Appendix E](https://ai-2027.com/scenario.pdf), Kokotajlo et al., 2025. This is a forecast, not an empirical paper.

Last reviewed: September 2, 2026.

Published by [Meiji Industries](https://meiji.industries/). For inquiries about neuralese.dev, email [adam@meiji.industries](mailto:adam@meiji.industries?subject=neuralese.dev%20inquiry).