Subtext is an open-source tool that visualizes what a language model is thinking during a live conversation by decoding its internal activations at nine layer depths and rendering them in real time. The visualization reveals reasoning steps that occur before or during text generation—verdicts forming while the model reads input, planned words held while other tokens are output—offering researchers and developers direct observation of a model's intermediate reasoning on consumer hardware.
Summaries like this, in your inbox every morning.
Sign up free →What happened
A new tool called Subtext applies Anthropic's Jacobian lens method to visualize a language model's internal reasoning during live conversation. It renders the model's internal state—which vocabulary words it is disposed to produce at each moment—continuously at nine layers of depth as the user types and the model responds, making the model's intermediate steps directly observable on consumer hardware.
Why it matters
The tool reveals a gap between what the model thinks internally and what it outputs. It shows that verdicts form before they are verbalized, that planned concepts hold active while unrelated tokens are being emitted, and that multi-step reasoning surfaces intermediate steps (e.g., the country Italy and currency euros on a question about Italy before generation begins). For researchers and developers, this offers a window into model reasoning that text output alone does not provide.
What to watch
Subtext runs on a single HTML file with Qwen3.5-4B (~10 GB VRAM, NVIDIA GPU required) and is open-source under Apache 2.0. Sessions can be exported as JSON and replayed in any browser without a GPU. The method's limitations include that it reads only single-token concepts and captures an approximation of the workspace, not the entirety of the model's internal state.
Subtext builds on recent research from Anthropic into a concept called the J-space—a small set of internal representations that behave like a global workspace, whose contents the model can report, deliberately modulate, and use for reasoning. The Jacobian lens is the technical tool Anthropic developed to read this workspace: it transports a residual-stream activation from any layer into the final-layer basis and decodes it through the model's unembedding, effectively asking which vocabulary words the model's internal state is disposed to produce at that moment.
The practical innovation here is packaging this method into a conversational, real-time visualization system that runs on consumer hardware. By reading the lens at every token position—both during the prefill phase (when the model reads the user's message) and during token-by-token generation—Subtext makes the model's intermediate steps directly watchable. The three concrete examples in the article (verdicts forming before output, judgments internally settled before verbalization, and planned caveat concepts held while unrelated tokens are being emitted) reproduce phenomena that were previously observable only on Claude-scale models running on research infrastructure. The ability to export sessions as JSON and replay them without a GPU further lowers the barrier to inspection and analysis.
The limitations are important: the lens reads only concepts that correspond to single vocabulary tokens, so multi-token concepts are invisible or fragmentary. It approximately captures the workspace identified in the paper rather than the entirety of the model's internal state, and layers below the fitted range are not observed. The authors emphasize that workspace readouts demonstrate functional availability of information for report and reasoning; they do not demonstrate subjective experience. For interpretability research and for developers seeking to understand model behavior beyond text output, Subtext offers a new empirical vantage point, though with clear epistemic boundaries.
No comments yet. Be the first to share your thoughts!
Log in to join the discussion
Get curated AI news from 200+ sources delivered daily to your inbox. Free to use.
Get Started FreeFree · takes 30 seconds · unsubscribe anytime
1 minute a day. The AI essentials.
200+ sources · Email / LINE / Slack
