A developer has built an interactive, mobile-friendly visualization of GPT-2-small's 32,070 token embeddings, allowing anyone to explore how the model organizes language by tapping tokens to see their nearest semantic neighbors. The layout uses t-SNE compression and minimum spanning tree edges to show real nearest-kin relationships in the model's word space.
Summaries like this, in your inbox every morning.
Sign up free →What happened
A developer created an interactive map of GPT-2-small's 32,070 alphabetic tokens, laid out using t-SNE (a visualization technique) over a compressed representation of the model's embedding table. The map is mobile-friendly and uses a minimum spanning tree to show nearest-neighbor relationships between tokens.
Why it matters
This tool makes the internal structure of a large language model visible and explorable in a way that is usually opaque. For anyone curious about how AI text models organize vocabulary — researchers, engineers, or the simply interested — it offers direct access to see which words the model considers semantically close to each other.
What to watch
The map is live and interactive now. Users can pinch to zoom, tap any token to see its nearest neighbors, tap a neighbor to walk the graph, and search the box to jump to a specific token.
The interactive map displays the token embeddings from GPT-2-small, one of OpenAI's smaller transformer-based language models, in a way that reveals how the model internally represents and relates words to one another. The visualization includes 32,070 alphabetic tokens and was designed to be accessible on mobile devices, which is notable given that many machine-learning tools are desktop-focused.
The layout method combines two techniques: t-SNE (t-distributed stochastic neighbor embedding), a dimensionality reduction method that preserves local structure when compressing high-dimensional data into two dimensions, and a minimum spanning tree that connects each token to its nearest neighbors. Because every edge in the visualization represents a genuine nearest-kin relationship in the compressed space, the graph is not a simplified schematic but a faithful representation of the model's semantic organization.
Usability is central to the design. Users can zoom by pinching, select any token by tapping to reveal its immediate neighbors, move through the graph by tapping neighbors in sequence, and search for specific tokens using a search box. This combination of browse and search modes makes it possible both to explore the space serendipitously and to jump directly to tokens of interest. The tool was submitted to the Machine Learning subreddit by /u/Limp-Contest-7309 and is available to use immediately without requiring a forward pass or model loading on the user's device.
This visualization addresses a fundamental opacity in large language models: the semantic organization of their vocabulary. GPT-2, despite being a relatively older model, remains one of the most studied in academic and hobbyist circles, making it a useful subject for exploration tools. The use of t-SNE to compress the embedding space and a minimum spanning tree to represent relationships means the edges shown are not arbitrary—each line represents a real nearest-neighbor pairing in the model's learned space. This allows researchers and curious users alike to inspect whether the model's internal organization of language matches human intuition about word similarity.
AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.
Free · takes 30 seconds · unsubscribe anytime
No comments yet. Be the first to share your thoughts!
Log in to join the discussion
Get curated AI news from 200+ sources delivered daily to your inbox. Free to use.
Get Started FreeFree · takes 30 seconds · unsubscribe anytime
1 minute a day. The AI essentials.
200+ sources · Email / LINE / Slack
