How does the tool check if a model was trained from scratch?

It uses three axes: comparing architecture parameters (hidden_size, intermediate_size, num_hidden_layers, attention heads) in config.json; measuring tokenizer vocabulary overlap between two models; and computing linear CKA (Centered Kernel Alignment) on embedding weights to account for rotational invariance. A from-scratch model scores near-zero on the weights axis, while a derived model shows inherited embeddings.

What is the main limitation of the weights fingerprint?

Linear CKA detects from-scratch training reliably (near-zero score) but is a weak detector of derivation. Continued-pretraining reshapes embeddings enough that a known derivative scores only ~0.25 CKA, barely above the ~0.21 baseline between unrelated models of the same family. For that reason, architecture and tokenizer fingerprints remain the primary evidence.

Why doesn't row-wise cosine similarity work for comparing embeddings?

Because of rotational invariance: a Transformer's hidden space has no privileged basis, so two models can encode identical information under an arbitrary orthogonal rotation. Row-wise cosine sees rotation as dissimilarity, making it useless for detecting lineage.

Large Language ModelsOpen-Source AIAI Safety & AlignmentHacker NewsAug 9, 2026Aug 9, 2026, 16:01 JST3 min read

Model Genome tool fingerprints LLM origins—from-scratch vs. derived

Send on LINE

Key takeaway

Researchers released a public tool that fingerprints language models on three axes—architecture, tokenizer vocabulary, and weights—to determine whether they were trained from scratch or derived from existing open-weight bases.
The method reliably flags from-scratch models but is weaker at detecting continued-pretraining, so architecture and tokenizer matches remain the primary evidence.
They applied it to nine Korean organizations' models and found a mix of native, adapted, mixed, and ported lineages.

3 Key Points

What happened
Researchers built a reproducible three-axis pipeline to verify whether a language model was trained from scratch or derived from an open-weight base (Qwen, Llama, DeepSeek, Mistral). The method inspects architecture (config.json), tokenizer vocabulary overlap, and embedding weights (linear CKA). They applied it to nine Korean organizations' public foundation models and found heterogeneous results: some matched foreign architectures exactly (Ported), others used entirely self-built designs (Native), and many fell in between (Adapted or Mixed).
Why it matters
When labs announced "self-developed" models in late July 2026 (e.g., LG K-EXAONE 2.0, 750B), Chinese tech communities debated the distinction—a Zhihu thread reached 2.7M views. The question is answerable objectively from public files. Building on open-weight bases is legitimate industry practice, but vendors do not always distinguish it from training from scratch. This tool lets outsiders verify claims without access to training logs.
What to watch
The weights axis reliably detects from-scratch models (near-zero CKA score) but weakly detects continued-pretraining derivatives (~0.25 CKA, barely above unrelated baseline ~0.21). Architecture and tokenizer fingerprints remain primary evidence. The live demo and dataset (Model Genome Korea) lets anyone inspect any Hugging Face model with the same yardstick, and corrections are welcome.

Ask the AI about this article →

Summaries like this, in your inbox every morning.

Get news like this every morning — free

Context & Analysis

The Model Genome pipeline addresses a real gap exposed by late July 2026 releases. When several Korean organizations announced DeepSeek-rivaling "self-developed" models, the Chinese tech community—particularly Zhihu—erupted in debate (2.7M views), signaling genuine public interest in verifying vendor claims. The tool's insight is that lineage leaves detectable traces across three independent surfaces: the declared architecture, the learned vocabulary, and the numerical embedding space. Each axis is noisy on its own (architecture alone can be copied, tokenizers can be retrained independently, weights shift under continued training), but together they form a fingerprint.

The researchers hit two instructive obstacles that shaped their method. First, naive row-wise cosine similarity fails because transformers store information in a rotationally-invariant way—two models can be identical but differ only by an arbitrary rotation. Second, even CKA (which is rotation-invariant) struggles to separate continued-pretraining from from-scratch because large-scale training so thoroughly reshapes embeddings that a derivative and a true independent model become indistinguishable by this metric alone. The honest conclusion is that the weights axis confirms from-scratch (near-zero CKA) but is a weak detector of derivation, pushing config and tokenizer evidence into the primary role. Applying the same pipeline to nine Korean organizations' public models revealed heterogeneity: some exactly matched foreign architectures (Ported), others built their own (Native), most occupied a middle ground (Adapted or Mixed).

FAQ

How does the tool check if a model was trained from scratch?: It uses three axes: comparing architecture parameters (hidden_size, intermediate_size, num_hidden_layers, attention heads) in config.json; measuring tokenizer vocabulary overlap between two models; and computing linear CKA (Centered Kernel Alignment) on embedding weights to account for rotational invariance. A from-scratch model scores near-zero on the weights axis, while a derived model shows inherited embeddings.
What is the main limitation of the weights fingerprint?: Linear CKA detects from-scratch training reliably (near-zero score) but is a weak detector of derivation. Continued-pretraining reshapes embeddings enough that a known derivative scores only ~0.25 CKA, barely above the ~0.21 baseline between unrelated models of the same family. For that reason, architecture and tokenizer fingerprints remain the primary evidence.
Why doesn't row-wise cosine similarity work for comparing embeddings?: Because of rotational invariance: a Transformer's hidden space has no privileged basis, so two models can encode identical information under an arbitrary orthogonal rotation. Row-wise cosine sees rotation as dissimilarity, making it useless for detecting lineage.

Hacker NewsRead Original Article

Send on LINE

Get the latest Large Language Models news every morning

For example, today's edition would include:

DataAgent launches with $10M to auto-fix Kubernetes faultsSiliconANGLE AI · 43m ago
SK Hynix custom HBM boosts inference up to 5.15xDIGITIMES Asia · 43m ago
Nvidia Earnings: Boring by Design, Avoiding a Consolidated WorldStratechery (Ben Thompson) · 43m ago

AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.

Subscribe free →

Free · takes 30 seconds · unsubscribe anytimeWhat is AIToday? →