A new compression technique called ExTernD lets AI models use ternary quantization (reducing weights to just three values) without sacrificing accuracy. By decomposing weights into two ternary matrices with a variable-rank scaling layer between them, the method sidesteps the accuracy limits that fixed ternary compression previously hit. The approach uses only slightly more memory than standard quantization, making it potentially practical for cutting inference costs while maintaining model quality.
Summaries like this, in your inbox every morning.
Sign up free →What happened
A researcher posted a paper on ExTernD, a new method for ternary post-training quantization (PTQ)—a technique that shrinks AI models by limiting weights to three values. Instead of compressing to a single ternary matrix, the method decomposes weights into two ternary matrices plus a scaling matrix in between, allowing the inner rank to grow as needed.
Why it matters
Previous ternary quantization hit accuracy limits because matrix size was fixed. By adding a variable inner rank, this approach can achieve accuracy arbitrarily close to any target level—meaning models can be heavily compressed without the accuracy floor that previously stopped ternary methods. For businesses and researchers running large AI models, this could lower inference costs significantly.
What to watch
The method uses only slightly more memory than current quantization approaches, making the trade-off practical for systems already running compressed models. The key advantage is that ternary math—operations on three-value weights—is computationally cheaper than full-precision math, so the modest memory overhead may pay for itself in speed gains.
The post describes a new approach to post-training quantization (PTQ)—a method applied after a model has finished training to compress it for deployment. The author, posting as /u/LMTLS5, argues that earlier attempts to apply ternary quantization (limiting weights to three discrete values) directly to a single matrix hit a fundamental wall: with a fixed matrix size, you cannot recover accuracy beyond a certain threshold, no matter how you tune the parameters.
To break this impasse, the researcher tested decomposing each weight matrix into two ternary matrices with an inner diagonal scaling matrix connecting them. The scaling matrix allows the inner rank to grow arbitrarily large, which in turn allows model accuracy to approach any desired level. In other words, instead of a hard accuracy ceiling, the method trades off inner rank (and thus a small amount of extra memory) for precision recovery.
The empirical finding is that this extra memory overhead is modest—only slightly more than what current quantization methods consume—making the trade-off practical. Since ternary arithmetic (operations on three-value weights) is significantly faster and cheaper than full-precision operations, the slight additional memory cost can be justified by the speed gains during inference. This combination of tunable accuracy, low memory overhead, and fast ternary math suggests the approach could be useful for reducing deployment costs of large language models in production environments.
Quantization—the practice of reducing the precision of AI model weights—has long been a cornerstone of making large language models practical to run. Ternary quantization is an extreme form: it restricts all weights to just three discrete values, which dramatically cuts memory footprint and accelerates inference. However, this aggressive compression typically incurs a sharp accuracy penalty, and past research showed that simply using fixed ternary matrices could not recover that loss beyond a certain point.
The contribution outlined in the ExTernD paper is a structural insight: that decomposing a weight matrix into two ternary matrices connected by a variable-rank diagonal scaling matrix transforms the problem. With a fixed single ternary matrix, the rank is bounded and immutable; with two ternary layers and an adjustable inner rank, the effective capacity of the model can grow smoothly. This shift from a hard ceiling to a soft trade-off (inner rank versus memory use) means that practitioners can tune the compression level to their hardware budget without hitting a brick wall in accuracy.
AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.
Free · takes 30 seconds · unsubscribe anytime
No comments yet. Be the first to share your thoughts!
Log in to join the discussion
Get curated AI news from 200+ sources delivered daily to your inbox. Free to use.
Get Started FreeFree · takes 30 seconds · unsubscribe anytime
1 minute a day. The AI essentials.
200+ sources · Email / LINE / Slack
