AIToday
Large Language ModelsOpen-Source AIZenn AI/MLPublished: Nov 7, 2026, 19:01 JST

LFM2.5-24B-A2B (2.68bit) matches Qwen3.8-27B on monocoder test

Send on LINE
LFM2.5-24B-A2B (2.68bit) matches Qwen3.8-27B on monocoder test

3 Key Points

  1. What happened

    In the author's fixed-task tests, LFM2.5-24B-A2B at roughly 2.68-bit quantization averaged 16.1s on four single-shot tasks, versus 33.4s for Bonsai 2 27B and 85s for Qwen3.8-27B, and passed the FizzBuzz check while both 27B models logged 4 errors.

  2. Why it matters

    The result suggests LFM2.5 keeps pace with larger 27B-class models on these specific tasks, but the smaller effective parameter count is a possible explanation the author did not verify.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

The test is part of a running series that measures locally runnable models under a fixed set of prompts, with the same Mac mini, the same llama-server engine, and the same 32768-token context. LFM2.5-24B-A2B is a mixture-of-experts model from Liquid AI with 24B total parameters but only about 2.45B active per token, per the official model card, and the author chose a roughly 2.68-bit quantization to fit it onto 16GB of memory.

A key complication is that LFM2.5 always thinks before answering and cannot be told to stop; a client request to disable thinking had no effect. That made the first monocoder run produce no usable answer within the response cap, which the author treated as a harness problem rather than a model failure, adding a guard to exclude empty capped responses and rerunning LFM2.5 alone with a 16384-token cap. The author also notes that quantization effects could not be separated, since no higher-bit version of the same model was tested as a control, and that official speed or performance figures were not checked in this environment.

FAQ
What quantization did the author use for LFM2.5-24B-A2B?
The author used unsloth's UD-Q2_K_XL, about 2.68 bits per weight and 8.04GB, to fit the model on a 16GB Mac.
Why did the first monocoder measurement fail?
On the initial run, LFM2.5 spent all 4096 tokens on thinking and produced a 0-byte implementation file, so the author added a guard and raised LFM2.5's cap to 16384 tokens for a rerun.
Does the author declare a winner between LFM2.5 and the 27B-class models?
No. The author says conditions were not aligned, because LFM2.5 alone ran with thinking enabled and a different response cap, and the number of runs was small.
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleLumentum Holdings: US$3.7b segment, US$46.2b market value