Hugging Face brings GGUF to transformers

Hugging Face's move brings two previously separate tools closer together. Its transformers library has long been the default place to define and run models, while llama.cpp has become the engine behind local AI tools such as Ollama, LM Studio, and Jan. GGUF, developed by the llama.cpp team, is a widely used format for local inference, and publishers including Unsloth, LM Studio Community, and bartowski provide ready-to-use GGUF checkpoints. Until now, working with those checkpoints meant leaving the transformers workflow.

This integration reuses llama.cpp's underlying ggml kernels through the kernels library and reduces overhead in generate, with an initial focus on Apple Silicon and the Qwen3.5 architecture. The same GGUF file remains usable in llama.cpp, which Hugging Face still recommends when efficient local inference is the priority. The benchmarks compare three checkpoints on a MacBook Pro M2 Max, and the article notes the conditions are not identical because the transformers measurement includes prefill while llama-bench reports decode-only throughput.

The practical stakes depend on the reader's setup. The packed path is MPS-only for now, padding and batching still need work, and architecture coverage is limited to Qwen3.5 dense and MoE checkpoints. For developers on supported Apple Silicon machines, the gain looks like convenience: experimenting, evaluating, and validating GGUF checkpoints without leaving their usual environment, provided the expected kernel loads rather than falling back to dequantization and higher memory use.