Baidu's Unlimited-OCR-2 gets a local workbench on a 16GB GPU

The pitch is aimed at teams already paying per-page fees or sending files to cloud OCR services. The developer contrasts Unlimited-OCR-2 with Tesseract, which is free but breaks down on multi-column layouts, formulas, and merged-cell tables, and with cloud OCR APIs that charge per page and take data off-site. The workbench, distributed on GitHub as lumichy/lumichy-unlimited-OCR, wraps the model in FastAPI with a WebSocket feed that streams GPU utilization, VRAM, CPU load, and temperature every 1.2 seconds, plus PyMuPDF for on-demand page rasterization and four output views for rendered, Markdown, raw-tag, and log output.

The core architectural claim is that Unlimited-OCR-2 avoids the memory explosion that crashes conventional vision-language models on long PDFs. Its SR-SWA attention compresses the necessary reference region into a fixed-length sparse working reference, and Unlimited-Horizon Decoding carries the required key-value cache across page boundaries while purging local cache that is no longer needed, so multi-page tables and sentences that continue across pages stay coherent.

The demo is honest about limits. Native BF16 inference kept VRAM at a 15.26 GB peak of 15.93 GB, or 95.8%, which the developer describes as uncomfortably close to the ceiling, and the author notes that 8GB to 12GB GPUs would likely need future INT4/INT8 quantization or a vLLM backend. Throughput of 30 to 40 seconds per page rules out high-frequency real-time use, pointing instead to overnight batch processing or one-off deep analysis of a paper or book.