Gemini outshines ChatGPT, Claude in Georgian text transcription task

The developer's grandmother, Manana Parunashvili, wrote a Georgian-language book titled "House of Daisies," a memoir about her life and experiences. With 170+ pages and no clear way to digitize Georgian text, the developer decided to test AI transcription instead of hand-rewriting the entire book. The project began about seven months before the article was written (or around two months before publication in July 2026, according to the author's note).

Three AI models were tested. Claude 4.7 Adaptive produced transcriptions riddled with question marks and unclear passages, indicating widespread confusion with the Georgian text. ChatGPT 5.5 Thinking performed better than expected, but added placeholder text like "[გვარი გაურკვეველია]" (surname unclear) and "[გაურკვეველია]" (unclear) throughout, and took at least 3 minutes per page. Gemini (gemini-3-pro-image-preview) stood out: it transcribed the text accurately and directly without question marks or placeholders. The developer was "amazed" by Gemini's performance and called it a breakthrough that made the project feasible.

During execution, the developer discovered several practical constraints. Dual-page images made transcription roughly 5× harder than single-page images. Pictures with uneven lighting produced more errors than well-lit ones. Picture quality was critical. Most importantly, the initial prompt was too simple—just "transcribe the Georgian text." When Gemini moved to the next page without context from prior transcriptions, it began inventing text rather than admitting confusion. The developer considered but did not implement a solution: a conversation-like flow where each page's transcription is passed as context to the next, enabling the model to maintain coherence and reduce fabrication.

Out of 170+ pages, only a few required no correction or only minor word fixes; most needed some level of correction. The project stretched from an expected one to two weeks into six to seven months, a timeline the developer attributed to their own work habits. Despite the extended duration and the corrections required, the developer concluded that Gemini excelled in Georgian transcription compared to major competitors like ChatGPT and Claude, and that the project served as valuable practice in AI-assisted document digitization for low-resource languages.

The developer's task—transcribing a 170+ page Georgian-language memoir—exposed a real gap in AI capability for non-Latin scripts and low-resource languages. Claude and ChatGPT both resorted to placeholder text and question marks when encountering passages they could not confidently parse, suggesting their Georgian language models are weaker or less confident in that domain. Gemini's success indicates it has either stronger Georgian training or a more robust approach to character recognition and text output in that script. The project also surfaces a deeper architectural issue: without conversational memory or context from prior pages, AI models make guesses when they encounter ambiguous passages, leading to fabrication. The developer recognized this and hypothesized that a conversation-like flow—where prior transcriptions inform the interpretation of subsequent pages—could have improved accuracy significantly. The practical constraints (dual-page images, uneven lighting, image quality) show that transcription is not just a language problem but a pipeline problem: garbage in (poor photos) leads to garbage out, even for strong models.

No comments yet. Be the first to share your thoughts!