French startup Kog claims 30x faster AI inference on standard GPUs

Kog, founded by Gaël Delalleau, is tackling one of AI infrastructure's most pressing challenges: the speed of inference—the computational step where an AI model produces an answer to a user prompt. While competitors like Cerebras have pursued custom silicon to accelerate inference, Kog is betting that conventional GPUs already in enterprise datacenters contain untapped performance waiting to be unlocked through software alone. In May, the startup published a tech preview on Hacker News demonstrating extremely fast single-request decoding on standard datacenter GPUs, specifically the AMD MI300X and Nvidia H200, achieving 3,000 tokens per second on a small open-sourced model called Laneformer 2B with 2 billion parameters. The company claims its approach can deliver 30x faster LLM inference overall, though proving this on larger models remains pending. The early response exceeded expectations: Kog accumulated 200 tangible business leads from the preview. Delalleau indicated that software engineering is the most immediate use case, as tools like Claude Code sometimes force users to wait hours for results. Anthropic itself has monetized speed by charging a price multiple for Claude's Fast Mode, validating that customers will pay for faster inference. Kog is also in talks with design partners enabling users to generate games and apps from prompts, where reduced latency would translate directly into more revenue per session. However, Kog faces a technical leap: its demo worked on a purpose-built small model, whereas enterprise demand is skewed toward large language models. Delalleau is confident the same optimization techniques will apply to larger systems, arguing that the misconception—that GPUs are poorly suited for decoding—stems from a failure to exploit newer GPUs' growing memory bandwidth. Delalleau himself studied solid-state physics at France's École Polytechnique and spent years in offensive cybersecurity (white-hat hacking), participating in DEFCON's CTF tournament four times. This background shaped his team's methodology: on the science side, understanding GPU physics to maximize efficiency; on the hacking side, reverse-engineering hardware down to assembly language and binary to repurpose it for unintended goals. The downside is labor-intensive: Kog dedicates several weeks or even months of GPU engineering research for each new chip it supports. With only 11 employees, this limits the chips the startup can target in the near term, though Delalleau hopes to automate the process via agent-based pipelines over time. For now, Kog is focused on delivering its first major model optimization at 10x speed by September, a milestone Delalleau flagged as necessary to demonstrate customer traction and secure Series A funding. The startup is backed by Scaleway, Bpifrance, and France's Tech 2030 program, positioning it within Europe's broader push for sovereign AI capability—a potential tailwind as the continent seeks independence from U.S. chip and software vendors.