The inference market is fragmenting into specialized segments by workload type, latency requirement, and deployment location, mirroring how the database market split into multiple categories.

The inference market is splitting into distinct segments: real-time (sub-100ms) for voice assistants and autonomous vehicles, near-real-time (100ms-2s) for chatbots and code completion, batch (seconds to hours) for document processing, multimodal for image and video generation, and edge for on-device deployment on phones and industrial sensors.

Different workload types create different bottlenecks. Chatbots are memory-constrained (the model must hold entire conversations), while image and video generation are compute-heavy (a single image requires 50 sequential passes). Edge devices like Apple's on-device model (3-billion-parameter) and Tesla's vision chips (drawing 72 watts) face power and memory constraints that differ from cloud inference.

NVIDIA's data center revenue grew 17× in three years following ChatGPT's launch, showing the scale of the inference opportunity. The article projects a $100B inference market fragmenting similarly to how the database market produced Oracle, MongoDB, Databricks, and Snowflake.