OpenAI's Decisions API mirrors Jev for fast, cheap AI choices

OpenAI's Decisions API arrived as an aside from CEO Sam Altman at the company's Dev Day on Tuesday, but it signals a direct response to a model called Jev that TypeSafe AI released earlier this month. Jev is built on an LLM as a kind of super-powered classifier, letting developers give it a set of choices that it outputs as probabilities cheaply and at high speeds. Altman described the new API in similar terms, as a way to give OpenAI's Luna model a predefined set of options to choose between.

The subtext is that LLMs as we know them are not the right solution for a lot of software because they are comparatively slow and expensive. Developers have been using Jev to augment LLMs and found the results faster and cheaper. TypeSafe CEO Diogo Almeida, a former OpenAI engineer, joked on X about the beginning of the clone wars, adding that OpenAI's interest could be a sign that building in a System One compatible way is the future. He also said his company's moat is the synthetic data it creates to generate statistically useful outputs, and that "intelligence-per-dollar" is his North Star.

One likely application is monitoring and securing AI agents. After a series of incidents where its agents misbehaved on the open internet, OpenAI began using a separate model to watch for bad actions at significant compute cost. Shapor Naghibzadeh, who leads the startup QueryStory, built a demo that uses Jev to check each agentic action against its task, blocking bad actions, flagging others, and permitting the rest. In theory, such monitoring could have stopped the Hugging Face incident, and it cost $2.94 with Jev versus $372 with a frontier LLM. It seems clear these models have a future, but a key question is how well calibrated each of these decision models' outputs will be to real life.