TrueForge harness + Jev model matches Opus at 150x lower decision cost

The agent was built as a weekly client follow-up tool that matches requests to GitHub issues, a task where the decision is picking one option from a short list. The stack pairs TrueForge as the agent harness, Jev for matching decisions, and Daytona as the sandbox for code execution. In testing on 40 requests, the agent achieved the same accuracy as letting Opus make every match, but at 150x lower decision cost. When Jev returned low-confidence matches, those went back to Opus for review, which caught 5 wrong matches that would otherwise have gone through.

The post also details how to train a JEV-style decision model locally. Using Qwen3.5 0.8B, accuracy went from 37% to 65% in 60 steps, with 10 minutes of training on 4GB of VRAM. TrueForge itself keeps context small by loading tool definitions only when needed and offloading large outputs to files. On DevRev's Enterprise Bench, it solved about as many tasks as Claude Managed Agents on roughly a third of the tokens, and it keeps credentials outside the sandbox while holding every write for approval. TrueForge is fully open source and runs with a single command.