GPT-6 Astra hits over 99% test correctness at $1

The original Zstd eval with GPT-5.6 Sol showed that asking agents to use specific testing techniques like TDD or QuickCheck often made them worse, and that not specifying any method produced many worthless unit tests but performed among the best. Re-running with GPT-6 Astra, the picture changed in two ways: capability rose enough to saturate the benchmark, and agents used test and verification techniques more effectively. For fuzzing, 5.6 Sol generated many worthless random inputs that hit the same code paths, while 6 Astra produced structured fuzzing that exercised code paths and found non-trivial bugs, though the Zstd benchmark is too easy to show whether that improved correctness. The author also tried the IMAP RFC, which was too hard for 5.6 Sol, and found that with 6 Astra the default condition once again looked pretty good, with some conditions giving somewhat better correctness at higher cost. Because the Pandoc eval only ran each condition 25 times at about $90k MSRP, the author cautions that the results have wide conceptual error bars and should not be over-interpreted. Still, several observations stood out: differential testing caused agents to examine boundary conditions and write tests that correctly implement both sides, TDD underperformed again, ACL2 performed well, and property-based testing libraries finally produced tests that do what one might hope. The author contrasts these findings with a viral claim that AI-written unit tests are empirically proven unhelpful, noting that claim was based on a 9% cost saving on smaller problems and that the DeepSWE grader penalizes test addition in ways not representative of real-world usage.