Study finds 11 frontier LLMs fail to faithfully sample from probability distributions, with batch generation achieving only 7% median pass rate and independent requests collapsing almost entirely.

Researchers benchmarked 11 frontier LLMs across 15 distributions using two protocols: Batch Generation (N=1000 samples in one response) and Independent Requests (N=1000 stateless calls). Batch generation achieved a 7% median pass rate; 10 of 11 models passed none of the distributions in independent requests.

Sampling fidelity degrades monotonically with distributional complexity and worsens as the sampling horizon N increases. The sharp protocol asymmetry suggests current LLMs lack a functional internal sampler (a mechanism to generate random samples matching specified probability distributions).

Downstream failures introduce systematic biases: models fail to enforce uniform answer-position constraints in Multiple Choice Question generation and systematically violate demographic targets in attribute-constrained text-to-image prompt synthesis, indicating a need for external tools when statistical guarantees are required.