Researchers benchmark AI financial analysts against professionals—and find AI reports still fall short on accuracy and credibility

Researchers introduced Deep FinResearch Bench, a standardized test for evaluating AI agents (autonomous AI systems that conduct research independently) that write financial investment reports. The benchmark measures three dimensions: how rigorous the qualitative analysis is, how accurate the numerical forecasts and valuations are, and whether the claims made can be verified. When applied to reports from frontier AI research agents versus reports written by human financial professionals, AI-generated reports consistently underperformed across all three measures.

Unlike earlier AI evaluation methods that only count speed or token output, Deep FinResearch Bench includes automated scoring for verifiability—meaning it checks whether the AI's claims actually match the data sources it cites, catching hallucinations (confident false statements). This lets researchers at scale identify where AI financial reports break down: shaky reasoning, bad number predictions, or made-up citations.

For investment professionals and fintech companies building AI-powered research tools, this benchmark signals that today's general-purpose AI models (like ChatGPT) are not ready to replace financial analysts. Finance requires domain expertise (bond valuations, regulatory filings, sector dynamics) that generic AI lacks. Teams considering AI research assistants should expect to fine-tune models specifically for finance rather than deploying off-the-shelf AI, or risk wrong stock picks and regulatory problems from unverifiable claims.