Study finds automatic prompt optimization rivals expert-crafted prompts for LLM linguistic tasks, with results varying by application.

Researchers compared hand-crafted expert prompts, base DSPy signatures, and GEPA-optimized DSPy signatures across translation, terminology insertion, and language quality assessment tasks.

In terminology insertion, optimized and manual prompts showed mostly statistically indistinguishable quality, suggesting automation may replace manual engineering in this domain.

Translation results were model-dependent, with different approaches outperforming on different configurations, indicating no universally superior prompt strategy.

For language quality assessment, expert prompts excelled at error detection while automated optimization improved characterization ability.

GEPA optimization successfully enhanced minimal DSPy signatures across all tasks, with most expert-optimized comparisons showing no statistically significant differences.