Study reveals LLM choice for political text analysis is highly task-dependent, challenging popular 'best practices' in AI-assisted annotation

Researchers tested six open-weight LLMs across four political science annotation tasks under identical conditions to evaluate implementation choices

Finding: interaction effects between model choice, size, learning approach, and prompt style dominate individual factors, making pipeline decisions critical researcher choices

No single model, prompt style, or learning approach performed best universally—optimal selection varies significantly by annotation task

Study challenges the assumption that popular 'best practices' in LLM configuration are universally applicable across political science research