Study of 34,000 AI skills reveals they boost benchmark scores but fail to improve real-world agent performance, with weaker models actually degrading when using them.

Researchers tested 34,000 modular skills designed to enhance AI agents' specialized knowledge capabilities

Skills that appear effective in benchmark tests provide minimal practical benefit in realistic conditions

Weaker AI models actually perform worse when equipped with skills compared to baseline performance without them

The findings suggest a significant gap between how skills perform in controlled laboratory settings versus practical deployment scenarios