AIToday
Large Language ModelsarXiv cs.LGPublished: Apr 28, 2026, 13:00 JST

CoFi-PGMA framework corrects learning signals for multi-agent LLM systems using counterfactual policy gradients

Send on LINE

3 Key Points

  1. Researchers introduced CoFi-PGMA, a framework for training multiple language models that either compete through routing mechanisms or collaborate to produce answers, addressing the problem that standard reinforcement learning objectives become misspecified when learning signals are filtered by system mechanisms.

  2. The approach derives per-agent training objectives based on marginal contribution: for routing systems it applies off-policy corrections for selection-gated feedback, while for collaborative systems it uses leave-one-out difference rewards for credit assignment (a method that measures each agent's individual contribution by comparing performance with and without that agent).

  3. The framework analyzes how softmax routing induces risk-sensitive incentives and provides practical training algorithms that integrate counterfactual estimators, multiturn-aware rewards, and policy optimization methods, with demonstration on a real-world reasoning dataset.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleChina orders Meta to unwind $2bn acquisition of AI app Manus