AIToday
Large Language ModelsAI Safety & AlignmentZenn AI/MLPublished: Nov 22, 2026, 04:00 JST

Muse operator shares multi-stage AI review results

Send on LINE
Muse operator shares multi-stage AI review results

3 Key Points

  1. What happened

    A practitioner customizing an agent called Muse had other models review documents and design proposals, found 5 of 6 critiques off-target in one inquiry-draft review, and built a multi-stage setup where models critique separately and a Gemini judge via Vertex AI and OpenRouter rates each point.

  2. Why it matters

    Because model critiques can be hit or miss and vary by case, the author treats AI feedback as a bundle of reference opinions — anonymizing points before judging and keeping a human final call — rather than adopting suggestions directly.

Not sure about something? Ask the AI

Questions and answers are published on this page.

Summaries like this, every morning.

Context & Analysis

The account is a detailed operational log rather than a product announcement. The author started by having a single model review documents, but found critiques could be confidently off-target, so they split review into two rounds: a first round where several models with different roles (logic check, devil's advocate, first-time reader, number audit, structure check) surface issues, and a second round where a judge model rates each point. The results across six review tasks showed wide variation: one inquiry-draft review had almost all critiques rejected, while a spec-compression review had most accepted. The author also describes practical hurdles — rate-limit errors, empty responses from reasoning models, and model availability that shifts by the minute on Vertex AI — and the fallback rules built to keep the workflow running. The overall takeaway is that AI review is treated as a set of opinions to be verified, not answers to be adopted.

FAQ
What is multi-stage review?
Multiple models critique the same target in parallel, their points are anonymized and numbered, then a separate judge model rates each as valid, partly valid, or off-target.
How does the judge model avoid bias?
Points are anonymized so the judge cannot see which model raised them, and the judge is chosen from models not in the first round.
Is this free to run?
Yes, the author uses OpenRouter free models and Gemini via Vertex AI trial credits, with no paid plan.
Send on LINE

AI news that matters for your work, delivered every morning.

Pick your industry and the AI tools you use, and get news related to your work every day.

Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →

Ask AI

Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.

Questions and answers are published on this page.

Related Articles

Next articleShortical debuts AI series "Inevitable" with Aki Avni