What does LLM-as-a-Judge mean in this context?

It refers to using a generative AI model to automatically evaluate Japanese ad copy, incorporating the copywriters' evaluation criteria.

Where will the research be presented?

The research will be presented as a paper at the 25th Information Science and Technology Forum (FIT2026), held from September 2 to 4, 2026.

Top Companies' AI MovesLarge Language ModelsTop Companies AI2h agoSep 3, 2026, 06:31 JST1 min read

Dentsu and SB Intuitions Build 116K Ad Copy Evaluation Dataset

Send on LINE

Key takeaway

Dentsu, Dentsu Digital, and SB Intuitions are developing a Japanese copywriting-focused generative AI.
They built a dataset with 116,532 ad copy evaluations by 64 copywriters.
The AI's automated scoring matches human judges in trained categories but drops in others.

3 Key Points

What happened
Dentsu, Dentsu Digital, and SB Intuitions announced that they have built a dataset of over 110,000 ad copy evaluations by 64 Dentsu copywriters, as part of their joint research on a Japanese copywriting-focused generative AI.
Why it matters
In tested categories, the AI's automated evaluation showed a positive correlation with human copywriter judgments, suggesting it can learn copywriting expertise. However, the correlation dropped for untrained categories, revealing that prompt adjustment alone is not enough for broad application.
What to watch
The research paper, titled "Verification of LLM-as-a-Judge for Japanese catch copy evaluation," will be presented at FIT2026, held from September 2 to 4, 2026, in Kitakyushu.

Ask the AI about this article →

Summaries like this, in your inbox every morning.

Get news like this every morning — free

Context & Analysis

The joint research combines Dentsu's creative expertise and Dentsu Digital's AI solutions with SB Intuitions' domestic generative AI "Sarashina," built on SoftBank's AI computing infrastructure. This collaboration aims to create a generative AI that understands Japanese nuances and can produce emotionally resonant ad copy. The dataset and systematic evaluation criteria are early steps toward practical automated evaluation, but the drop in performance for untrained industries highlights the need for more data and model refinement. The findings will be shared with the academic community at FIT2026.

FAQ

What does LLM-as-a-Judge mean in this context?: It refers to using a generative AI model to automatically evaluate Japanese ad copy, incorporating the copywriters' evaluation criteria.
Where will the research be presented?: The research will be presented as a paper at the 25th Information Science and Technology Forum (FIT2026), held from September 2 to 4, 2026.

Get the latest Top Companies' AI Moves news every morning

For example, today's edition would include:

Vertiv to Buy UtilityInnovation for $1.45BTop Companies AI · 2h ago
MBody AI Orchestrator Named Finalist for AI Deployment of the YearTop Companies AI · 2h ago
Interactive Brokers bets on AI-assisted investingTop Companies AI · 2h ago

AI-summarized, only the topics you pick — one digest a day via Email, Slack, or Discord.

Subscribe free →

Free · takes 30 seconds · unsubscribe anytimeWhat is AIToday? →