
What happened
Jane Street published socratic-tutor, a 102-line text file in its agent_skills repo, on 2026年11月3日; a test on Gemma-3-1b-it, Qwen3.5-0.8B and Qwen2.5-1.5B-Instruct found none ran its diagnostic test, skill map or one-question flow.
Why it matters
The skill is only an instruction sheet, so its success depends on the instruction-following ability of the model it is pasted into, and 1B–1.5B local models appear too small to run it.
Summaries like this, every morning.
Jane Street published the agent_skills repository on 2026年11月3日, and socratic-tutor was its first skill. Interest built quickly: the repo hit the Hacker News front page on 2026年11月10日 with 697 points, and GitHub stars climbed from 8,273 on 2026年11月14日 to 8,434 the next day and 9,618 by 2026年11月17日. Several English explainer articles appeared between 2026年11月14日 and 16, but the tester found almost no Japanese-language hands-on guides.
The design assumes a strong instruction-follower. The skill splits a topic into 4–10 sub-skills scored 0–5, runs a 3–5 question diagnostic one question at a time, withholds answers until two or three failed attempts, and schedules spaced review and a final exam. The test environment was deliberately weak — a shared Linux VM with about 1GB of free memory and no GPU — and response times ranged from 6.5 to 130.2 seconds. Gemma ignored the instructions or echoed the prompt text, Qwen3.5-0.8B repeated the same diagnostic wording until it hit the 1,024-token limit, and Qwen2.5-1.5B-Instruct asked for an equation and then offered an answer, contrary to the skill's rules.
The tester notes that Jane Street has not published any evaluation, so whether Socratic tutoring actually improves learning remains a design intention, and Hacker News commenters debated whether the skill is just a prompt. The skill itself says not to use it for one-off "explain this" questions. The body also leaves open which model size the skill starts working at, and how it behaves in long Japanese sessions on commercial chatbots — the setup it was designed for — was not tested.
Pick your industry and the AI tools you use, and get news related to your work every day.
Free · 30 seconds with Google · unsubscribe anytimeWhat is AIToday? →
Ask AI anything about this article. The AI reads this article, earlier AIToday articles, and Wikipedia, and cites its sources. Q&As are published on this page for other readers too.
Since late October, users of OpenAI's paid plans have reported Codex and ChatGPT Work burning through usage qu…

AgeAI, led by CEO Gen Mikami, joined the Gotanda Valley Acceleration Program 2026 from its September 17 kickof…

HTC NIPPON starts domestic pre-orders for VIVEE on 2026年11月20日(金) via its official online store and Amazon, wi…

Wenge AI (1956.HK) added Google and Apple sign-in options to ScienceOra's international site, letting users au…

Google Research on 2026-09-24 presented four studies — Co-Director, CANVAS, A²RD and VQQA — in which a directo…

The Information reported that Meta's Muse Charm AI device will cost around $300, and a Meta spokesperson did n…
