Study reveals that AI verification helps fix flawed tutoring feedback for logic proofs, but creates problems when initial guidance is accurate

Researchers evaluated LLMs as tutors for propositional logic proofs using a new benchmark with 516 unique proof states and step-level annotations

Three specialized AI roles were tested: Tutor (partial solution access), Teacher (full derivation access), and Judge (verification capability)

Verification improved outcomes by 85% when upstream feedback was error-prone, but the study identified a critical shared limitation in the verification approach

Unlike previous tutoring evaluations relying on model self-assessment or binary correctness checks, this framework enables fine-grained analysis against verified solution paths