Educators worldwide shift to oral exams, code reviews as AI solves assignments

The ACM Task Force on Generative AI and Programming Assessment, an expert group set up by the education committee of the international computing society ACM, surveyed 763 educators from 49 countries between May and October 2025. Analysis focused on about 500 nearly complete questionnaires. Respondents came largely from North America (227) and Europe (106), with smaller representation from Asia (57), South America (10), Oceania (9), and Africa (3); 77 percent worked at universities, with K-12 schools and vocational programs barely represented.

The findings paint a picture of widespread adaptation. Sixty-four percent of respondents have already changed how they teach, with a thematic analysis of open-ended answers pointing toward a clear direction: away from writing code from scratch and toward code comprehension, debugging, and problem-solving. Thirty-nine responses explicitly mentioned teaching AI use and prompt engineering as topics. Steven Gordon, a professor at Ohio State University and lead author of the report, framed the underlying philosophy: "We know that students will likely be developing with AI tools when they enter the workforce. The key is to produce graduates who are competent in programming and who understand both the capabilities and limitations of AI."

Changes in assessment are even more striking. Sixty-eight percent of respondents have adjusted their testing methods. The most common shifts include more proctored in-person exams (56 mentions), reduced weight for homework (38), oral exams and code defense sessions (36), and paper-based tests (35). Project-based assessments (34) are also on the rise. Some educators now require students to disclose their AI use and submit logs of their interactions with AI tools. Institutional policies remain fragmented: 45 percent said their institution has guidelines on AI use, while 39 percent said it does not, with policies ranging from outright bans to explicit permission or encouragement as long as usage is documented.

The biggest barrier cited by respondents is the lack of proven best practices. Forty-eight percent flagged this, while 28 percent said they lack sufficient expertise with AI, and 20 percent see no need to integrate it into their teaching. When it comes to professional development, 74 percent want training on effective teaching methods and 66 percent want help redesigning assessments. Research published separately validates educator concerns: an Anthropic study found that software developers who learned a new Python library with AI assistance scored 17 percent lower on a follow-up knowledge test than a control group without AI access, and a subset that fully delegated coding to AI finished tasks fastest but averaged just 39 percent on the knowledge test. A Chinese longitudinal study tracking more than 26,000 students found that AI use boosted homework grades by 18 percent and cut completion time by 30 percent, but on closed-book exams after six months, scores dropped by 20 percent, with later entrance exams showing declines of 18 and 24 percent depending on the test. A UC Berkeley analysis covering more than 500,000 grades found a sharp rise in top marks after ChatGPT's release, especially in courses heavy on writing and programming assignments, with the spike most pronounced in courses where unproctored homework counted heavily toward the final grade—precisely the type of assessment many educators are now weighting less.

The survey, conducted by the ACM Task Force on Generative AI and Programming Assessment between May and October 2025, captures a profession in rapid transition. The core problem is structural: traditional assessment—homework assignments and take-home coding problems—has become unreliable as a measure of actual skill when AI tools can produce average-quality solutions automatically. Educators recognize this; 69 percent believe AI has changed the skills needed for software development, and the top two concerns are student dependency on technology (87 percent) and cheating or plagiarism (72 percent).

The response is a measured shift toward assessment methods that are harder to game with AI. Oral exams, code defense sessions, and project-based work require real-time reasoning and explanation—tasks where a student cannot simply submit an AI-generated artifact. This mirrors broader moves away from unproctored homework, which the Berkeley analysis showed spiked with top marks after ChatGPT's release. The trade-off is substantial: educators must invest in new methods and training, and 48 percent lack proven best practices to guide them. Notably, 45 percent of respondents work at institutions without formal AI guidelines, suggesting institutional policy is lagging behind classroom practice.

No comments yet. Be the first to share your thoughts!