UIUC builds AI teaching assistant for electrical engineering, open-sources core system

The University of Illinois Urbana-Champaign has launched a multi-modal AI teaching assistant tailored to their introductory electrical engineering course, ECE 120. The system architecture runs 11 separate models in parallel, each handling different tasks: text retrieval, image retrieval, answer generation, content moderation, and ranking. Despite this computational parallelism, the system achieves a median response time of 2 seconds, making it practical for real classroom and student-help scenarios.

The system ingests three categories of training data: textbooks, lecture videos, and student Q&A forums, which the authors ordered subjectively by importance. Because the project was not granted rights to release this data publicly, those source materials remain private. However, the team has released the reinforcement learning from human feedback (RLHF) dataset openly on HuggingFace. This dataset was iteratively built by hiring five electrical engineering students and covers the specific material taught in ECE 120. This approach—having actual domain experts in the subject matter create comparison labels—strengthens the quality of training signals compared to generic crowdsourced feedback.

Evaluation is continuous: whenever a new feature is pushed to the system, the team re-runs an assessment using an in-house dataset of question-and-answer pairs written by expert electrical engineers. For each question, the system generates an answer from each candidate model, and GPT-3 judges whether that answer is better or worse than the human ground-truth. The team is candid about a limitation of this approach: GPT-3 nearly always rates GPT-3-generated answers as superior, which likely does not reflect reality. They note they should run the same evaluation using other models, such as Cohere's, to obtain a more unbiased comparison.

The project is fully open source (except for the proprietary textbook content), and the team encourages others to adapt it. Installation requires Python 3.8 and standard dependencies, plus API keys for the services the system uses. The critical step is building a Pinecone database—a vector store for retrieval—populated with documents from the user's own course. The team provides helper scripts to convert textbook PDFs and lecture slide images into searchable embeddings, as well as a script to ingest video transcripts from OpenAI's Whisper. Once set up, users run a single bash command to launch the Gradio web interface. The full codebase, including the aggregation logic, prompt engineering, evaluation code, and real user feedback logs, is available on HuggingFace.

The UIUC teaching assistant addresses a genuine constraint in deploying AI to education: most large language models are trained on broad internet data and lack access to course-specific materials like textbooks, lecture recordings, and institutional Q&A forums. By combining retrieval-augmented generation (pulling context from a searchable database) with reinforcement learning from human feedback, the team built a system that is aware of the exact material students are being taught.

A key technical choice is the use of semantic search during RLHF—rather than training the model only on raw comparison labels, they use semantic similarity to retrieve relevant context before ranking answers. The publicly released RLHF dataset, created by five electrical engineering students and covering ECE 120 material, is freely available on HuggingFace, lowering the barrier for others to replicate or extend this approach.

The evaluation methodology reveals both the system's strength and a known weakness. Because the team uses GPT-3 to judge whether answers are good, and GPT-3 tends to favor its own outputs, the evaluation results may not reflect the true quality gap versus other models. The authors are transparent about this and propose validating results with alternative evaluation models, suggesting a mature approach to system building even as they acknowledge their current methodology's limits.