Researcher trains Snake AI to 86-point average in under 10 hours on free GPU

The project tackles the classic Snake game using reinforcement learning, a machine learning approach where an AI learns by trial and error, receiving rewards (high score when the snake eats food) and penalties (low score when it crashes). The developer's goal was to reach high scores while minimizing training time.

The final system achieves an average score of 86 points—just one point shy of the theoretical maximum of 87—after less than 10 hours of training on a free Google Colab T4 GPU, a modest but capable graphics processor. The speed comes from running 4,096 Snake games simultaneously on the GPU rather than one at a time. This parallel execution is combined with PPO (Proximal Policy Optimization), a popular reinforcement learning algorithm, and GAE (Generalized Advantage Estimation), a technique that reduces training noise by estimating future rewards more accurately.

The neural network uses a CoordConv architecture, a design that explicitly preserves the spatial layout of the game grid. Rather than forcing the network to infer the grid's structure from raw game observations, CoordConv provides coordinate information directly, allowing the model to focus on learning strategy rather than geometry.

The developer posted the project to r/MachineLearning seeking feedback from the community on what optimization to pursue next: whether to refine exploration strategies (the way the AI discovers new moves), improve the reward function (what scores the AI receives for different actions), redesign the neural network, or try a different approach entirely. The full source code is available on GitHub for anyone interested in experimenting with or extending the work.

The post represents a practical demonstration of efficient reinforcement learning optimization on constrained hardware. By running thousands of game environments in parallel on a single GPU rather than sequentially on CPU, the developer compressed what might ordinarily require days of training into a single-digit hour span. The architectural choice of CoordConv—which maintains explicit spatial coordinates in the neural network—is a deliberate design decision to help the model learn the inherent grid structure of Snake without relearning it implicitly from pixels.

The near-ceiling performance (86 out of 87 maximum) achieved in under 10 hours suggests the pipeline is well-tuned for the problem domain. The developer's explicit invitation for feedback on what to optimize next (exploration, reward design, or architecture) signals an understanding that marginal gains likely require more sophisticated tuning rather than raw compute scaling.

No comments yet. Be the first to share your thoughts!