AI Security Lab opens Ollama-based LLM test setup on GitHub

This Zenn article is part of the AI Security Lab series. The previous article explained why the Lab was started and what it aims to clarify about how far work can be delegated to AI. This article focuses on the shared experiment environment that will be used in future articles, separating that explanation from the actual experiment results, which will be covered next.

The Lab uses Ollama to run LLMs locally, and experiments are executed by Python programs that call the Ollama API directly. The code is managed by experiment number under an experiments directory, and the whole setup is public in the GitHub repository kondonator/ai-security-playground. This is meant to make experiments reproducible and comparable by keeping conditions as consistent as possible.

The Lab stresses that results depend on many conditions, including the model, so a finding from one model does not automatically hold for all LLMs, and the same model may not give the same result under different conditions. Because of that, the Lab treats results as observations tied to specific conditions rather than absolute properties, and it is not using multiple models to rank them. The next article is set to move from environment setup to actual experiments, starting with Prompt Injection. Whether that demonstration will generalize beyond the Lab's environment is likely to depend on the specific models and conditions used, and the Lab's approach suggests readers should watch for those details in the upcoming results.