Open-Source AI

Today's Stories

1. 1

   Free open-source book 'How to Make Your Model Fast' targets ML performance

   A Reddit user published a free, open-source book called "How to Make Your Model Fast: A Systems View of Efficient Machine Learning, from Silicon to Agents," which covers roofline analysis, hardware, kernels, compilers, quantisation, pruning, vision, on-device LLMs, robotics, profiling, serving, and agents. The book argues that reducing FLOPs doesn't necessarily make a model faster, so before optimizing anything, you need to understand what the system is actually bounded by — compute, bandwidth, memory, or the system itself.

2. 2

   NVIDIA Open Agent Safety Platform guards AI agents

   NVIDIA announced the NVIDIA Open Agent Safety Platform, made of the open-source OpenShell runtime and the NVIDIA Sentry reference system on BlueField-4 DPUs, which it says can isolate or halt agents in milliseconds. The platform enforces boundaries from outside the agent itself, so control holds even if the agent misbehaves or is compromised—a check the agent's own safety mechanisms cannot provide.

   Adoption hinges on whether the many listed partners actually ship integrations; Anthropic, Salesforce, and SAP each plan to embed OpenShell, and Figure in robotics.

3. 3

   Google Cloud Developer Plugin for AI Coding Agents announced

   Google Cloud announced the Google Cloud Developer Plugin for AI Coding Agents, which gives coding agents such as Claude Code and Google Antigravity Google Cloud expertise and tools. Coding agents are strong programmers but lack Google Cloud service and architecture knowledge by default, so the plugin is meant to enable proper system building on Google Cloud.

   The plugin is built on an AI agent plugin specification combining skills and related MCP server settings, and is available from its GitHub repository, so the test is whether one-step installation works across different agents.

4. 4

   Jev-compatible Jeff hits 22ms per decision on RTX PRO 6000

   Jeff, a small decision model packaged like Jev, comes in three versions built on Qwen3.5 0.8B, Qwen3.5 2B and Gemma 4 E2B, with a decision taking about 22 milliseconds on RTX PRO 6000 and 28 milliseconds on M4 Max. Jeff's README stresses it does not reason, and its backers say no model of this parameter size should be expected to handle multi-step reasoning, so its edge looks confined to fast, narrow choices.

5. 5

   NVIDIA ships Isaac ROS 5.0 with AI agents built in

   NVIDIA released NVIDIA Isaac ROS 5.0, an open-source robotics development environment that now integrates AI agent capabilities directly into the ROS developer ecosystem, per the company. Developers can reportedly get AI agent help automating repetitive work and understanding complex codebases, which NVIDIA says speeds the move from idea to a working application.

   Adoption hinges on the robotics companies already building it into their platforms. The body names Universal Robots and ROBOTIS, and the release supports ROS Lyrical and Ubuntu 24.04.

6. 6

   MicroLLM lab lets users benchmark tiny 135M models in their browser

   MicroLLM lab launched as a browser tool for benchmarking tiny LLMs, including 135M-parameter models, on objective checks, reporting speed in tokens/s and accuracy as a pass rate, with all numbers kept on the local machine. The tool treats failures as the measurement itself — a 135M model is allowed to fail — and uses objective checks rather than writing-quality judgments, so results are meant to be comparable.

   Runs are confined to the user's browser and hardware, so scores are not directly comparable across devices. Watch which models and objective tests users actually select when the tool is used.