Humanet's airgap-check finds AI app offline downloads at 89.7% precision

The tool targets a gap between the assumption that pointing an LLM base URL at a local vLLM or Ollama instance makes an app offline-ready and what libraries actually do on first use. Humanet says many libraries fetch tokenizer ranks, ONNX embedding weights, corpora, price maps or telemetry once and cache them, so the problem only appears when the network is gone.

The benchmark ran probes against 20 open-source Python and TypeScript AI repositories in two phases, with the rules frozen before the held-out half. At component level it counted 35 true positives, 4 false positives and 5 false negatives; the false positives came from dev and test dependency groups in pyproject.toml, while the false negatives were downloads like OCR weight fetchers and FlagEmbedding that were not in the rule table. Humanet notes the --run mode is meant to catch such misses, but only on the paths it exercises.

The clearest caution in the write-up is about applying --fix: if your app loads Hugging Face models, pre-warm them at build time before setting HF_HUB_OFFLINE=1 and TRANSFORMERS_OFFLINE=1, or the app fails at startup instead of hanging. Humanet also states airgap-check is not a security certification, and that a clean result does not prove an application is safe to run in an isolated network.