Shared base cuts 100 fine-tunes from 1.5 TB to 19.3 GB

The starting point for this experiment is a simple mismatch. A fine-tuned variant of a 7B model begins as a rank-8 LoRA adapter of roughly 40 MB, but the conventional serving stack merges that adapter into a complete 15.2 GB copy of the model. Stack 100 variants and the footprint reaches 1.5 TB. The article walks through four deployment options — merged with one endpoint per variant, unmerged with adapters registered at startup, unmerged with adapters resolved at request time, and a hosted per-tenant option — and notes that merging still makes sense when variants need different base models, different GPUs, or complete isolation. When every adapter comes from the same base, though, the copies mostly duplicate memory and workers.

The write-up then tests that claim on Runpod Serverless with a 1.5B Qwen model, training three adapters for extraction, SQL generation, and request routing, and comparing one shared endpoint against three per-model endpoints under the same five-minute traffic stream at one request per second. The measured cold starts were 409, 245, and 214 seconds, with execution staying near half a second; the article attributes that spread to an unoptimized setup where each cold start pulls the vLLM worker image and downloads the base model and adapters from Hugging Face. The separate layout's lower billed worker-seconds came with more requests waiting more than two minutes, which is the operational point the memory calculation misses.

The stakes here look practical rather than architectural. For teams running a family of adapters on one base, the article's default is to keep them separate and let every request share the worker pool. Whether that holds up in a given deployment likely hinges on the serving provider's cold-start configuration — caching the model, baking it into the image, or keeping FlashBoot enabled — and on how steady the traffic is, since startup consumes a larger share of the bill when traffic is sporadic.