Distributed LLM inference over the open internet faces fundamental constraints from high-latency networks, limiting viable parallelism strategies.

A 400-billion-parameter model at FP16 precision requires roughly 800 GB of memory, exceeding any single GPU's capacity and forcing distributed inference across multiple devices.

Three parallelism methods exist: data parallelism (identical copies on separate GPUs), pipeline parallelism (model split vertically by layers), and tensor parallelism (single layer split horizontally with all-to-all synchronization). Tensor parallelism works within centralized racks using NVLink's 900 GB/s bandwidth and microsecond latency, but over the open internet—offering 10 to 100 MB/s with tens to hundreds of milliseconds latency—would cause GPUs to spend 99 percent of time waiting for network round trips rather than computing.

Pipeline parallelism becomes the only viable route for decentralized scenarios, accepting sequential layer execution but enabling point-to-point communication between stages rather than collective synchronization; however, this creates bootstrap problems (minimum GPU threshold before inference is possible) and weight migration challenges when network composition changes.