i14 runs Laplace 3B on Trainium2 via TorchNeuron Native

i14 positions Laplace 3B as an alternative to standard transformer models, whose key-value cache grows with context and can reduce concurrency and increase serving cost. By replacing softmax attention with Gated Linear Attention, the model compresses history into a fixed-size recurrent state. Bringing this custom architecture to Trainium2 required more than a port: the original GPU-optimized Triton kernels had to be re-expressed with standard PyTorch operators, and the Muon optimizer's communication pattern was redesigned under FSDP2. The team chose the beta version of TorchNeuron Native after comparing approaches because it performed better. The reported training run, still in progress at about 44 billion of 100 billion tokens, showed a smooth loss decline and stable gradients. On the serving side, an unmodified vLLM engine matched i14's reference outputs token for token, and profiling showed that operator dispatch—not arithmetic or memory bandwidth—dominated decode time, which compiled decode addressed. These results suggest that Trainium2 can accommodate architectures beyond standard transformers, though teams would need to assess retrieval quality for their own workloads.