AWS shows three ways to train Qwen3 on one trn2.48xlarge

The post is structured around three configurations that differ in how the model is built, not in how the job is launched. In the first, Transformers v5 supplies the Qwen3 implementation through AutoModelForCausalLM, and the config simply sets the Hub ID. The second swaps in torchtitan's native Qwen3Model by changing config.model_spec, starting from qwen3_32b() and replacing the model spec with the 4B variant because the tested build lacks a generic 4B builder. The third goes further, replacing the mixture-of-experts block with NKI grouped-experts kernels through qwen3_nki_moe_config(), reduced to 4 layers and 64 experts so it fits on one trn2.48xlarge.

The custom-kernel section is the most procedural. A kernel from nkilib is wrapped with wrap_nki and registered as a PyTorch operator, and a second operator handles the backward pass with saved routing metadata. A numerical check compared rope_hf against torchtitan's PyTorch rotary embedding with float32 inputs; the run reported a maximum absolute difference of 0.0 for query and key. The authors describe this part as demonstrating registration and autograd integration, not performance improvement.

What ties the three paths together is the launcher and the override mechanism. Data parallel sharding is set with data_parallel_shard_degree = -1 across 64 ranks, and tensor parallelism can be layered on the same multidimensional DeviceMesh. Overrides are scoped with fully qualified names so that, for example, fqns=["layers.1[0-9].attention.rope"] targets rotary embeddings in layers 10 through 19. torchtitan halts when two overrides claim the same node. This points to a workflow where switching between Hub-defined and native implementations, or attaching custom kernels, is a configuration change rather than a different training stack.