A single RTX 5090 GPU achieves impressive throughput speeds of 20 tokens/second for text generation and 700 tokens/second for prompt processing with the massive 397B-parameter Qwen3.5 model.

Qwen3.5-397B-A17B model in Q4_K_M quantization format runs on a single NVIDIA RTX 5090 with 256GB DDR4 RAM and AMD EPYC 7532 CPU

Text generation speed reaches 20 tokens/second while prompt processing achieves 700 tokens/second throughput

System uses PCIe 4.0 x16 connection, 2TB NVMe SSD storage, and llama-bench benchmarking tool with batch size of 8192

Performance data addresses the lack of publicly available speed metrics for running 397B-parameter models on consumer/enthusiast single-GPU setups