AIToday

RTX 5090 dominates single-GPU performance but hits memory walls, while dual AMD R9700 setups scale better for massive language models in llama.cpp benchmark tests.

r/LocalLLaMAMar 25, 20261 min read
RTX 5090 dominates single-GPU performance but hits memory walls, while dual AMD R9700 setups scale better for massive language models in llama.cpp benchmark tests.

Summaries like this, in your inbox every morning.

Sign up free →

3 Key Points

  1. RTX 5090 achieves exceptional speeds (5,988 t/s prompt processing, 205 t/s generation) on Qwen 3.5 35B MoE but cannot load 72B and 122B models due to 32GB VRAM limitation

  2. Dual AMD R9700 setup (60GB total VRAM) successfully runs 70B models, achieving 11.49 t/s generation speed and scaling beyond single-GPU constraints

  3. Benchmark tested dense models (32B, 70B) and mixture-of-experts models (35B, 122B) from Qwen family using latest llama-bench build 8463

  4. Enterprise DGX Spark (GB10) and AMD AI395 (Strix Halo) with unified memory tested as alternatives to consumer-grade RTX 5090 for local AI inference

Discussion

No comments yet. Be the first to share your thoughts!

Log in to join the discussion

Related Articles

Stay ahead with AI news

Get curated AI news from 200+ sources delivered daily to your inbox. Free to use.

Get Started Free

Free · takes 30 seconds · unsubscribe anytime

1 minute a day. The AI essentials.

200+ sources · Email / LINE / Slack

Get it free →