How well does NVIDIA GeForce RTX 5090 scale across multiple GPUs?

Odpověď

419 FP16 TFLOPS and 1,792 GB/s of memory bandwidth put NVIDIA GeForce RTX 5090 squarely in the class of accelerators targeted at modern transformer workloads. FP32 caps at 104.8 TFLOPS, which still handles most non-AI scientific compute comfortably.

For training from scratch, token throughput roughly tracks FP16 TFLOPS. For production inference on foundation models, throughput tracks bandwidth. Real-world numbers will depend heavily on the framework stack (PyTorch, TensorRT-LLM, vLLM), and can vary 30-50% depending on how aggressively you quantise.

Ready to deploy? Vast.ai has NVIDIA GeForce RTX 5090 from $0.34/hr. Novita AI and RunPod also carry it.