Best Cloud GPU Providers with NVIDIA A40

What the NVIDIA A40 actually is

The A40 is an Ampere-generation data-center GPU built on the GA102 die — the same silicon family that powers NVIDIA’s high-end Ampere workstation and consumer cards. It sits in NVIDIA’s professional visualization line rather than its compute-first A100 line, which shapes everything about how it rents and what it is good at. The headline feature for anyone renting it is memory capacity: the A40 carries 48 GB of GDDR6 with ECC, a large pool that lets you load sizeable models and datasets without resorting to multi-GPU sharding.

Because it uses GDDR6 rather than the HBM2e found on the A100, its raw memory bandwidth is lower — in the rough range of hundreds of GB/s rather than the multiple-terabytes-per-second that HBM cards deliver. That single difference is the most important thing to understand when reading the comparison above: the A40 gives you abundant VRAM at modest bandwidth, which is a very different value proposition from a bandwidth-bound HBM card.

Compute and precision support

The A40 includes Ampere’s third-generation Tensor Cores and second-generation RT cores. For AI work, the relevant precisions are well covered:

• FP16 and BF16 for mixed-precision training and inference, with Tensor Core acceleration.
• TF32, Ampere’s tensor format that speeds up FP32-style training math with minimal code changes.
• INT8 and INT4 for quantized, high-throughput inference.
• Structural sparsity support, which can roughly double effective Tensor Core throughput on models trained to exploit it.

What it does not have is FP8, which arrived with the later Hopper generation (H100) and the Ada and Blackwell families. If your workflow specifically targets FP8 training or inference kernels, the A40 is the wrong card and you should filter for a newer generation in the list above.

Interconnect, scaling and power

The A40 connects over PCIe Gen4 and supports NVLink to bridge a pair of cards, which can present a combined 96 GB pool for memory-hungry jobs. It does not scale across many GPUs with the same dense NVLink fabric (NVSwitch) that the SXM A100/H100 use, so it is best thought of as a strong single- or dual-GPU card rather than the foundation of an eight-way training cluster. It is a passively cooled, server-oriented board in roughly the 300 W class, which is part of why providers can pack it densely and offer it at attractive rates.

Which workloads the A40 genuinely fits

The A40’s profile — lots of VRAM, moderate bandwidth, full Ampere AI features, plus real RT cores — makes it a versatile mid-tier rental. It is a strong match for:

• Fine-tuning and LoRA/QLoRA of mid-sized language models, where 48 GB lets you keep optimizer states and longer sequences resident.
• High-throughput batch inference for models in the 7B–34B parameter range (especially quantized), where capacity matters more than peak bandwidth.
• Rendering, 3D, virtual workstations and visualization, where the RT cores and large frame buffer are directly useful — this is the card’s home territory.
• Computer vision, diffusion-model image generation, and general ML experimentation that fits comfortably in 48 GB.

It is overkill for tiny models or light dev work where a smaller, cheaper card would do, and it is underpowered for frontier-scale pretraining of very large models, where you want HBM bandwidth, FP8, and dense multi-GPU interconnect. For latency-critical real-time inference at extreme throughput, a newer inference-optimized card will usually win on tokens per second per dollar even if it has less VRAM.

Renting the A40: cost and availability context

In the rental market the A40 generally sits in the mid tier — meaningfully cheaper per hour than A100/H100-class HBM cards, but above entry-level consumer GPUs. Its appeal is the cost-per-gigabyte-of-VRAM: when your bottleneck is fitting the model in memory rather than crunching it at maximum bandwidth, the A40 is often the most economical way to get 48 GB. Because it is a mature, widely deployed card, on-demand availability tends to be good and it is less prone to the scarcity and waitlists that hit the newest accelerators.

Many providers also offer it on spot or interruptible tiers at a further discount, which suits checkpointed fine-tuning and batch jobs that tolerate restarts. Exact rates move constantly and differ by provider, region, and commitment, so use the comparison above for live per-hour pricing rather than any fixed figure — and when you compare, weigh on-demand versus spot, billing granularity, and whether NVLink pairing is offered if you need the 96 GB combined pool.

Frequently asked questions

How much VRAM does the NVIDIA A40 have?

The A40 has 48 GB of GDDR6 memory with ECC. That large capacity is its main draw for rental, letting you fine-tune mid-sized models or run quantized large-model inference on a single card.

Is the A40 good for training large language models?

It is well suited to fine-tuning and training mid-sized models thanks to its 48 GB pool, but it is not ideal for frontier-scale pretraining. For that you want HBM bandwidth, FP8 support, and dense multi-GPU interconnect found on newer Hopper or Blackwell cards — filter the list above for those if pretraining is your goal.

How does the A40 compare to the A100?

Both are Ampere, but the A100 uses high-bandwidth HBM2e memory for far greater bandwidth and supports dense NVSwitch scaling, making it stronger for large-scale training. The A40 trades that bandwidth for a large 48 GB GDDR6 pool plus RT cores, and rents for less — a better fit when capacity and cost matter more than peak throughput.

Does the A40 support NVLink for multi-GPU jobs?

Yes, two A40s can be bridged with NVLink to present a combined 96 GB memory pool, which helps for models that do not fit on a single card. It does not, however, scale across many GPUs with the dense fabric used by SXM data-center cards, so think of it as a single- or dual-GPU rental.