Best Cloud GPU Providers with NVIDIA H100

What the NVIDIA H100 actually is

The H100 is NVIDIA’s data-center accelerator built on the Hopper architecture, the generation that sits between the older Ampere A100 and the newer Blackwell parts. It is the GPU most teams reach for when they want serious large-model training or high-throughput inference without dropping all the way down to consumer cards. When you rent an H100 instance from the comparison above, you are renting a purpose-built AI accelerator rather than a repurposed gaming GPU, and that shapes both what it can do and what it costs.

The key hardware traits that matter for a renter:

• Memory: the SXM5 variant ships with 80 GB of HBM3, while the PCIe variant uses HBM2e, also at 80 GB on the mainstream part. A later refresh, the H100 NVL, raises capacity per card. The large, fast HBM is the single biggest reason to pick this card over GDDR6-based options.
• Memory bandwidth: HBM3 on the SXM5 part delivers roughly 3 TB/s, far above what consumer GDDR6/GDDR6X cards reach. Bandwidth, not raw FLOPs, is what keeps large transformer layers fed.
• Tensor cores and precisions: fourth-generation tensor cores support FP16, BF16, TF32, INT8, and — new with Hopper — FP8. FP8 is the headline feature for modern LLM workloads, roughly doubling throughput versus FP16 on supported kernels while keeping accuracy acceptable with the right scaling.
• Transformer Engine: Hopper pairs FP8 hardware with software that dynamically manages precision per layer, which is why H100 throughput on transformer training and inference can pull well ahead of the previous generation on the same model.
• Interconnect: SXM5 boards use fourth-generation NVLink and NVSwitch for high-bandwidth GPU-to-GPU links inside a node, while PCIe cards rely on the PCIe bus (with optional NVLink bridges on some variants). This distinction matters enormously for multi-GPU jobs.
• Power class: the SXM5 card is a roughly 700 W part in an 8-GPU server chassis; the PCIe card sits lower, around 350 W. You do not pay the power bill directly when renting, but it explains why these instances are dense, hot, and priced accordingly.

Which workloads the H100 genuinely fits

The H100 earns its keep on memory-bound, throughput-hungry jobs. It is a strong match for:

• Large-model training and fine-tuning: 80 GB of HBM lets you hold bigger batches and larger parameter shards per GPU, and NVLink makes multi-GPU and multi-node scaling efficient for data- and tensor-parallel training.
• High-throughput LLM inference: FP8 and the Transformer Engine make it excellent for serving large language models at high request volume, where you care about tokens per second per dollar.
• Fine-tuning of mid-to-large models: full fine-tunes and parameter-efficient methods on multi-billion-parameter models fit comfortably where smaller-VRAM cards force aggressive offloading.
• Scientific and HPC compute: strong FP64 throughput (unlike consumer cards, which are deliberately weak here) makes it viable for simulation and numerical work, not just AI.

It is overkill for small-model experimentation, light inference of compact models, classical ML, notebooks, and most rendering or visualization work — those run fine on far cheaper cards, and renting an H100 for them mostly wastes money. It is rarely underpowered for single-node work; the main thing that pushes teams past it is needing more aggregate memory than a single node of these cards provides, or wanting the newest-generation efficiency gains.

SXM vs PCIe — the variant you rent matters

This is the most overlooked detail when renting. SXM5 boards offer higher bandwidth, full NVLink/NVSwitch fabric, and higher sustained clocks, which is why serious multi-GPU training almost always uses them. PCIe cards are cheaper to host and fine for single-GPU inference or smaller jobs, but their inter-GPU links are slower. If the listing above does not state the variant, treat that as a question to resolve before committing to a multi-GPU training run.

Rental cost, availability, and scarcity

In the cloud GPU cost spectrum, the H100 sits near the top — above A100 and far above consumer cards like the RTX 4090 — though it has been displaced from the absolute ceiling by newer Blackwell-class hardware. Because exact rates move constantly and differ by provider, region, commitment term, and variant, you should read live numbers from the comparison above rather than trust any fixed figure. A few qualitative points hold steady:

• On-demand vs spot: interruptible or spot H100 capacity can be substantially cheaper than on-demand, which suits checkpointed training and fault-tolerant batch inference, but is risky for long single-shot jobs without good checkpointing.
• Scarcity: H100 supply has historically been tight, so availability — not just price — varies by region and provider. The cheapest listing is worthless if the capacity is sold out in your region.
• Per-GPU vs full-node: many providers rent single H100s, but the best multi-GPU performance comes from full 8-GPU NVLink nodes; check whether you are getting fractional, single, or full-node access.
• Billing granularity: per-second or per-minute billing favors short fine-tunes and bursty inference; hourly minimums favor long runs. Match the model to your workload pattern.

Frequently asked questions

How much VRAM does a cloud H100 have?

The mainstream H100 carries 80 GB of high-bandwidth memory — HBM3 on the SXM5 variant and HBM2e on the PCIe variant. A later H100 NVL refresh increases per-card capacity. Always confirm the exact variant and memory figure in the listing above, since that determines the largest model and batch size you can run without offloading.

Is the H100 worth renting over an A100?

For transformer training and inference, usually yes: Hopper’s FP8 support and Transformer Engine can deliver materially higher throughput per GPU than Ampere’s A100, which often offsets the higher hourly rate on a cost-per-token or cost-per-step basis. For small or memory-light jobs that do not exploit FP8, a cheaper A100 or consumer card can be the better value.

Should I choose the SXM5 or PCIe H100?

Pick SXM5 for multi-GPU training and tightly coupled distributed jobs, because its NVLink/NVSwitch fabric and higher bandwidth scale far better across GPUs. PCIe is fine and often cheaper for single-GPU inference or smaller workloads. If the variant is not specified, ask before booking a multi-GPU run.

Can I save money with spot or interruptible H100 instances?

Yes, interruptible capacity is typically much cheaper than on-demand, and it works well for training with frequent checkpoints or for batch inference that tolerates restarts. Avoid it for long, un-checkpointed jobs where a mid-run eviction would waste hours of paid compute. Compare both pricing modes in the table above.