Scope & boundaries
This guide covers leasing accelerated compute from the specialist clouds — what an hour of a given GPU costs, how contract length rather than the hourly rate decides the bill, and what you give up on each rung down the reliability ladder.
It does not cover the managed layer above them, where you pay per token and never see a GPU (AI Model Serving & Inference Platforms), general-purpose cloud infrastructure and the hyperscaler relationship around it (Cloud Infrastructure & IaaS), or governing cloud spend once it is running, across every service and account (Cloud Cost Management & FinOps).
Executive Summary
Nobody loses money on the hourly rate. They lose it on the hours they reserved and did not use, or on the discount they never qualified for because they would not commit.
GPU capacity is sold on a spectrum from “available this second, gone when someone outbids you” to “yours for a year whether you use it or not,” and the price moves by a large multiple across that range. The engineering question is which GPU. The procurement question — the one that decides the invoice — is how far along that spectrum your workload actually sits.
Most teams answer it wrong in the same direction. Training runs and steady inference have a floor that could be reserved, and get paid for at on-demand rates because reserving felt like a commitment nobody wanted to sign. Experimental work has no floor at all, and gets a reserved cluster because someone was worried about availability. Both mistakes are expensive, and both are visible in a week of utilization data.
Why GPU Capacity Became a Procurement Problem
This category exists because the hyperscalers could not supply it fast enough. A generation of purpose-built AI clouds grew up selling one thing — dense, well-networked accelerators — and they compete on terms as much as on silicon. The result is a market where the same GPU carries wildly different prices depending on how you agree to buy it, and where the terms are negotiable in ways cloud compute has not been for a decade.
The interruptibility question is the one most teams have not asked themselves. Vast.ai sells on-demand, interruptible and reserved capacity side by side, with prices set by supply and demand across more than 40 data centers; interruptible is dramatically cheaper and completely unsuitable for anything that cannot checkpoint. Whether your training code checkpoints well is therefore a procurement input, not an implementation detail, and it is worth knowing before the first quote rather than after.
The second force is that this market is unusually willing to sell you flexibility, and unusually good at charging for it. RunPod states that multi-GPU clusters launch in minutes with no commitments, scale to 64 GPUs, and are paid for by use; Denvr Dataworks bills on-demand instances by the minute. Both are genuine advantages for work that comes and goes. Both are also the most expensive way to buy an hour of compute, and a team that never revisits its default is paying a premium for an option it stopped exercising months ago.
Which type of GPU Cloud & AI Compute fits your organization?
Owning hardware is a real option at the top of this market and a bad one almost everywhere else. The arithmetic that matters is utilization: a cluster you keep busy most of the day beats rented capacity, and a cluster you keep busy eight hours a day does not, once power, cooling, staff and the depreciation curve of a two-year-old accelerator are counted.
The other thing worth settling early is who operates the cluster. Renting nodes and running a training program on them are different jobs, and the second one needs a platform engineering function — schedulers, image management, failure handling, the unglamorous work that decides whether expensive silicon is busy. Crusoe describes a fully managed cluster for deploying and scaling AI applications across GPU and CPU resources, which is a different purchase from raw capacity even where the accelerators are identical. Teams without that function on staff should price the managed option honestly rather than assuming they will grow one.
| Approach | What you are buying | What it costs you instead |
|---|---|---|
| Interruptible / spot | The cheapest capacity available, on the understanding it can be taken back | Checkpointing discipline, and runs that restart |
| On-demand | Capacity now, no commitment, released when you stop | The highest per-hour rate in the market |
| Reserved term | A discount in exchange for a horizon — weeks to a year | Paying for the capacity whether or not you use it |
| Dedicated cluster | Nodes and network that are yours alone, often bare metal | A contract, a minimum size, and a procurement cycle |
| Owned hardware | The asset, and every hour of it | Power, cooling, staff, and a depreciation schedule nobody enjoys |
How do you evaluate GPU Cloud & AI Compute?
Score these providers on the things that are hard to change after signing. The accelerator model is the easy part and the part every comparison leads with; interconnect, storage locality, region, and what happens during a failure are the parts that decide whether the cluster does useful work.
One criterion deserves separating because it is the one that gets discovered late: what the provider does when hardware fails mid-job. A multi-day training run on a large cluster will lose nodes; that is arithmetic, not misfortune. What differs is whether the platform detects it, whether your job resumes from a checkpoint or from zero, and whether anyone tells you before you notice the loss curve flatten. None of this appears on a rate card, all of it is testable in an afternoon, and it is worth more than a small difference in hourly price on any workload long enough to matter.
| Capability | What it does | Buyer translation |
|---|---|---|
| Interconnect | Network fabric between GPUs in a job | Decides whether multi-node training scales at all. Voltage Park lists on-demand capacity with 3200 Gbps InfiniBand; a provider that does not publish this figure is telling you something. |
| Commitment ladder | The range of terms on offer | Where the discount lives. Nebius states that reserving large-scale clusters for multiple months pays up to 35% less than on-demand rates. |
| Billing granularity | The smallest unit you are charged for | Denvr Dataworks bills on-demand instances by the minute; hourly rounding on bursty work is a real surcharge. |
| Cluster scale ceiling | The largest single configuration available | Constrains you later, not now. Voltage Park lists single configurations from 32 to more than 8,000 H100 GPUs. |
| Bare metal vs virtualized | Whether a hypervisor sits between you and the device | Denvr Dataworks offers both; bare metal matters for some frameworks and for nothing else. |
| Storage economics | Where checkpoints and datasets live, and what egress costs | Frequently a larger line than anyone modeled. RunPod lists storage from $0.05 per GB per month. |
| Region and residency | Where the silicon physically is | The constraint that removes most of the market when it applies, and none of it when it does not. |
Which vendors lead in GPU Cloud & AI Compute?
The camps below divide by commitment and by who operates the cluster, because that is what the contracts actually differ on. Within a camp the providers are more alike than their marketing suggests; across camps they are barely comparable.
Four vectors separate providers inside a camp, and price is not the interesting one. Term flexibility — whether the ladder runs from minutes to a year, or starts at six months. Interconnect transparency — whether the fabric and its bandwidth are published or have to be extracted. Failure handling — what happens to a running job when a node dies, and whether you find out from the platform or from your metrics. Exit terms — what a mid-term migration costs when a new accelerator generation ships, which in this market is roughly annually.
The honest broker's note is that this is a capital-intensive market in a capacity cycle, and the buyer bears the consequences of both. Providers here are financing hardware against forward contracts, which means their pricing reflects their cost of capital as much as their efficiency, and it moves. Availability moves too: capacity that is scarce this quarter is discounted next quarter and vice versa, on a cycle nobody in the market controls. Two practical implications. Prefer the shortest term that carries a real discount, because a long commitment in a repricing market is a bet you were not intending to place. And keep the workload portable — container images, no provider-specific orchestration, checkpoints in storage you can move — because the ability to leave is the only bargaining position that survives the next renegotiation.
| Vendor | Approach | Where it fits |
|---|---|---|
| DigitalOcean | General-purpose clouds | Teams whose data and services already sit in one general cloud |
| Crusoe | Managed AI platforms | Teams that want the cluster operated rather than handed over |
| Vast.ai | Marketplaces | Checkpointable work that can take the cheapest capacity available |
| RunPod | Per-second on-demand | Bursty workloads that must not pay for idle time |
| Lambda | Purpose-built AI clouds | The common case of a steady floor with occasional burst above it |
| Voltage Park | Reserved-cluster operators | Sustained training with a horizon you can commit to |
One representative of each approach is named here; the category runs to roughly two dozen providers, and several appear in more than one camp. The camps were written before the vendors were chosen, and no placement here is for sale. Any vendor in this category can speak for themselves in the Spotlight below.
How much should you budget for GPU Cloud & AI Compute?
There is no defensible single figure for what GPU capacity costs, because the same accelerator carries different prices by term, by interruptibility, by region and by how much of it you take. What is knowable is the shape of the discount and what each tier bills you for when the job is not running.
The costs that surprise people sit around the compute rather than in it. Storage is the usual one: checkpoints accumulate, nobody deletes them, and the monthly line grows quietly — RunPod lists storage from $0.05 per GB per month, which is small until it is measured in petabytes. Egress is the next, and it is the one that makes leaving expensive rather than merely inconvenient. Then there is the management plane: CPU nodes for orchestration and data processing are real machines that bill like real machines, and Crusoe describes exactly that alongside its GPU offering. None of these appear in a per-GPU-hour comparison, and together they routinely add a meaningful fraction to the invoice.
| Basis | You are charged for | Idle cost | Where it goes wrong |
|---|---|---|---|
| Interruptible | Capacity until someone else wants it more | None | Anything that cannot checkpoint and resume |
| On-demand hourly | Wall-clock time the instance exists | None once released | Steady workloads, which never earn the discount |
| On-demand per-minute | The same, at finer granularity | None once released | Nothing — it is strictly better than hourly rounding |
| Reserved, multi-month | The term, at a discount to on-demand | Full rate | A forecast that did not hold |
| Reserved, annual | The year, at the deepest discount | Full rate | A model generation change mid-term |
| Dedicated cluster | A minimum configuration for a minimum term | Full rate | Sizing to peak instead of to floor |
| Storage | Volume held, per month | Full rate | Checkpoints nobody deletes |
Every figure above is a published rate read from the provider's own page on the date in the sources note. None of it is an estimate of what you will pay.
How long does implementation take for GPU Cloud & AI Compute?
Sequence this so the commitment comes last and the measurement comes first. Every provider will quote against the numbers you give them, and most teams give them a peak.
One thing is worth doing before week one: establish whether a data-residency or in-account constraint applies. It removes most of the market when it does, and it is far cheaper to know that before assembling a shortlist than after running a pilot with a provider you cannot use.
Run on-demand, everywhere, and instrument utilization by hour. The output is one number: the GPU count you are above every day. That is what you reserve, and nothing else.
Kill a node mid-run on each shortlisted provider. What you are buying is what happens next — whether the job resumes, how long it takes, and whether anyone tells you. This is invisible in a benchmark and decisive in production.
Reserve the measured floor on the shortest term that carries a real discount, and burst above it on-demand. Resist the deeper annual discount until you have one quarter of data, because the discount is only a discount if the forecast holds.
Utilization moves, accelerator generations turn over, and the market reprices. A reservation that was right at signing is frequently wrong two quarters later, and nobody notices unless someone is looking.
Renting compute is infrastructure procurement: the provider executes what you send it and decides nothing about anyone. The obligations that attach follow the data — which jurisdiction the silicon sits in, what is retained on shared storage, and who can reach the tenancy. Those are contract and architecture questions rather than model-governance ones, and they sit with you rather than with the provider.
Classified under the EU AI Act's risk tiers, as they apply to a supplier of compute rather than of an AI system placed on the market
What should you ask vendors about GPU Cloud & AI Compute?
Take these to the quote. The ones that produce a pause are the ones worth the meeting.
-
Can the workload checkpoint and resume?Yes Interruptible capacity is on the table, and it is dramatically cheaper. Price it before anything else.No On-demand or reserved only. Rule out the interruptible tiers now rather than discovering them in a failed run.
-
Is there a GPU count you are above every single day?Yes Reserve exactly that, and burst above it. This is the single largest saving available.No Stay on-demand. A reservation against work that is not there is a donation.
-
Does the job span more than one node?Yes Interconnect is now a first-order criterion, and a provider that will not publish it should be asked directly.No Compare on price and region, and stop overthinking the fabric.