Hyperscaler vs Neocloud: an honest cost comparison

If you ask most enterprise buyers why they’d choose CoreWeave, Lambda, or Nebius over AWS, Google, or Azure for AI compute, the answer usually comes back as some version of “they’re cheaper.” Which is true. On raw GPU-hour pricing, good Neoclouds run 40-45% below hyperscaler list rates.¹ For a company burning $50 million a year on training compute, that’s over $20 million in annual savings — real money, worth the sales pitch.

But that answer is incomplete in a way that leads to expensive mistakes. The interesting question isn’t whether Neoclouds are cheaper. It’s why they’re cheaper, and what you’re giving up when you take the discount. Understanding this determines whether the trade is right for your situation — and there are situations where it very much isn’t.

Here’s what the numbers actually say.

Note: All GPU-hour pricing in this analysis reflects US and North America rates. Prices vary by 15-40% in other regions due to power costs, tax structures, and local supply-demand.

The paradox

Start with the counterintuitive fact: Neoclouds actually pay more than hyperscalers per unit for most infrastructure components. Not less.

A hyperscaler ordering half a million GPUs from Nvidia in a single year negotiates a discount that a Neocloud ordering ten thousand simply cannot match. On a $40,000 GB200,² the difference is $8,000-12,000 per GPU — hyperscalers get somewhere in the 20-30% off list range, Neoclouds get maybe 5-10% if they’re lucky and have long-standing relationships.

Networking is the same story. Google built Jupiter, its optical intra-datacenter network.³ Microsoft designed custom networking silicon for Azure. AWS designed its own Nitro-based Ethernet fabric. All three internalized what would otherwise be Nvidia InfiniBand purchases at Nvidia’s margins. Neoclouds have no such option — they pay Nvidia for InfiniBand switches, Spectrum-X, ConnectX network cards, and the optical transceivers that dominate the total networking spend. Per-port cost of hyperscaler in-house networking is roughly 40-60% cheaper than the equivalent Nvidia InfiniBand solution a Neocloud must buy.

Storage tells the same story. Google runs Colossus at commodity storage costs. Azure operates Blob Storage the same way. AWS runs S3 similarly. Neoclouds pay VAST Data, WEKA, or DDN at commercial rates — vendors that must earn their own margins on top of the hardware cost. The hyperscaler storage cost per usable petabyte is a fraction of what a commercial storage vendor charges the Neocloud.

And then there’s the cost of capital. Hyperscalers finance capital expenditure at 5-8% weighted average cost of capital,⁴ backed by investment-grade credit and generational balance sheets. Neoclouds finance at 12-18% — asset-backed debt, mezzanine, high-yield paper.⁵ Over the 4-5 year depreciable life of a GPU cluster, that spread alone accounts for 25-40% of the total cost of ownership.

Add it up: on raw hardware inputs plus financing, Neoclouds are structurally at a 30-50% disadvantage per GPU deployed. And yet they can offer their compute 40-45% cheaper than hyperscalers. How?

Where the hyperscaler premium actually goes

The gap is explained entirely by what happens above the physical infrastructure layer. A hyperscaler isn’t selling you a GPU cluster. It’s selling you a GPU cluster plus two hundred other services that Neoclouds don’t run, don’t want to run, and couldn’t afford to build.

Platform engineering. Every hyperscaler runs thousands of engineers building and maintaining a portfolio of infrastructure services: virtual private clouds, identity and access management, key management, secrets rotation, monitoring, tracing, logging, DDoS protection, global load balancing, edge networks, autoscaling infrastructure, backup and disaster recovery tooling. This is enormously expensive. Per GPU-hour, hyperscaler platform overhead runs roughly 6-10x higher than what Neoclouds carry — Neoclouds run a few hundred engineers total and offer maybe a dozen services above the compute layer.

Compliance. Hyperscalers spend hundreds of millions annually maintaining certifications — SOC 2, HIPAA, PCI-DSS, FedRAMP High, IRAP, C5, IL5, IL6, ISO 27001, and dozens of country-specific and industry-specific frameworks.⁶ Every region is separately audited. Every service is separately attested. Neoclouds typically maintain SOC 2 and sometimes ISO 27001. Serious government workloads, regulated healthcare, or financial services often require certifications that Neoclouds simply don’t have.

Sales and marketing. Google Cloud Next alone costs more than most Neoclouds’ entire annual sales budget. Hyperscalers maintain enterprise sales teams in every major market, account executives assigned to Fortune 500 accounts, developer relations programs, marketplaces, partner ecosystems. Neoclouds have small, focused sales teams selling to a narrower buyer.

Custom silicon and workload flexibility. Hyperscalers can put internal workloads on their own custom silicon — Google TPU, Microsoft Maia, AWS Trainium and Inferentia — bypassing Nvidia’s margins entirely for the workloads that fit. Neoclouds are 100% Nvidia-dependent and pay Nvidia’s tax on every GPU-hour they sell.

Margin structure and reliability infrastructure. Hyperscalers target higher gross margins on their compute business, which funds ongoing R&D, physical redundancy, and the 24×7 white-glove support infrastructure enterprise customers expect. Neoclouds accept lower gross margins, typically half of hyperscaler levels, and offer support through Slack channels during business hours.

Adding these up: per GPU-hour, a hyperscaler needs to charge substantially more than a Neocloud just to cover platform, compliance, sales, custom silicon investment, and margin structure. That’s more than the entire raw-hardware disadvantage a Neocloud carries. Net result: the Neocloud can price 40-45% below hyperscaler list and still turn a healthy profit.

The math, at a category level

For a Nvidia GB200 GPU-hour rented in North America, the approximate rental prices work out to roughly $5 to $6 for a Neocloud and $9 to $10 for a hyperscaler at list price.

The breakdown reveals where the money actually flows:

Notice the asymmetry: Neocloud raw cost is lower than hyperscaler raw cost (they pay less for platform, compliance, sales). But Neocloud unit hardware cost is higher (they pay more for GPUs, networking, storage, capital). The savings on the softer overhead outweigh the disadvantage on the hardware.

These aren’t published numbers — no vendor discloses them. They’re order-of-magnitude estimates drawn from public financial disclosures, industry benchmarks, and operator-side experience. The pattern is what matters, not the second decimal place.

When to choose which

The economics only matter in context. The right choice depends on what you’re actually trying to build.

Choose Neocloud when:

• The workload is training, fine-tuning, or batch inference — compute-intensive and platform-light.
• You have (or can hire) an ML platform team competent enough to run production inference on bare-metal infrastructure.
• Your latency tolerance allows single-region deployment or a small number of regions.
• Your compliance requirements are limited to SOC 2 and general enterprise security.
• You can commit to 1-3 year reserved capacity, which gets you additional 20-30% discounts.
• Your workload doesn’t need dozens of adjacent platform services (managed databases, serverless, complex identity federation, etc.).

Choose hyperscaler when:

• You’re building production consumer-facing systems that need multi-region reliability with tight SLAs.
• Your compliance requirements include HIPAA, PCI-DSS, FedRAMP, or the equivalent in another jurisdiction.
• You genuinely need the ecosystem — managed databases, serverless compute, event streaming, data warehouses, developer tooling — and would otherwise have to build or integrate them yourself.
• Your team is small and can’t sustain a platform engineering practice.
• Global consumer inference is central to the workload (you need edge CDN, global anycast, DDoS protection at the network edge).
• Total cost of ownership across compute plus platform plus operations favors bundling.

Use both when:

• You’re at meaningful scale ($10M+ annually on AI compute).
• Different workloads have different economics. Training is a Neocloud workload. Production customer-facing inference is often a hyperscaler workload.
• You want price leverage in vendor negotiations, which you don’t have as a single-vendor customer.

The largest AI labs and infrastructure companies use both, deliberately. Anthropic uses AWS and Google. OpenAI uses Microsoft’s Azure and CoreWeave. Meta and Microsoft operate their own infrastructure but also partner with Neoclouds for burst capacity. This isn’t hedging. It’s engineering to the actual shape of each workload.

What this means for your decision

If you’re a fund evaluating an investment in a Neocloud, the question isn’t whether their per-GPU-hour pricing is competitive — it will be. The question is whether they’ve built any structural advantage that a hyperscaler can’t replicate by cutting prices. Most Neoclouds haven’t. The good ones are betting on a combination of: (a) hyperscalers being unwilling to compete at Neocloud margins because it cannibalizes their premium services, (b) a specific vertical or geography where they have deep relationships, (c) contractual capacity commitments from a small number of large AI labs that lock in demand for 3+ years. If a Neocloud can’t credibly cite two of those three, its long-term economics are shakier than the pricing suggests.

If you’re an enterprise CIO deciding where to run production AI, the question is not just “which is cheaper per GPU-hour.” It’s “what does my total cost look like when I include the platform services I’ll need anyway, the compliance certifications my industry requires, the operations team I’d need to hire, and the risk of a smaller vendor’s business continuity over the 3-5 year commitment I’m about to make.” For most enterprise buyers, the honest answer favors hyperscalers for production workloads and Neoclouds for training and experimentation.

The 40% raw compute savings are real. They’re just not the whole story.

Aureak advises enterprises and institutional investors on AI infrastructure decisions: sizing, build vs buy, and vendor selection. Book a call.