Amazon Fires at Nvidia's Software Moat With TorchNeuron

Amazon Web Services released a tool on Monday that moves AI applications onto its Trainium chips by changing a single line of configuration. The tool, called TorchNeuron, attacks the software advantage that has made Nvidia the default hardware choice for AI — and it does so inside the framework Amazon says 85% of AI apps already use.

TorchNeuron works with PyTorch, the free, open-source toolkit developers use to build, train and run AI models. According to Amazon, apps written with PyTorch can run on Trainium with one configuration change and no code rewrite. Facebook originally developed PyTorch; since 2022 the Linux Foundation has maintained it.

The target here is not a chip. It is CUDA, Nvidia's software infrastructure, which has supported PyTorch seamlessly for years. That integration has embedded Nvidia hardware into the toolkit essentially by default. TorchNeuron, adapted from PyTorch itself, means developers who know PyTorch already know how to use it.

Amazon is not the first to try this. PyTorch users can already port apps to AMD GPUs, or even Apple's chips, with a simple configuration change. The difference, Amazon argues, is what comes after the port.

What does Amazon claim startups gain?

Porting to Trainium gives AI startups access to Amazon's entire cloud, not just compute. AWS positions this as superior to the neoclouds — Coreweave, Nebius, Firmus — many of which run on Nvidia hardware.

"You're building on AWS. You're not betting on a single-purpose compute provider and scrambling to build everything else around it," Colin Brace, VP of Technology Software at AWS Annapurna Labs, the unit that makes Amazon's custom chips, told TechCrunch via email. "On AWS, the infrastructure to go from prototype to production is already there, allowing a founder to move fast and scale when they need to."

AWS says Trainium handles both training — teaching the model — and inference — putting it to work answering user questions. That single-platform design, plus tight integration with cloud infrastructure, will save startups money even if AWS does not offer the cheapest per-token price. That is a notable concession: Amazon is admitting it may not win on sticker price, and betting the full-workload economics close the gap.

"Trainium's value is price-performance plus the fact that it's one platform for both training and inference, which improves the economics across a full workload," Brace said. "Look at Deepgram — they moved to Trainium using native PyTorch, support and they're processing 70% more audio per dollar at production-grade accuracy."

Deepgram is an AI voice app provider and a competitor to ElevenLabs. A 70% gain in audio processed per dollar is the strongest number in Amazon's pitch — though it comes from a single customer Amazon itself named, not an independent benchmark.

Is Amazon going further than its own chips?

Yes, and this is the part with wider consequences. Amazon says it will contribute code upstream to the main PyTorch project to make the framework less automatically centered on Nvidia.

"Amazon's approach extends beyond its own hardware. The company is contributing changes upstream to PyTorch itself, working with the PyTorch community to remove code that ties the framework to specific chips. The goal is to make PyTorch more hardware-agnostic for the entire developer community, not just Trainium users," the company wrote in its announcement blog post.

The move is undoubtedly good for anyone selling hardware and cloud services that compete with Nvidia. But one of PyTorch's core points is to hide the difficulties of getting AI code to run on hardware. Removing Nvidia-specific code may deliver the choice and affordability Amazon promises — or it may simply deliver more configuration headaches to developers. Amazon's blog post does not address that trade-off.

How big a threat is this to Nvidia?

Bigger than a chip rivalry, if it works. Deeply uncoupling PyTorch from Nvidia would strike at the lock-in that underpins the GPU maker's dominance in AI software, not just its hardware sales.

"The real story is removing the lock-in that kept developers from having a real choice at all," Brace said.

The claim is bold; the execution is unproven. TorchNeuron shipped Monday, and the upstream PyTorch contributions depend on community acceptance, which Amazon does not control. If developers find the hardware-agnostic path as smooth as promised, Nvidia's software moat narrows. If they find friction instead, CUDA's default status holds.