Modal to deploy Nvidia Vera CPUs for AI agents, citing doubled throughput
AI infrastructure startup Modal plans to integrate Nvidia’s Vera CPUs for hosting agents, expanding its hardware mix beyond Intel and AMD processors.
Key points
- Modal plans to run its Sandboxes product on Nvidia’s Arm-based Vera CPU, marking its first shift away from x86 chips.
- In internal tests, the Vera chip handled up to twice as many containerized tasks as an Intel server, though results lack independent verification.
- The company did not specify a launch date for Vera-powered sandboxes, but noted that Nvidia delivered the first server last week.
Modal announced on Monday that it will deploy Nvidia’s Vera CPU for its AI agent sandboxes, marking the company’s first move away from x86 processors. According to The Next Web, the AI infrastructure startup plans to integrate the Arm-based chip to handle workloads that previously relied on Intel and AMD hardware. This strategic shift aims to improve throughput for isolated computing environments where AI agents execute code.
In internal benchmarks, Modal reported that the Vera chip handled up to twice as many tasks as an Intel server when packed with containers. The company stated that Vera maintained consistent task completion speeds while rival chips hit performance ceilings under similar density. These results have not been independently checked by third parties. Modal credited the performance gap to the Vera processor’s high memory bandwidth, which helps manage the variable demands of sandboxed AI agents.
Infrastructure and Performance Context
Modal’s Sandboxes product generates $300 million in annual revenue, the company said last week during a $750 million funding round that valued the business at $15.75 billion. Sandboxes are isolated computers where AI agents run code they write, a workload that is difficult to plan for due to unpredictable resource usage. An agent might sit idle while waiting for an AI model response, then spike to ten times more computing power during a software installation.
Because hundreds of sandboxes run side by side on each server, one customer’s sudden resource spike can slow down another’s work. To test the Vera chip, Modal built a benchmark of 50 coding tasks, including compiling software and searching files. When run one at a time, Vera was often faster than chips from Intel, AMD, and Amazon, but not always, and not by a significant margin. The performance advantage became clear only when the server was densely packed with containers.
Industry Landscape and Competition
Nvidia unveiled Vera in March as a CPU built specifically for AI agents, featuring 88 cores. The company has named Anthropic and OpenAI among its first users for the chip. Perplexity also stated it will run agent work on Vera, reporting that coding tasks were 1.5 times faster on the hardware. As Threat Times reported on October 19, Modal had already planned to integrate Nvidia’s Vera CPUs alongside Intel and AMD chips to expand its hardware mix for hosting agents.
Rivals are simultaneously building up their own sandbox offerings. Baseten acquired Blaxel in September to strengthen its position in the market. Modal did not say when the Vera-powered sandboxes would reach customers, but noted that Nvidia hand-delivered the company’s first Vera server to its San Francisco office last week. The company will need to adjust its infrastructure and the system it uses to build software images to support the Arm-based architecture.
What should you do?
Security and technology professionals should monitor their own infrastructure for compatibility issues when shifting from x86 to Arm-based processors. Teams running AI agents in sandboxes should evaluate how variable workloads impact shared resources and consider isolating high-spike tasks. Organizations should verify vendor performance claims through their own internal benchmarks before committing to new hardware architectures. Watch for official release dates and migration guides from providers like Modal as they update their platforms.
What to do and how to stay safe: Modal
- Audit current sandbox environments to identify workloads that cause significant resource spikes or contention between tenants.
- Review internal benchmarking processes to ensure they test performance under high-density container loads, not just single-task speeds.
- Plan for infrastructure changes required by Arm-based chips, including updates to software image building systems and dependency management.
- Monitor vendor announcements for specific deployment timelines and compatibility notes before migrating production AI agent workloads.
Step-by-step guide: Prompt Injection Warning Signs: How to Spot AI Manipulation
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
How does Nvidia's Vera CPU compare to Intel chips in Modal's tests?
In Modal's internal tests, the Vera CPU handled up to twice as many tasks as an Intel server when the server was packed with containers.
When will Modal launch Vera-powered sandboxes?
Modal did not specify a launch date for the Vera-powered sandboxes, though it received its first server last week.
What is Modal's Sandboxes product?
Sandboxes are isolated computers where AI agents run code they write, generating $300 million in annual revenue for Modal.
