Apple Researchers Publish Details on Minions, a Distributed GPU Cloud Using Idle Macs
Apple researchers describe an internal system that aggregates idle Apple Silicon GPUs to accelerate AI inference workloads and reduce queue times.
Key points
- The Minions system distributes AI inference tasks to idle Macs on the network, reducing wait times from hours to minutes.
- The architecture uses a pull-based model where Macs retrieve subtasks from a central server and return results to Amazon S3.
- Each Mac runs entire models locally using its unified memory, rather than splitting a single model across multiple devices.
Apple researchers have published a paper describing Minions, an internal distributed virtual GPU cloud that leverages idle Apple Silicon devices to execute machine learning inference workloads. The system addresses limitations in centralized GPU infrastructure, such as high costs and long wait times, by aggregating underutilized hardware across the company’s network. This approach allows teams to utilize existing Macs instead of procuring additional dedicated GPU servers.
According to the paper published on Apple’s Machine Learning Research site and discussed on Hacker News, the system significantly reduces processing delays. Jobs that previously faced company GPU queues lasting more than ten hours now see wait times drop to mere minutes. The platform specifically handles AI inference tasks, including embeddings with open-source Hugging Face transformer models and text generation from large language models via Ollama.
System Architecture and Operation
The Minions infrastructure consists of three main components: a Minion Server, a Minion Executor, and a Minion Watchdog. The server splits submitted tasks into smaller subtasks and stores them in a PostgreSQL database. There is no central scheduler pushing work to devices; instead, the system relies on a pull-based execution model.
Each participating Mac runs a Minion Executor that requests the next available subtask over REST APIs. The device reads input data from Amazon S3, retrieves the necessary model if it is not already cached, and processes the subtask on its local GPU. Once complete, the Mac returns the result to Amazon S3. This design ensures efficient resource utilization without requiring constant central oversight.
Fault Tolerance and Data Concerns
The Minion Watchdog monitors the health of participating devices by tracking heartbeats sent every five minutes. If a Mac goes to sleep, disconnects from the network, powers off, or if a user stops the executor to regain control, unfinished subtasks are returned to the queue. This fault tolerance mechanism prevents data loss and ensures that dropped tasks are reassigned to other available devices.
Users can prioritize specific jobs using dynamic task binding, allowing them to prefer tasks from certain teams or projects. Alternatively, the system defaults to a first-in, first-out scheme. However, researchers noted a common concern among teams regarding their data running on devices outside their immediate group, highlighting potential privacy or security considerations for internal deployments.
What should you do?
Security professionals should monitor for unauthorized distributed computing software on corporate networks, as idle devices can be repurposed without explicit consent. Organizations should review their data governance policies to ensure that sensitive information is not processed on shared or unmanaged endpoints. IT teams should verify that all distributed workloads are authorized and that devices are properly segmented to prevent lateral movement.
Background: Large language models
Large language models are a specific type of neural network trained on text. Generative AI is the broader umbrella term for any system that creates new content, including images and code. Not all generative AI uses large language models, but most modern text generators do.
What to do and how to stay safe: Apple
- Audit endpoint logs for unexpected network connections to cloud storage services like Amazon S3, which may indicate unauthorized data exfiltration or distributed computing activity.
- Implement strict access controls and data encryption policies for any devices participating in shared or distributed workloads to protect sensitive information.
- Monitor device resource usage for abnormal GPU activity during off-hours, which could signal that a machine is being used for background inference tasks without user knowledge.
- Review and enforce policies regarding the installation of third-party AI tools like Ollama on corporate devices to ensure they align with security standards.
Step-by-step guide: Cloud Misconfiguration Response: Containment, Recovery and Prevention Steps
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
How does the Minions system handle failed tasks?
The Minion Watchdog tracks device heartbeats every five minutes; if a device disconnects or fails, unfinished subtasks are returned to the queue for reassignment.
Does Minions split a single AI model across multiple Macs?
No, each Mac runs the entire model on its own hardware using unified memory, rather than distributing parts of a single model across multiple devices.
What types of AI workloads does Minions support?
The system handles embeddings using open-source Hugging Face transformer models and text generation from large language models through Ollama.
