Elevenlabs Launches Realtime Speech-to-Speech AI Models

Elevenlabs has released two new speech-to-speech artificial intelligence models called Eleven Realtime, designed specifically for real-time conversational applications. According to a report by The Decoder, these models combine high accuracy with low latency to improve interactive voice experiences. The release targets developers seeking faster response times in AI-driven dialogue systems.

The larger v1 Max variant scores 98.8 percent on the Big Bench Audio benchmark while maintaining a 0.57-second response time. Elevenlabs states that no other model on the Artificial Analysis leaderboard currently offers both faster speed and higher accuracy. This performance level aims to reduce awkward pauses in automated customer support interactions and other complex tasks.

Model Variants and Performance

The smaller v1 Mini model is built for natural conversations at a lower cost than the Max variant. It includes capabilities to distinguish between a user finishing a sentence and simple vocalizations like "mm-hmm." This feature helps prevent the AI from interrupting users during natural speech patterns. The v1 Max targets more complex tasks where higher accuracy is required.

Both models are available through the ElevenAgents platform and an OpenAI-compatible Realtime API. This compatibility allows developers to integrate the technology into existing workflows without significant architectural changes. The API support facilitates easier deployment across various software environments that rely on standard OpenAI protocols.

Pricing and Availability

Pricing for the new models is structured per minute of usage plus additional token costs. The v1 Max variant costs more than the v1 Mini due to its higher performance capabilities. Full pricing details are available on the Elevenlabs pricing page. This tiered structure allows businesses to choose between cost efficiency and maximum accuracy based on their specific needs.

Elevenlabs claims that its new models lead the Artificial Analysis leaderboard in the combination of speed and accuracy. The company highlights this achievement to demonstrate the competitive advantage of its latest technology. No other model on the leaderboard is currently both faster and more accurate according to the vendor.

What should you do?

Security and technology professionals should evaluate the integration of real-time AI speech models into their customer support systems. Teams should test the latency and accuracy of these models against current benchmarks to determine suitability. Organizations should also review the pricing structures to understand long-term costs associated with token usage and minute-based billing.

What to do and how to stay safe: Elevenlabs

• Review the technical specifications of the Eleven Realtime models to ensure they meet your latency requirements.
• Test the v1 Mini model for cost-effective applications where extreme accuracy is less critical.
• Evaluate the OpenAI-compatible Realtime API integration with your existing development stack.
• Monitor the Artificial Analysis leaderboard for future updates on model performance comparisons.

Step-by-step guide: Prompt Injection Warning Signs: How to Spot AI Manipulation

General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.

Frequently asked questions

What are the names of the new models released by Elevenlabs?

The new models are called Eleven Realtime, with variants named v1 Max and v1 Mini.

How fast is the response time for the Eleven Realtime Max model?

The v1 Max model has a response time of 0.57 seconds.

What benchmark score does the Max variant achieve?

The Max variant scores 98.8 percent on the Big Bench Audio benchmark.