Mistral Large 4 Leads Non-Chinese Models in AI Benchmarks, Trails Chinese Peers
Independent tests show Mistral’s new Large 4 model outperforms Western rivals but lags behind several Chinese open models in overall intelligence scores.
Key points
- Artificial Analysis ranks Mistral’s Large 4 as the top model from outside the US and China.
- The model achieves an 81.7% score on CyberGym-E2E-AA, leading the field in that specific metric.
- Several Chinese models, including GLM-5.3 and MiMo-V2.6, score higher on the overall Intelligence Index.
Mistral AI launched its Large 4 model this week, claiming it pushes the frontier of open-weight performance. According to Tom's Hardware, independent benchmarking firm Artificial Analysis has ranked the new Large 4 as the most intelligent model available from outside the United States and China in its current index.
The mixture-of-experts model, trained on Nvidia Grace Blackwell GPUs, scored 38 on the Artificial Analysis Intelligence Index v4.3.2. While this places it at the top of its regional category, at least five Chinese open models achieved higher overall scores. These include DeepSeek V4.1 Flash, GLM-5.3-Flash, Kimi K3, GLM-5.3, and MiMo-V2.6-Pro, which scored between 39 and 46.
View the original post on X
Benchmark Performance and Costs
Mistral’s Large 4 achieved its strongest result in cybersecurity testing, scoring 81.7% on the CyberGym-E2E-AA measure. Artificial Analysis described this as the highest score of any model in that specific end-to-end cybersecurity test. In comparison, GLM-5.3-Flash scored 74% on the same metric, though Mistral notes that model’s post-training run is still ongoing.
Despite the strong cybersecurity score, Large 4 performed lower on other tests within the Cyber Index. It scored 16% in DeepsecBench-AA and 51% in CWE-Bench-AA. Artificial Analysis projects that once the model’s weights are released, it will rank among the top three open-weight models on the broader Cyber Index.
Cost remains a significant differentiator. At list price, Large 4 costs $1.13 per Artificial Analysis task, or $0.57 at the discounted launch price. This is substantially more expensive than MiMo-V2.6-Pro at $0.13 per task and GLM-5.3-Flash at $0.25 per task.
Model Availability and Specifications
Mistral states that Large 4 is not yet fully open, with weights scheduled for release by the end of the month. The model uses a mixture-of-experts architecture with 49B to 52B active parameters. In contrast, GLM-5.3-Flash has only 18B active parameters and 320B total parameters, making it roughly one-third the size of Large 4.
Mistral claims Large 4 is state-of-the-art among open models for enterprise workloads in cybersecurity, finance, and law. The company also highlights its 1.6B-parameter vision encoder, stating it can surpass frontier closed models in visual grounding tasks. Additionally, Mistral reports that Large 4 solves 93% of Cybench, a set of 40 challenges from security competitions.
What should you do?
Security professionals evaluating large language models for enterprise integration should review specific benchmark results relevant to their use cases, such as cybersecurity or legal analysis. Organizations should compare the total cost of ownership, including per-task pricing and compute requirements, against alternative models. Teams should monitor for the release of model weights to conduct independent validation of performance claims.
Background: Large language models
Large language models are a specific type of neural network trained on text. Generative AI is the broader umbrella term for any system that creates new content, including images and code. Not all generative AI uses large language models, but most modern text generators do.
What to do and how to stay safe: Mistral AI
- Evaluate model performance on specific benchmarks relevant to your industry, such as cybersecurity or finance, rather than relying solely on overall intelligence scores.
- Compare the operational costs of different models, including per-task pricing and infrastructure requirements, to determine the most efficient option for your workload.
- Monitor vendor announcements for the release of model weights to enable independent testing and validation of performance claims in your own environment.
- Review the specific capabilities of vision encoders and other specialized components if your use case requires multimodal processing or visual grounding.
Step-by-step guide: Prompt Injection Warning Signs: How to Spot AI Manipulation
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
How does Mistral Large 4 compare to Chinese AI models in benchmarks?
Mistral Large 4 scores lower than several Chinese open models on the Artificial Analysis Intelligence Index, which ranks it at 38 while models like MiMo-V2.6-Pro score up to 46.
What is the cost of using Mistral Large 4?
The model costs $1.13 per task at list price or $0.57 at the discounted launch price, which is higher than competitors like MiMo-V2.6-Pro at $0.13 per task.
When will the weights for Mistral Large 4 be released?
Mistral states that the weights for Large 4 are due for release by the end of the month, after which it will be fully open.
