Google Gemini 4 Argon scores 97.5 percent on ARC-AGI-3 benchmark using specialized testing

Google DeepMind’s Gemini 4 Argon achieved a 97.5 percent score on the ARC-AGI-3 benchmark, making it the second AI lab after OpenAI to surpass the 95 percent threshold on this general problem-solving test, according to The Decoder.

The result was obtained using the Provider Adapter Harness, a testing setup that allows vendor-specific features such as retained reasoning and context compaction. This configuration cost approximately $4,800 to run. Only OpenAI’s GPT-6 Astra scored higher, reaching 99.9 percent on the same benchmark.

Benchmark Variability and Testing Methods

Performance results vary drastically depending on the testing environment used. In the Standard Harness, which restricts models to simple notes between rounds, Gemini 4 Argon’s score dropped to 49.8 percent at low reasoning levels. The cost for this standard test was $5,800.

This disparity highlights ongoing disputes about benchmark fairness. A July 2026 controversy involved OpenAI’s GPT-5.6 Sol outperforming Claude Opus 5 by using specific API settings. When tested in the standard environment without those features, GPT-5.6 Sol scored only 7.8 percent.

ARC Prize co-founder Francois Chollet subsequently allowed general, publicly available API settings in tests. This change aimed to ensure models are evaluated under comparable conditions. OpenAI argued that benchmarks must measure the technical environment as well as the model itself.

Performance Limitations and Comparisons

Despite the high ARC-AGI-3 score, Gemini 4 Argon trails rival models in math and software engineering tasks, according to Epoch AI. In the Standard Harness, the model failed to complete all 55 games due to API issues.

At low reasoning settings, Argon finished 42 games. At medium reasoning, it completed 46 games, scoring 46.3 percent. At the highest reasoning level, it only finished 19 games, so ARC Prize did not publish a final score for that tier.

The reported results include partial points from aborted runs to cover all 55 games. On older benchmark versions, Argon scored 91.3 percent on ARC-AGI-2 and 98.5 percent on ARC-AGI-1. These older tests used introductory pricing of $2 per million input tokens.

What should you do?

Security and technology professionals should monitor how AI vendors report benchmark results, as testing methods significantly impact outcomes. Organizations evaluating AI models should demand transparency regarding the specific harnesses and API settings used in performance claims.

What to do and how to stay safe: Google DeepMind

• Review the specific testing harness and API settings used in any AI model benchmark report before comparing performance.
• Evaluate AI models on domain-specific tasks such as math or software engineering, not just general problem-solving benchmarks.
• Monitor updates from ARC Prize and other benchmarking bodies regarding changes to testing standards and allowed configurations.
• Verify cost estimates for high-reasoning tasks, as prices can vary significantly between standard and provider-specific setups.

Step-by-step guide: Prompt Injection Warning Signs: How to Spot AI Manipulation

General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.

Frequently asked questions

What score did Gemini 4 Argon achieve on the ARC-AGI-3 benchmark?

It scored 97.5 percent using the Provider Adapter Harness, which allows vendor-specific features like retained reasoning.

How does Gemini 4 Argon perform in the Standard Harness?

Its score drops to 49.8 percent at low reasoning levels in the Standard Harness, which restricts API features.

Is Gemini 4 Argon the highest-scoring model on ARC-AGI-3?

No, OpenAI’s GPT-6 Astra scored higher at 99.9 percent, making Google the second lab to cross the 95 percent mark.