GPT-6 Astra Leads StarSkirmish Bench as Gemini 4 Argon Ties for Third Place
New benchmark results show OpenAI’s model outperforming Google and Anthropic in StarCraft bot coding, though human bots remain superior.
Key points
- GPT-6 Astra achieved the highest score on the StarSkirmish Bench v0.2, surpassing other flagship models in coding StarCraft bots.
- Gemini 4 Argon, Claude Opus 5.5, and Claude Fable 5.1 tied for third place with win rates between 37 and 39 percent.
- Human-written bots significantly outperformed all AI models, with the best AI winning only 7 percent of games against top human bots.
OpenAI’s GPT-6 Astra emerged as the top-performing large language model on the StarSkirmish Bench v0.2, a new benchmark evaluating how well AI systems can code StarCraft: Brood War bots. According to a report on Hacker News, the model scored higher than competitors from Google, Anthropic, and Meta in this specific coding challenge.
The benchmark requires each model to write a Protoss bot in C++ within one hour of wall clock time. The resulting bots are then evaluated against a roster of competitive human-written bots and other AI-generated bots. GPT-6 Astra’s performance marked a clear step ahead of GPT-6.1 Sol, which placed second in the rankings.
Benchmark Methodology and Results
StarSkirmish Bench v0.2 tests long-horizon reasoning and agentic coding by forcing models to incorporate learnings from practice games into their code. Each model ran in its vendor’s first-party CLI harness for one hour. Gemini 4 Argon ran at high reasoning effort, the highest available setting, while every other model ran at maximum settings.
Gemini 4 Argon, Claude Opus 5.5, and Claude Fable 5.1 were functionally tied behind the top two, all scoring between 37 and 39. These scores represent the average win rate of five runs against the roster. Muse Wonder Full placed seventh, replacing Meta’s Muse Spark 1.3 from the previous version of the benchmark.
Human Bots Remain Superior
Despite the AI advancements, the strongest human-written bots remain well ahead of every model tested. Against top human bots named Stardust, PurpleWave, and BananaBrain, the best-performing AI model, GPT-6 Astra, won only 7 percent of its games. This highlights the significant gap between current AI coding capabilities and expert human strategy in complex real-time environments.
The benchmark notes that one-hour reasoning limits are a factor, but matches were played on three two-player maps. Real human-written bot tournament conditions are described as significantly harder than the benchmark environment. The strongest predictor of a model’s score was how much it practiced against the seven strongest bots on the roster during its hour.
Context from Previous Reporting
As Threat Times reported on October 18, 2026, Google DeepMind’s Gemini 4 Argon is helping researchers develop subsequent versions through recursive self-improvement. Earlier reporting on October 16, 2026, noted that Gemini 4 Argon scored 97.5 percent on the ARC-AGI-3 benchmark using specialized testing, though performance dropped in standard environments.
The StarSkirmish results provide a different perspective, showing Gemini 4 Argon tied for third in a practical coding task. GPT-6.1 Sol offered uniquely good value, scoring second at just over a tenth of GPT-6 Astra’s cost per run. This cost-performance ratio places it in the most attractive quadrant of the analysis.
What should you do?
Security and technology professionals should monitor these benchmark results to understand the evolving capabilities of AI in coding and strategic reasoning. While these models are not yet matching human experts in complex gaming scenarios, their ability to write functional code in constrained timeframes is improving.
WATCH FOR:
- Updates to the StarSkirmish Bench as new models are released.
- Changes in AI pricing that may make high-reasoning models more accessible for development tasks.
- Research into how AI models incorporate feedback from practice sessions to improve code quality.
Background: Large language models
Large language models are a specific type of neural network trained on text. Generative AI is the broader umbrella term for any system that creates new content, including images and code. Not all generative AI uses large language models, but most modern text generators do.
What to do and how to stay safe: GPT-6 Astra
- Review internal policies on AI-generated code, especially for complex or security-sensitive applications.
- Monitor vendor announcements for changes in reasoning effort settings that may impact performance or cost.
- Evaluate the cost-effectiveness of different models for specific coding tasks, considering both performance and price.
- Stay informed about benchmark methodologies to accurately interpret claims about AI capabilities in coding and strategy.
Step-by-step guide: Prompt Injection Warning Signs: How to Spot AI Manipulation
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
Which AI model scored highest on the StarSkirmish Bench v0.2?
GPT-6 Astra scored the highest, placing ahead of GPT-6.1 Sol and other flagship models from Google, Anthropic, and Meta.
How did Gemini 4 Argon perform in the StarSkirmish benchmark?
Gemini 4 Argon tied for third place with Claude Opus 5.5 and Claude Fable 5.1, scoring between 37 and 39 percent.
Did any AI model beat human-written StarCraft bots?
No, the best AI model, GPT-6 Astra, won only 7 percent of games against top human-written bots like Stardust and PurpleWave.
