Underdog Saluki 27B Outperforms Qwen3.8-27B in Tool Calling Tasks
Conway Research released a compressed 2-bit model that beats the original Qwen3.8-27B on tool calling benchmarks while reducing storage needs to 7.89 GB.
Key points
- Saluki 27B achieves higher tool calling scores than the full 54 GB Qwen3.8-27B model.
- The model fits in 7.89 GB and runs on stock llama.cpp software without custom forks.
- Performance drops in competition math and multi-step reasoning compared to the original.
Conway Research released Underdog Saluki 27B, a compressed version of the Qwen3.8-27B artificial intelligence model, under the Apache 2.0 license. According to MarkTechPost, this 2-bit GGUF file is designed for on-device use and prioritizes tool calling capabilities over other metrics.
The new model weighs only 7.89 GB, a significant reduction from the original Qwen3.8-27B’s 54 GB BF16 requirement. This compression allows developers to run a 27-billion-parameter agent model on standard hardware using stock llama.cpp. The release includes optional vision add-ons of 629 MB or 928 MB for extended functionality.
View the original post on X
Performance and Compression Details
Saluki 27B excels in parallel tool calling, scoring 42 against the full model’s 35 on BFCL v4 tasks. This represents a 120% retention rate for that specific skill. However, the model struggles with complex reasoning, scoring 79.2 on AIME 2025 compared to the original’s 96.7.
The compression process involves three layers. The base is the dense Qwen3.8-27B model with 64 layers and native support for 262,144 tokens. ISTA-DASLab’s GSQ-RCO method provided the second layer, creating an 8.4 GB file. Underdog applied a final pass to reach 7.89 GB, targeting tool calling performance.
Comparison with Other Builds
Saluki 27B retains 96% of the original model’s average performance across nine benchmarks. In the Underdog Bench, it scored 88, beating the full model’s 84. PrismML’s Bonsai 2 scored 70 on the same test but requires a custom llama.cpp fork.
Bonsai 2 reports higher retention in thinking-mode benchmarks and stronger math scores, including 95.00 on AIME25. However, it does not run on standard llama.cpp software. Saluki 27B offers compatibility with stock software at the cost of lower math and reasoning scores.
What should you do?
Developers interested in local agent capabilities should test Saluki 27B against their specific workloads. The model is available under Apache 2.0, allowing for unrestricted use and modification. Users should verify that their hardware can handle the 7.89 GB file with full GPU offload for optimal performance.
What to do and how to stay safe: Underdog Saluki 27B
- Evaluate your tool calling requirements to determine if Saluki 27B’s strengths align with your agent workflows.
- Test the model on your local hardware using stock llama.cpp to verify compatibility and performance.
- Compare Saluki 27B’s math and reasoning scores against your application’s needs to ensure acceptable accuracy.
- Monitor for updates from Conway Research regarding the full compression recipe and potential future optimizations.
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
Does Saluki 27B require special software to run?
No, Saluki 27B runs on stock llama.cpp and apps built on it, unlike some other compressed models that need custom forks.
How does Saluki 27B compare to the original Qwen3.8-27B in size?
Saluki 27B is 7.89 GB, while the original Qwen3.8-27B in BF16 format requires 54 GB of storage.
What are the main performance trade-offs of Saluki 27B?
The model beats the original in tool calling but sees significant drops in competition math and multi-step reasoning tasks.
