vLLM Semantic Router Releases Decision 3.0 Multimodal Models
The team released five Apache-2.0 models that return decision probabilities for text and images without generating natural language output.
Key points
- Five model sizes range from 0.8 billion to 27 billion parameters.
- Models answer choice, yes/no, or score questions in a single call.
- The largest model leads internal evaluations for document extraction tasks.
The vLLM Semantic Router team released Decision 3.0, a family of open multimodal decision models. According to MarkTechPost, these five models are licensed under Apache-2.0 and range in size from 0.8 billion to 27 billion parameters. They are designed to process text, JSON, and images to answer specific typed questions without generating natural language text.
Decision 3.0 models return calibrated probabilities for each answer option in a single API call. Developers can use them to classify, route, or gate requests by setting thresholds on the scores. This approach avoids the latency and parsing complexity associated with prompting large language models and extracting answers from their generated text output.
Technical Specifications and Architecture
The release includes five distinct sizes: d3-lite at 0.85 billion parameters, d3-nano at 2.21 billion, d3-mini at 4.54 billion, d3-flash at 8.39 billion, and the largest d3 model at 26.09 billion. Each model is a fine-tune of a Qwen base architecture. The 27 billion parameter model is built on Qwen3.8-27B, while the smaller variants use Qwen3.5 checkpoints at 0.8 billion, 2 billion, 4 billion, and 9 billion parameters.
Every model in the Decision 3.0 family includes a vision encoder. The d3-lite model has a 0.10 billion parameter encoder, while d3-nano and d3-mini share a 0.33 billion parameter encoder. The d3-flash and d3 models utilize a larger 0.46 billion parameter vision component. Inputs can include multiple PNG, JPEG, or WebP images, with each image processed at up to 1.6 megapixels.
Performance and Evaluation Metrics
The research team reports that the 27 billion parameter d3 model leads both text and vision tables in its model card, based on internal evaluation. It achieved a score of 97.7 on the KIE document extraction benchmark using CORD and FUNSD datasets. The team notes that a 9 billion parameter model now outperforms the previous generation’s 27 billion parameter model in certain metrics.
On the Jev Decision Index 0.3.1 board, d3 scored 64.1 on text tasks, ahead of Perplexity Decider v1.1 at 62.8 and the original Jev system at 60.1. However, Torchcast Decision 27B posted a higher public-suite score of 65.1 compared to d3’s 64.9. On vision tasks, d3 scored 71.6 against Perplexity Decider v1.1’s 70.6. All performance figures are self-reported by the vLLM team.
What should you do?
Developers interested in using these models should review the installation requirements, which include transformers version 5.17.0 and the optional flash-linear-attention package. Users must set trust_remote_code=True when loading the models. That suggests teams should verify their infrastructure supports the required dependencies before integrating these decision models into production routing or guardrail systems.
Background: Large language models
Large language models are a specific type of neural network trained on text. Generative AI is the broader umbrella term for any system that creates new content, including images and code. Not all generative AI uses large language models, but most modern text generators do.
What to do and how to stay safe: vLLM
- Verify that your environment supports transformers version 5.17.0 and the required flash-linear-attention package.
- Ensure your deployment configuration allows trust_remote_code=True for loading the model weights.
- Test the models with your specific JSON and image payloads to validate latency and accuracy.
- Monitor the Jev Decision Index board for updated public benchmark comparisons.
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
Frequently asked questions
What license covers the Decision 3.0 models?
All five models in the Decision 3.0 family are released under the Apache-2.0 license.
Do these models generate natural language text?
No, the models return probabilities for choice, yes/no, or score questions without generating text.
What hardware was used for latency measurements?
Latency was measured on a single AMD Instinct MI325X GPU using BF16 weights.
