Microsoft and NVIDIA warn enterprises assume more security risk with open-weight AI models
A joint paper from Microsoft and NVIDIA states that organizations running open-weight AI models must implement safety controls that providers typically handle for closed systems.
Key points
- Open-weight models shift security and safety control responsibilities from AI providers to the enterprises deploying them.
- Microsoft and NVIDIA advise using cryptographic signing to verify model provenance before deployment.
- The guidance categorizes risks into five main areas including cybersecurity, data protection, and bias.
Microsoft and NVIDIA have issued a joint paper warning that organizations deploying open-weight artificial intelligence models on their own infrastructure must supply safety and security controls that providers usually manage. According to Help Net Security, the companies emphasize that the burden of ensuring model safety shifts significantly to the enterprise when weights are published openly rather than kept closed by the vendor.
Open-weight models are defined as AI systems whose weights, which represent the values learned during training, are published for anyone to download, inspect, modify, and run. In contrast, closed-weight models remain with the building company and are typically accessed through online services. The joint guidance notes that while weight openness does not solely determine risk, the deployment context dictates how underlying risks materialize for the organization.
Context and Background
Michael Scovetta, Partner Security Assurance Manager at Microsoft, stated that open- and closed-weight models can share similar underlying risks, but the deployment method changes how those risks appear. The authors wrote that risk depends on the specific model, its intended use, and the safeguards surrounding it. This distinction is critical for security teams evaluating whether to adopt open-weight architectures for their internal operations.
The primary difference lies in who provides the necessary controls. Closed-weight models often include guardrails and security monitoring from the provider, whereas open-weight models are frequently distributed as bare files. The document states that safeguards run by the provider are no longer universally enforceable once the model is downloaded and run locally by an enterprise.
Who Is Affected
Enterprises that hold the model weights can alter or remove a model’s safety behavior, creating new vulnerabilities. The Microsoft and NVIDIA guidance sorts these risks into five categories: cybersecurity, data protection, content, human and AI interaction, and bias. These categories encompass 30 more specific risk types that organizations must evaluate before integrating open-weight models into their production environments.
An open-weight model may be modified, repackaged, or redistributed before an organization receives it, meaning safeguards added by the original developer might be missing. Fine-tuning, merging, or compressing a model can weaken built-in protections while leaving task performance intact, according to the joint paper. This creates a scenario where a model appears functional but lacks critical security layers
What to do and how to stay safe
- Sudden role changes in output signal a successful override of system instructions.
- Hidden metadata in images or documents often carries invisible injection payloads.
- Immediate isolation of the affected session prevents lateral movement within the AI workflow.
Prompt injections exploit the model’s ability to follow instructions by hiding malicious commands in input data. Isolate affected sessions immediately and implement external guardrails to filter inputs before they reach the model.
Step-by-step guide: Prompt Injection Warning Signs: How to Spot AI Manipulation
General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.
