Google Messages Adds Optional Cloud AI Scam Detection via Gemini

Google announced on October 14 that its Messages app will offer an optional Enhanced protection mode. This feature leverages cloud-based Gemini models to analyze conversation context from unknown senders, aiming to detect sophisticated scams that unfold over multiple messages rather than relying on a single malicious text.

The new system operates alongside existing on-device Scam Detection. While the on-device tool remains active, the Enhanced protection mode introduces a cloud analysis layer. According to TechRepublic, this shift moves certain data handling processes to Google’s cloud infrastructure, specifically using Private AI Compute to process the information while maintaining encryption standards.

How the Cloud Analysis Works

Google states that the cloud-based analysis targets complex message contexts to identify nuanced social engineering attempts. The processing occurs within Private AI Compute, an environment designed with remote attestation and encryption. Documentation indicates that data processed in this environment is inaccessible to Google employees, ensuring that the company does not have direct access to the raw content being analyzed for scam indicators.

Despite these privacy assurances, Google has not disclosed the precise nature of the data transmitted. The documentation does not specify whether the system sends complete messages, selected excerpts, derived signals, or sender identifiers. It also fails to define a retention period for the Messages data processed through this specific feature, leaving the duration of data storage unclear to users.

Privacy Implications and Background

This development marks a shift in Google’s approach to spam and scam detection within the Messages application. Previously, the company relied on machine-learning models running on the device to identify spam. TechRepublic notes that Google’s existing documentation also allows for the ephemeral processing of unencrypted message contents to detect spam, as well as uploading URLs to check for malicious links.

The distinction between on-device processing and cloud-based analysis is significant for privacy-conscious users. While on-device processing keeps data local, the new Enhanced protection mode requires sending data to the cloud. Although Google emphasizes the security of Private AI Compute, the lack of transparency regarding exactly which parts of a conversation are sent for analysis remains a point of uncertainty for security professionals.

What should you do?

Users interested in the enhanced scam protection should review the specific settings within the Google Messages app to enable or disable the Enhanced protection mode. Those concerned about cloud processing of their message data may choose to rely solely on the existing on-device Scam Detection feature. It is also advisable to monitor Google’s privacy documentation for updates regarding data retention policies and specific data fields transmitted to the cloud.

Background: Social engineering

Look for urgency, secrecy, and deviation from standard channels. Verify identity through a separate, known communication path before acting. Isolate suspicious interactions immediately to prevent lateral movement within your network environment.

Read the full guide: Social Engineering Warning Signs and How to Respond

What to do and how to stay safe: Google

• Review Google Messages settings to determine if Enhanced protection is enabled and adjust based on personal privacy preferences.
• Monitor official Google privacy documentation for clarifications on data retention and specific fields sent to the cloud.
• Continue using on-device Scam Detection as a baseline layer of protection against obvious spam and scams.
• Stay informed about updates to Private AI Compute documentation to understand how data is handled in the cloud environment.

Step-by-step guide: Prompt Injection Warning Signs: How to Spot AI Manipulation

General security guidance from the Threat Times newsroom. It is not confirmed advice from the organisations named in this story.

Frequently asked questions

Does Google Messages send my messages to the cloud for scam detection?

Yes, the optional Enhanced protection mode sends conversation context from unknown senders to cloud-based Gemini models for analysis.

Can Google read my messages when using Enhanced protection?

Google states that data is processed in Private AI Compute where it is encrypted and inaccessible to the company.

What specific data is sent to the cloud?

Google has not disclosed exactly which parts of a conversation, such as complete messages or excerpts, are sent for analysis.