Meta has unveiled an early version of Scam Alert, an optional WhatsApp security feature that uses an on-device machine learning model to identify messages that may be part of a scam.
The company says message content remains on the user's phone during classification and is not automatically sent to WhatsApp, Meta, or third parties.
The feature is beginning a limited beta rollout, with Meta also opening parts of the system to scrutiny through its Bug Bounty program. The company published the technical architecture ahead of a broader release so security researchers can examine whether its privacy and anti-targeting protections work as described.
WhatsApp says the system is intended to address increasingly sophisticated fraud involving impersonation, social engineering, and AI-generated lures without weakening the service's end-to-end encryption. Once enabled, Scam Alert downloads a machine-learning model to the device and analyzes incoming messages from people not in the user's contacts.
Scam detection stays on the device
The model performs probabilistic classification based on conversational structure and linguistic signals learned from scam conversations previously reported by users.
If it detects a likely scam, WhatsApp displays a private warning inside the chat. The recipient can then block or report the sender, continue the conversation, or mark the chat as trusted so future messages in that conversation are no longer flagged.
Users who mark a warning as incorrect can choose to share the five most recently received messages with WhatsApp separately to help improve the system. That sharing is opt-in rather than automatic.
Meta says the model weights will also be published, allowing independent researchers to examine whether the classifier is limited to scam detection.
Privacy-preserving performance measurements
WhatsApp still needs statistics showing whether Scam Alert is producing useful warnings. Instead of collecting message contents, devices generate aggregate counts covering how often warnings appear and what users do afterward.
Those metrics are processed using a confidential federated analytics system built around confidential virtual machines (CVMs) that operate as Trusted Execution Environments.
Before sending data, the WhatsApp client verifies the code running inside the secure environment and checks privacy settings, including differential privacy parameters and minimum anonymity thresholds. If those checks fail, the client refuses to transmit the metrics.
Requests are also routed through an Oblivious HTTP (OHTTP) relay that removes the device's IP address. WhatsApp says only noisy, aggregated statistics covering sufficiently large groups eventually become accessible to the company.
Meta adds safeguards against targeted models
Meta is also attempting to prevent WhatsApp from silently delivering a modified classifier to a particular user.
Every model version, including experimental variants, must first be recorded on a third-party append-only transparency ledger. Model manifests contain SHA-256 hashes of the weights, tokenizers, and other assets and are signed by Cloudflare using Ed25519 keys before deployment.
The app verifies the signature, checks the transparency ledger, validates freshness, and compares downloaded files against their expected hashes. A model that fails these checks will not load.
Experiment assignments are made locally using randomness generated on the device, rather than being selected by Meta's servers.
Users will also be able to inspect Scam Alert activity through Account > Request Info > Scam Alert Activity, including which model version analyzed a message and whether a warning was generated.
The architecture provides stronger privacy guarantees than server-side message scanning, but those guarantees ultimately depend on researchers' ability to verify Meta's implementation.
Users participating in the beta should keep WhatsApp updated, treat unsolicited requests for money, credentials, or verification codes with caution, and continue reporting suspicious accounts even when Scam Alert does not issue a warning.
Leave a Reply