New AI attack can reconstruct typed text from keyboard sounds

Researchers have developed a new acoustic side-channel attack that can reconstruct text typed on a laptop by analyzing nothing more than the sound of keystrokes.

Unlike previous approaches, the method does not require attackers to first train the system on recordings from the victim's specific keyboard, making the attack more practical in real-world scenarios.

The related research was conducted by Atsunori Okada of Tohoku University alongside researchers from Kyoto University and the Nara Institute of Science and Technology.

Acoustic keyboard attacks are not new, but earlier techniques typically relied on collecting labeled recordings from the target keyboard beforehand or required specialized hardware and large amounts of captured typing. The new approach instead combines unsupervised audio analysis with AI language models to infer what someone is typing using only recordings of keystroke sounds.

The researchers describe three realistic attack scenarios, where an attacker could:

1. Place a smartphone near a victim in a public space
2. Use a contact microphone attached to a shared desk or wall to capture vibrations
3. Recover keystroke sounds transmitted as background noise during online meetings

The attack assumes only access to audio and does not require malware, privileged access, or prior knowledge of the victim's device.

The system works by first isolating individual keystrokes from an audio recording, grouping similar sounds together, and then using a Transformer-based language model to determine the most likely sequence of characters. The researchers also incorporated a feedback mechanism and an LLM to iteratively improve the decoded text, allowing the system to correct mistakes using the surrounding context rather than relying solely on the acoustic signal.

90-99% reconstruction

In laboratory tests using a smartphone placed next to a 2019 MacBook Pro, the researchers report reconstruction accuracy exceeding 99% after observing only 100 to 150 keystrokes. The technique also remained effective across several laptop models, including Dell, HP, Lenovo, and Apple systems, although some required more captured keystrokes to achieve similar accuracy.

The team also evaluated more challenging scenarios. When recording vibrations from approximately three meters away using a contact microphone placed on the same desk, or through a wall using a contact microphone attached to the wall, the system frequently achieved more than 90% reconstruction accuracy after around 150 to 250 observed keystrokes.

The researchers further tested the attack against audio transmitted over Google Meet, Microsoft Teams, and Zoom. With noise suppression disabled, keystroke sounds carried through meeting audio were often sufficient to reconstruct typed text with high accuracy after a few hundred keystrokes, although results varied depending on the conferencing platform and laptop model. The paper notes that some users may still be exposed because advanced noise cancellation features are not always available or enabled.

The researchers argue that acoustic side channels should be treated as a more serious privacy risk, particularly in shared workspaces and online meetings where microphones may unintentionally capture keyboard sounds.

To reduce the risk, users should avoid typing sensitive information while unmuted in online meetings, enable noise suppression when available, and be mindful of nearby recording devices in public spaces.