Apple Watch Launches Beta Test for AI-Powered Conversation Rewind and Recap Features
Apple has introduced its AI-driven “Audio Intelligence” technology for the Apple Watch, which is currently available for beta testing among developers in the iOS 27.3 update. This innovative feature was first revealed during Apple’s Surprise and Shine hardware event in September. It consists of four voluntary AI functions designed for users of the Apple Watch Series 12, enabling them to identify significant sounds, recognize songs, review conversations, and take notes.
The sound recognition feature aims to assist individuals who are deaf or hard of hearing by alerting them to important sounds such as sirens, alarms, or doorbells through their watch. Meanwhile, the song recognition capability utilizes Apple’s Shazam technology to identify music playing in the background, providing details about the song and its artist.
However, the features that monitor human dialogue, namely Live Rewind and Siri Recaps, have raised potential concerns.
In light of public apprehension surrounding AI recording devices, Apple is making efforts to implement these features in a way that is beneficial to Apple Watch users while also maintaining the privacy of others.
Live Rewind allows users to revisit what someone just said, useful in situations where verbal instructions or a list of information is shared quickly. By double-pressing the digital crown on the watch, users can retrieve a text snippet of what was communicated in the last 15 seconds. They can choose to view this snippet briefly, after which it will disappear unless saved to the Siri app for future reference, which can be especially handy for remembering names or notes from a presentation.
When this feature is activated, a sound notification and visual cue signify to those nearby that something has been recorded.
If users opt not to save a snippet, it will be automatically erased after 30 seconds.
The Siri Recaps feature expands upon this by summarizing conversations.
Importantly, Apple clarifies that it does not identify speakers, retain raw audio, or create verbatim transcriptions. Instead, the feature generates a brief summary with bullet points highlighting key aspects of the conversation. These might include significant dates, deadlines, or financial details.
This feature could be beneficial in various scenarios such as work meetings, parent-teacher meetings, or training sessions, allowing individuals to recall essential information without the need for handwritten notes.
The recaps won’t activate when the watch detects music playback or when noise-cancellation features are in use.
Users are reminded to review their saved recaps, with the option to save them to the Notes app or export them for more personal use. If left unused, they will delete after a period of seven days.
Additionally, reminders to check for saved recaps appear every morning once the watch is unlocked.
Users have the flexibility to manually activate Siri Recaps or set a specific schedule for when the feature should operate, allowing them to disable it whenever necessary from the Control Center.
According to Apple, the system is designed to protect sensitive data, omitting identifiers like social security numbers or any information pertaining to self-harm.
The AI capabilities for the Apple Watch Series 12 and Apple Watch Ultra 4 leverage a combination of Apple’s hardware, software, and services. Key components include the S11 chip and built-in microphone, alongside Apple’s Private Cloud Compute infrastructure.
Apple’s AI technology employs a secure enclave to ensure audio data is processed privately, making it inaccessible to anyone, including the company itself. Audio prompts are securely sent to the associated iPhone, which condenses them into text. This condensed information, stripped of filler words, is then encrypted and sent back to the watch for user access, while any underlying data is promptly deleted from the cloud.
Apple confirms these features will not negatively impact battery life and will soon be available for public beta testing before wider release.
