Instinct’s Advanced AI Assistant Sparks Debate Over Privacy and Security Risks

The excitement surrounding Instinct, a new AI personal assistant still in a private testing phase, stems not only from its impressive features but also from notable privacy concerns. Many users describe the AI as “magical” and herald its launch as one of the most thrilling since OpenClaw. Nonetheless, some beta testers have expressed apprehensions regarding the security protocols and the contentious terms of service associated with the application.

Currently, Instinct remains in a private testing phase, meaning these concerns have not yet reached a broader audience.

Developed by a small team led by Noah Shinn, a former researcher at Sierra, Instinct operates under Spear Street Technology in San Francisco. The company is maintaining a low profile as indicated by industry sources.

The AI functions by integrating with user applications and devices, such as email, messaging platforms, calendars, and even accessing audio, location, and screen data. Users can communicate with the assistant via text or WhatsApp to execute different tasks, like scheduling appointments, organizing their inbox, and finding economical flight options.

While many testers have reported a performance exceeding their expectations, significant concerns linger over the company’s commitment to user privacy and security. This prompts a critical question: are the benefits of granting such extensive access to an AI assistant worth the potential risks?

Some early users have shared alarming details from the Terms of Service, which grant Instinct a sweeping “perpetual and irrevocable” license to “access, use, store, reproduce, transmit, display, publish, distribute, and modify” user content, even for AI training purposes. Moreover, the terms specify that Instinct can gather data from users’ devices, including screenshots and typing behavior.

Additionally, these terms empower Instinct to engage in “agreements” or transactions on behalf of users, making them binding without prior consent.

One user, Peter Yang, expressed frustration when he learned that Instinct would not delete his Gmail entries upon request, although the company later added a feature for users to remove external data.

Another early tester, Claire Vo, noted that the assistant continued to summarize her emails even after she disconnected its access, revealing that the emails were stored in plain text for later use.

Concerns about security also persist. A user was startled to find that Instinct could retrieve a sign-up code from their email to complete a reservation task. Subsequently, Alex Cohen, co-founder of Hello Patient, decided to delete his account once he realized Instinct could be easily phished.

Katie Jacobs Stanton, another user, stated that she lost trust in Instinct after it sent out an email on her behalf without her approval.

She summed up the ongoing dilemma surrounding personal AI tools by stating, “We are sacrificing privacy and control for advanced AI functionalities, often without understanding the implications.” She added that trust is pivotal, and while every successful action builds trust, any unauthorized action can severely undermine it.

Michael Mignano, founder of Anchor, noted that such products would reshape consumer security norms, warning that individuals might unknowingly provide sensitive credentials to third-party applications.

Interest in personal AI has surged since the introduction of OpenClaw, which gained traction due to its sophisticated features. Another assistant, Poke, has also recently been acquired by Cognition.

Despite the critiques, Instinct’s team has not responded to any user concerns, opting to maintain a low profile for now. Reports suggest that notable investors like Kleiner Perkins have shown interest in the startup, which is still evaluating its public rollout.

Multiple users have praised Instinct’s efficiency for travel arrangements, restaurant bookings, and CRM management, but the ongoing discussions about privacy and data access continue to raise eyebrows in the tech community.

Requests for further comments have gone unanswered as Instinct navigates this critical phase of user feedback and trust-building.