AI assistants, such as Otter.ai, are exposing negative remarks about colleagues

Virtual assistants like Otter.ai, if you don’t know the proper settings to select, will send a recording and transcript to all meeting attendees, even if a guest has left the meeting early. That means if you’re talking bad about your coworkers, discussing confidential information, or sharing shoddy business practices, the AI will pick up on it. And it will rat you out. 

That happened to researcher and engineer Alex Bilzerian recently. He had been on a Zoom meeting with a venture-capital firm and Otter.ai was used to record the call. After the meeting, it automatically emailed him the transcript, which included “hours of their private conversations afterward, where they discussed intimate, confidential details about their business,” Bilzerian wrote in an X post last week. Otter.ai was founded in 2016, and provides recording and transcription services that can be connected through Zoom or manually when in a virtual or in-person meeting.

The transcript showed that after Bilzerian had logged off, investors had discussed their firm’s “strategic failures and cooked metrics,” he told The Washington Post. While Bilzerian alerted the investors to the incident, he still decided to kill the deal after they had “profusely apologized.”

This is just one of many examples of how nascent AI technologies are misunderstood by users. In response to Bilzerian’s post on X, other users reported similar situations.

“Literally happened to my wife today with a grant meeting at work,” another user, Dean Julius wrote on X. “[The] whole meeting [was] recorded and annotated. Some folks stayed behind on the call to discuss the meeting privately. Kept recording. Sent it all out to everyone. Suuuuper awkward.”

Other users pointed out this could become a major issue in the health-care industry as virtual therapy and telehealth sessions become more prominent. 

“This is going to become a pretty terrible problem in health care, as you can imagine, regarding protected health information,” Danielle Kelvas, a physician and medical adviser for medical software company IT Medical, told Fortune. “Health care providers understandably have concerns about privacy. Whether this is an AI-scribe device or AI powered ultrasound device, for example, we as doctors are asking, where is this information going?”

Otter.ai, however, insists users can prevent these awkward or embarrassing incidents from happening. 

“Users have full control over their settings and we work hard to make Otter as intuitive as possible,” an Otter.ai spokesperson told Fortune. “Although notifications are built in, we also strongly recommend continuing to ask for consent when using Otter in meetings and conversations and indicate your use of Otter for full transparency.” The spokesperson also suggested visiting the company’s Help Center to review all settings and preferences.

The power of AI virtual assistants

As a means of increasing productivity and having records of important conversations, more businesses have begun implementing AI features into workflows. While it can undoubtedly cut down on the tedious practice of transcribing and sending notes out to stakeholders, AI still doesn’t have the same sentience as humans.

“AI poses a risk in revealing ‘work secrets’ due to its automated behaviours and lack of discretion,” Sukh Sohal, a senior consultant at data advisory Affinity Reply, told Fortune. “I’ve had clients express concerns over unintended information sharing. This can come about when organizations adopt AI tools without fully understanding their settings or implications, such as auto-transcription continuing after participants have left a meeting.”

Ultimately, though, humans are the ones who are enabling the tech.

“While AI is helping us work faster and smarter, we need to understand the tools we’re using,” Hannah Johnson, senior vice president of strategy at The Computing Technology Industry Association (CompTIA), told Fortune. “And we can’t forget that emotional intelligence and effective communication are just as vital. Technology may be evolving, but human skills remain the glue that holds it all together.”

Other AI assistants, like Microsoft’s Copilot, work similarly to Otter.ai, in that meetings can be recorded and transcribed. But in the case of Coiplot, there are some backstops: A user has to either be a part of the meeting or have the organizer approve the share of the recording or transcripts, a Microsoft spokesperson told Fortune. 

“In Teams meetings, all participants see a notification that the meeting is being recorded or transcribed,” the Microsoft spokesperson said in a statement. “Additionally, admins can enable a setting that requires meeting participants to explicitly agree to be recorded and transcribed. Until they provide explicit permission, their microphones and cameras cannot be turned on, and they will be unable to share content.” 

Still, these permissions don’t always address human naivety or error. To apply more guardrails to virtual assistant usage, Lars Nyman, chief marketing officer of AI infrastructure company CUDO Compute, said to think of your AI assistant as a junior executive assistant. 

It’s “useful, but not yet seasoned,” Nyman told Fortune. “Avoid auto-sending follow-ups; instead, review and approve them manually. Shape AI processes actively, maintaining firm control over what gets shared and when. The key is not to entrust AI with more autonomy than you’d give to a new hire fresh out of college at this stage.”