Launch HN: Vocera (YC F24) – Testing and Observability for Voice AI
Vocera AI automates testing and monitoring of AI voice agents, addressing manual testing inefficiencies by simulating diverse personas and generating scenarios, providing real-time insights and detailed analytics for developers.
Vocera AI, founded by Shashij, Sidhant, and Tarush, is a platform designed to automate the testing and monitoring of AI voice agents. The founders, who previously worked on voice agents in healthcare, encountered significant challenges with manual testing, which was time-consuming and prone to errors. They faced difficulties in demonstrating reliability to customers, covering edge cases, simulating diverse conversations, and monitoring production calls effectively. To address these issues, Vocera automates the simulation of real personas and generates a variety of testing scenarios based on prompts and call scripts. This allows for comprehensive monitoring of production calls and provides real-time insights into the performance of voice agents. The platform not only automates evaluation but also generates scenarios and metrics automatically, saving developers time. Users can still define scenarios and metrics manually if desired. Vocera offers detailed analytics on agent performance, reducing the need for developers to listen to all call recordings. The founders invite feedback and discussions from those interested in the challenges of Voice AI or who are building voice agents.
- Vocera AI automates testing and monitoring of AI voice agents.
- The platform addresses challenges like manual testing inefficiencies and reliability demonstration.
- It simulates diverse personas and generates testing scenarios automatically.
- Developers receive real-time performance insights and detailed analytics.
- The founders seek feedback and discussions on Voice AI challenges.
Related
Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents
Hamming automates testing for LLM voice agents, enhancing efficiency and accuracy through realistic scenarios. Founders from Tesla and Anduril prioritize data privacy and plan future automation and optimization tools.
An Age of Hyperabundance
Laura Preston's article discusses her role as the contrarian speaker at the Project Voice conference, addressing ethical concerns of conversational AI, including its impact on vulnerable populations and human interaction.
Started with an AI agent, now doing a thing that won't scale
Vivek Agrawal's startup automates QA testing with AI, evolving from initial struggles to developing an MVP using GPT-4 Vision, emphasizing user engagement and non-scalable tasks for immediate value.
AI agents invade observability: snake oil or the future of SRE?
AI agents are emerging in observability and SRE, automating tasks and transforming monitoring. However, skepticism persists due to past AI failures, highlighting the need for benchmarks and addressing privacy concerns.
Show HN: With SpeakMyVoice, you're always part of the conversation
SpeakMyVoice is a text-to-speech app that helps individuals with vocal challenges express themselves using customizable voices, AI suggestions, and secure local data storage, enhancing communication and personalization.
- Many commenters express enthusiasm for the launch and recognize the value of automated testing for voice agents.
- There are questions about the demand for testing tools in a niche market where many developers use existing SDKs with built-in observability.
- Concerns arise about the potential confusion with the existing healthcare communication platform named Vocera, suggesting a need for a name change.
- Users inquire about customization options for scenario generation and handling sensitive data in production calls.
- Some commenters express interest in trying the product without going through traditional sales processes.
17 commentsGiven the size of the niche (developers building voice agents), do you find there's a lot of demand for testing and observability? From my anecdata, many of the voice AI agent builders are using SDKs and builder tools (Voiceflow, Vapi, Bland, Vocode, etc). Observability is usually already baked-in pattern with these SDKs (testing I'm not so sure of).
One conversation I had with a voice agent builder: "Our product is complex enough where external testing tools don't make sense. And we know when things are not working because we have close relationships with power users and companies." Whose problem are you solving?
Your tool looks very powerful, but might the broader opportunity be just to use your evals to roll out the best voice agents yourself?
One trend I’ve noticed is there’s a really heavy focus on pre-deploy tests which makes a lot of sense. But one big gap is the lack of ability to surface the scenarios that you don’t know are even occurring after deployment. These are the scenarios that human agents are great at handling and ai agents often fall flat: in a sales context that can have a direct impact on a customers bottom line.
I think rather than attempting to deploy a perfect agent, having a mechanism to surface issues would lend much more peace of mind when launching ai voice agents. Would be happy to chat more if additional context/real world examples would be helpful. Congratulations again on the launch!
Background: have worked on contact center unstructured analytics for several years.
I've been working with auto-generated content for the past 8 years (both algorithmic and LLM-based). One of the biggest challenges is detecting and preventing regressions after "improving" the prompt, algorithm, or model.
Just yesterday, I deployed a voice agent (OpenAI + Twilio), and it's clear that there are countless scenarios where the agent might not perform as expected. For example, if you ask the agent to speak in German, but your tool uses call names or returns data in English, the agent might suddenly switch to speaking English.
Overall, I believe voice agents will primarily be used and developed by SMEs, but they often lack the time or expertise to account for all these edge cases.
Btw: here is the number agent. sorry it's in German: +43732 350011
Great to see the focus on robust and exhaustive evaluations. With large-scale usage of products, everything that can go wrong usually does so such evals will go a long way!
How do you intend to grow the product?
Having some experience with the healthcare industry, seeing the name Vocera here is incredibly confusing.
Vocera is a very common communication platform and set of devices used in hospitals: https://vocera.stryker.com/s/product-hub/vocera-smartbadge These things are everywhere in healthcare already. If someone came to me and suggested using “Vocera” for a healthcare related tech thing, my mind would assume it’s the Stryker product. It’s that common.
So unfortunately I’d recommend a name change as a high priority. Dealing with healthcare tech is difficult enough, but using the same name as a very popular and established healthcare tech product is going to be an unnecessary obstacle in getting traction. Not to mention that Stryker’s Vocera division will have some things to say about this.
https://www.stryker.com/us/en/portfolios/medical-surgical-eq...
Related
Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents
Hamming automates testing for LLM voice agents, enhancing efficiency and accuracy through realistic scenarios. Founders from Tesla and Anduril prioritize data privacy and plan future automation and optimization tools.
An Age of Hyperabundance
Laura Preston's article discusses her role as the contrarian speaker at the Project Voice conference, addressing ethical concerns of conversational AI, including its impact on vulnerable populations and human interaction.
Started with an AI agent, now doing a thing that won't scale
Vivek Agrawal's startup automates QA testing with AI, evolving from initial struggles to developing an MVP using GPT-4 Vision, emphasizing user engagement and non-scalable tasks for immediate value.
AI agents invade observability: snake oil or the future of SRE?
AI agents are emerging in observability and SRE, automating tasks and transforming monitoring. However, skepticism persists due to past AI failures, highlighting the need for benchmarks and addressing privacy concerns.
Show HN: With SpeakMyVoice, you're always part of the conversation
SpeakMyVoice is a text-to-speech app that helps individuals with vocal challenges express themselves using customizable voices, AI suggestions, and secure local data storage, enhancing communication and personalization.