Launch HN: Hamming (YC S24) – Automated Testing for Voice Agents
Hamming automates testing for LLM voice agents, enhancing efficiency and accuracy through realistic scenarios. Founders from Tesla and Anduril prioritize data privacy and plan future automation and optimization tools.
Hamming is a platform designed to automate the testing of LLM (Large Language Model) voice agents, allowing users to simulate interactions with difficult end users. The service aims to streamline the iterative process of developing voice agents, which often involves extensive manual testing to ensure accuracy and effectiveness, particularly in contexts like fast food drive-throughs where order accuracy is critical. Hamming's testing process includes creating diverse user personas, simulating real-world scenarios, scoring interactions based on predefined criteria, and tracking quality metrics in production. The founders, Sumanyu and Marius, draw on their experiences from Tesla and Anduril, emphasizing the importance of realistic simulations in testing autonomous systems. Currently, Hamming is manually onboarding users but plans to transition to a self-serve model soon. The platform prioritizes user data privacy and is working towards HIPAA compliance. Future developments include automating scenario generation and LLM judge creation, as well as exploring optimization tools for voice agents. Hamming invites feedback from users and developers in the voice technology space to enhance their offerings.
- Hamming automates testing for LLM voice agents to improve efficiency and accuracy.
- The platform focuses on creating realistic user scenarios and scoring interactions.
- Founders leverage experience from Tesla and Anduril to enhance testing methodologies.
- Hamming is transitioning to a self-serve model while ensuring data privacy.
- Future plans include automation of testing processes and optimization tools for voice agents.
Related
Self hosting a Copilot replacement: my personal experience
The author shares their experience self-hosting a GitHub Copilot replacement using local Large Language Models (LLMs). Results varied, with none matching Copilot's speed and accuracy. Despite challenges, the author plans to continue using Copilot.
Using Agents to Not Use Agents: How we built our Text-to-SQL Q & A system
Ask-a-Metric is a WhatsApp-based AI tool for SQL queries in the development sector, improving accuracy and efficiency through a pseudo-agent pipeline, achieving under 15 seconds response time and low costs.
Language model can listen while speaking
Recent advancements in speech language models led to the development of the listening-while-speaking language model (LSLM), enabling real-time interaction and robust performance in interactive speech dialogue systems.
My chatbot builder is over-engineered, and I love it
The article details the development of Fastmind, a scalable chatbot builder, emphasizing user feedback, the importance of familiar technologies, and the need to launch products sooner for iterative improvement.
Show HN: AI co-worker for system software development (Rust,C,C++,pdf)
H2LooP.AI accelerates system software development by converting unstructured data into structured datasets for model fine-tuning, offering tools for debugging and emphasizing user control over data and models.
- Some users express concerns about the efficiency of voice agents compared to non-voice interfaces, questioning the need for voice interactions.
- Others celebrate the potential of the product, highlighting its usefulness for real-time testing and quality control.
- There are worries about the impact of LLMs on low-income workers, with calls for AI to create new opportunities rather than just replace existing jobs.
- Several commenters inquire about open-source options and integration possibilities with existing systems.
- Some express doubt about the reliability of current voice agents, suggesting that the market may be premature for such testing services.
19 commentsIf you're going to develop AI voice agents to tackle pre-determined cases, why wouldn't you just develop a self-serve non-voice UI that's way more efficient? Why make your users navigate a nebulous conversation tree to fulfill a programmable task?
Personally when I realize I can only talk to a bot, I lose interest and end the call. If I wanted to do something routine, I wouldn't have called.
This will work with a https://www.pipecat.ai type system? Would love to wrap a continuous testing system with my bot.
I'm most excited to see well-done concepts in this space, though, as I hope it means we're fast-forwarding past this era to one in which we use AI to do new things for people and not just do old things more cheaply. There's undeniably value in the latter but I can't shake the feeling that the short-term effects are really going to sting for some low-income people who can only hope that the next wave of innovations will benefit them too.
If it did, wouldn't all the companies with production AI text interfaces be using similar techniques? Now being able to easily replay a conversation that was recorded with a real user seems like a huge value add.
It would essentially be another form of a behavioral interview. I wonder if this exists already, in some form?
would love to have something like this integrated as part of our open source stack.
How will this really check that the models are performing well vs just listening?
Selling shovels on a gold rush seems to have become the only one mantra here.
Related
Self hosting a Copilot replacement: my personal experience
The author shares their experience self-hosting a GitHub Copilot replacement using local Large Language Models (LLMs). Results varied, with none matching Copilot's speed and accuracy. Despite challenges, the author plans to continue using Copilot.
Using Agents to Not Use Agents: How we built our Text-to-SQL Q & A system
Ask-a-Metric is a WhatsApp-based AI tool for SQL queries in the development sector, improving accuracy and efficiency through a pseudo-agent pipeline, achieving under 15 seconds response time and low costs.
Language model can listen while speaking
Recent advancements in speech language models led to the development of the listening-while-speaking language model (LSLM), enabling real-time interaction and robust performance in interactive speech dialogue systems.
My chatbot builder is over-engineered, and I love it
The article details the development of Fastmind, a scalable chatbot builder, emphasizing user feedback, the importance of familiar technologies, and the need to launch products sooner for iterative improvement.
Show HN: AI co-worker for system software development (Rust,C,C++,pdf)
H2LooP.AI accelerates system software development by converting unstructured data into structured datasets for model fine-tuning, offering tools for debugging and emphasizing user control over data and models.