Using Agents to Not Use Agents: How we built our Text-to-SQL Q & A system

Ask-a-Metric is a WhatsApp-based AI data analyst designed to facilitate SQL database queries in the development sector using Large Language Models (LLMs). Initially, a simple pipeline was created for rapid feedback, but it faced challenges in accuracy and scalability. An agentic approach was tested using CrewAI, which improved accuracy but resulted in high costs and slow response times. This led to the development of a pseudo-agent pipeline that combined the strengths of both approaches, achieving better performance while reducing costs and response times.

The system must accurately understand user questions, comprehend database structures, and ensure safety and security. The initial simple version allowed for quick user feedback but struggled with accuracy and prompt engineering. The agentic approach involved two agents that utilized various tools to execute tasks, but it was costly and slow. The insights gained from this approach informed the creation of the pseudo-agent pipeline, which optimized task execution by breaking down processes into smaller steps and adopting an object-oriented design.

The pseudo-agent pipeline now achieves an average response time of under 15 seconds and costs less than USD 0.02 per query. Future improvements will focus on enhancing accuracy, speed, and cost-effectiveness while adding features like multi-turn chat and multi-language support. The goal is to make data access more efficient and accessible for stakeholders in the social impact sector.