Creating ChatGPT based data analyst: first steps

Sightfull, a business data analytics platform, has integrated Generative AI (GenAI) to enhance user experience by providing personalized insights into their data. The initiative aims to address the challenge of delivering informative content about client data efficiently. The development process focused on three main areas: discovery, productivity, and explainability, ultimately selecting explainability as the primary focus. This led to the creation of a feature called "Data storytelling," which summarizes and highlights key metrics for users.

The implementation involved creating a microservice to interact with OpenAI's ChatGPT, processing user views and returning summarized insights. Initial attempts at prompt engineering yielded inconsistent results, prompting the team to refine their approach. Techniques such as few-shot prompting and chain-of-thought reasoning were employed to improve accuracy. However, response times were initially slow, leading to the decision to preprocess data for faster and more accurate summaries.

The final solution involved a structured prompt template that included code-generated summaries and relevant metadata, significantly improving response speed and accuracy. The team emphasized the importance of a quick feedback loop for iterative improvements. Looking ahead, Sightfull aims to enhance user interaction with data through conversational capabilities and real-time information retrieval, marking a significant evolution in their use of GenAI.