Member-only story
A step-by-step guide to building a chatbot based on your own documents with GPT
Chatting with ChatGPT is fun and informative — I’ve been chit-chatting with it for past time and exploring some new ideas to learn. But these are more casual use cases and the novelty can quickly wean off, especially when you realize that it can generate hallucinations.
How might we use it in a more productive way? With the recent release of the GPT 3.5 series API by OpenAI, we can do much more than just chit-chatting. One very productive use case for businesses and your personal use is QA (Question Answering) — you ask the bot in natural language about your own documents/data, and it can quickly answer you by retrieving info from the documents and generating a response [1]. You can use it for customer support, synthesizing user research, your personal knowledge management, and more!
In this article, I will explore how to build your own Q&A chatbot based on your own data, including why some approaches won’t work, and a step-by-step guide for building a document Q&A chatbot in an efficient way with llama-index and GPT API.
(If you only want to know how to build the Q&A chatbot, you can jump directly to the section “Building document Q&A chatbot step-by-step”)
Exploring different approaches
My day job is as a product manager — reading customer feedback and internal documents takes a big chunk of my life. When ChatGPT came out, I immediately thought of the idea of using it as an assistant to help me synthesize customer feedback or find related old product documents about the feature I’m working on.
I first thought of fine-tuning the GPT model with my own data to achieve the goal. But fine-tuning costs quite some money and requires a big dataset with examples. It’s also impossible to fine-tune every time when there is a change to the document. An even more crucial point is that fine-tuning simply CANNOT let the model “know” all the information in the documents, but rather it teaches the model a new skill. Therefore, for (multi-)document QA, fine-tuning is not the way to go.
The second approach that comes into my mind is prompt engineering by providing the context in the prompts. For example, instead of asking the question…
