This repo is an fork of https://github.com/hwchase17/chat-langchain/. It is an implementation of a locally hosted chatbot specifically focused on question answering over Atlassian Confluence wiki content. The significant change is the inclusion of a new document loader capable of parsing Confluence wiki pages as HTML.
Built with LangChain and FastAPI.
The app leverages LangChain's streaming support and async API to update the page in real time for multiple users.
- Install dependencies:
pip install -r requirements.txt - Create an OpenAI API key and save the key to a new .env file in the project root folder in the format OPENAI_API_KEY=XXXX.
- Manaully export the Confluence wiki pages as HTML to a folder called 'confluence_docs' in the root directory of this project. Note you can bulk export a space.
- You can use other Document Loaders to load your own data into the vectorstore.
- Run the app:
make start- To enable tracing, make sure
langchain-serveris running locally and passtracing=Truetoget_chaininmain.py. You can find more documentation here.
- To enable tracing, make sure
- Open localhost:9000 in your browser.
Refer to https://github.com/hwchase17/langchain/blob/master/README.md for information on deployed versions of the original code base and more documentation.
There are two components: ingestion and question-answering.
Ingestion has the following steps:
- Manaully pull html from Confluence Wiki site
- Load html with LangChain's ReadTheDocs Loader
- Split documents with LangChain's TextSplitter
- Create a vectorstore of embeddings, using LangChain's vectorstore wrapper (with OpenAI's embeddings and FAISS vectorstore).
Question-Answering has the following steps, all handled by ChatVectorDBChain:
- Given the chat history and new user input, determine what a standalone question would be (using GPT-3).
- Given that standalone question, look up relevant documents from the vectorstore.
- Pass the standalone question and relevant documents to GPT-3 to generate a final answer.