Interact with your documents using Retrieval-Augmented Generation (RAG)

Imagine having a personal chatbot that can answer questions directly from your documents—be they PDFs, research papers, or books. With Retrieval-Augmented Generation (RAG), this is not only possible but also straightforward to implement. In this tutorial, we'll learn how to create a chatbot that interacts with your documents, such as PDFs, using Retrieval-Augmented Generation (RAG). We will use Groq to find the meaning of the language model, Chroma like a vector store, too Gradio user equipment.

In the end, you'll have a chatbot that can answer questions directly from your documents, retain the context of your conversation, and give you short, accurate answers.

What is Retrieval-Augmented Generation (RAG)?

Retrieval-Augmented Generation (RAG) is an AI architecture that improves the capabilities of Large-Scale Language Models (LLMs) by integrating information retrieval systems. This system retrieves relevant data from external sources, providing LLM with the ground-based information to generate more accurate and contextually appropriate responses. By combining LLM generation capabilities with real-time data retrieval, RAG reduces inaccuracies and ensures up-to-date information on AI-generated content.

What is required

1. Python installation: Make sure Python 3.9+ is installed on your system.
2. Groq API Key: Sign up for a Groq account and generate an API key:
   • Visit the Groq Console.
   • Navigate to API keys and create a new key.
   • Copy your API key to use in the project.

Dependency: Install required libraries:

pip install langchain langchain-community langchain-groq gradio sentence-transformers PyPDF2 chromadb

These libraries will help with language processing, user interface creation, model integration, PDF management, and vector database management.

Downloading PDF Resource

In this tutorial, we will use a publicly available PDF that contains information about diseases, their symptoms, and treatments. Download the PDF and save it in your project directory (you are free to use any pdf).

Step 1: Extract Text to PDF

We will use PyPDF2 to extract text from PDF:

from PyPDF2 import PdfReader

def extract_text_from_pdf(pdf_path):
    reader = PdfReader(pdf_path)
    text = ""
    for page in reader.pages:
        text += page.extract_text()
    return text

pdf_path="diseases.pdf"  # Replace with your PDF path
pdf_text = extract_text_from_pdf(pdf_path)

Step 2: Break the Text into Fragments

Long documents are broken down into smaller, manageable chunks for processing.

from langchain.text_splitter import RecursiveCharacterTextSplitter

def split_text_into_chunks(text, chunk_size=2000, chunk_overlap=200):
    text_splitter = RecursiveCharacterTextSplitter(
        chunk_size=chunk_size,
        chunk_overlap=chunk_overlap
    )
    return text_splitter.split_text(text)

text_chunks = split_text_into_chunks(pdf_text)

Step 3: Create a Vector store with Chroma

We will embed text fragments using a pre-trained model and store them in Chroma vector database.

from langchain.embeddings import SentenceTransformerEmbeddings
from langchain.vectorstores import Chroma

embedding_model = SentenceTransformerEmbeddings(model_name="all-MiniLM-L6-v2")

vector_store = Chroma(
    collection_name="disease_info",
    embedding_function=embedding_model,
    persist_directory="./chroma_db"
)

vector_store.add_texts(texts=text_chunks)

Step 4: Run the Groq Language Model

To use Groq's language model, set your API key and get started ChatGroq for example.

import os
from langchain_groq import ChatGroq

os.environ["GROQ_API_KEY"] = 'your_groq_api_key_here'  # Replace with your API key

llm = ChatGroq(model="mixtral-8x7b-32768", temperature=0.1)

Step 5: Create a Conversation Retrieval Thread

About LangChain ConversationalRetrievalChainwe can connect the language model and the vector database.

from langchain.chains import ConversationalRetrievalChain

retrieval_chain = ConversationalRetrievalChain.from_llm(
    llm=llm,
    retriever=vector_store.as_retriever(topk=3),
    return_source_documents=True
)

Step 6: Implement Chatbot Logic

We describe a logical way to keep a conversation history and make replies.

conversation_history = []

def get_response(user_query):
    response = retrieval_chain({
        "question": user_query,
        "chat_history": conversation_history
    })
    conversation_history.append((user_query, response['answer']))
    return response['answer']

Step 7: Build the User Interface with Gradio

Finally, create a Gradio interface to interact with the chatbot.

import gradio as gr

def chat_interface(user_input, history):
    response = get_response(user_input)
    history.append((user_input, response))
    return history, history

with gr.Blocks() as demo:
    chatbot = gr.Chatbot()
    state = gr.State([])
    with gr.Row():
        user_input = gr.Textbox(show_label=False, placeholder="Enter your question...")
        submit_btn = gr.Button("Send")
    submit_btn.click(chat_interface, inputs=[user_input, state], outputs=[chatbot, state])

Using the Code

Save the text as application.py and run

python app.py

Hurray! You are done. The Gradio interface will be launched, allowing you to interact with your document.

But why are you standing here? You can go further by trying to build any of the following functionality in the chatbot.

1. Advanced Vector Store: Use other vector sites like Milvus or Pinecone for scalability.
2. Well-designed models: Experiment with fine-tuned Groq models for domain-specific accuracy.
3. Support for Multiple Scripts: Extend the system to handle more documents.
4. Better Content Management: Improve chat logic to better manage long chat histories.
5. Custom UI: Create a highly polished user interface with advanced styling and features.

Congratulations! You have successfully built a document-based chatbot using Groq and LangChain. Experiment with development and build something amazing! 🚀

Resources:

2. LangChain (
3. Groq (

Also, don't forget to follow us Twitter and join our Telephone station again LinkedIn Grup. Don't forget to join our 65k+ ML SubReddit.

🚨 Recommend Open Source Platform: Parlant is a framework that is changing the way AI agents make decisions in customer-facing situations. ^(Promoted)

Vineet Kumar is a consulting intern at MarktechPost. He is currently pursuing his BS at the Indian Institute of Technology (IIT), Kanpur. He is a machine learning enthusiast. He is interested in research and recent developments in Deep Learning, Computer Vision, and related fields.