How to create a powerful and powerful program to answer the question by using the hot API, chroma, Google Gemini LLMS, along with Langchain Frames
In this lesson, we show how we can create a powerful and wise questioning for the search for more new API, Chroma, Google Gemini LLMS, along with Langchain. The pipe complies with the actual web search using the Advocate of documents, the preservation of the Chronic Vector document, as well as responding to the content of the Gemini model. These tools are included in Langchain components, such as runnablambalambelama, Chatprottette, Chat Prompttette, Conversations, and GoogleGrindweeedDings. Exceeds the Q & A by adding a hybrid restoration process to check the saved embeddown before requesting a new web Search. Refunded documents are firmly organized, summarized, and immediately insults a formal llm, monitoring the history of the combat user, user history, and score. The main functions such as developing engineers, sensitivity and analysis, and the revitalization of the vectrine makes this Pipeline ready for developed use facilities, background size, and wise agents.
!pip install -qU langchain-community tavily-python langchain-google-genai streamlit matplotlib pandas tiktoken chromadb langchain_core pydantic langchainWe also install and improve a complete set of libraries needed to build a high-quality AI search assistant. Including Tavily-Python, Chromadb), Langchain-Genai-Geni, data directions form a basic basis for building real-time, knowing QA recognition.
import os
import getpass
import pandas as pd
import matplotlib.pyplot as plt
import numpy as np
import json
import time
from typing import List, Dict, Any, Optional
from datetime import datetimeWe import keywords of Python's important libraries used throughout a letter of writing. It includes the usual libraries of the environment, secure input, tracking time, and data type (OS, digestion, time, type, datetate period). Additionally, it brings core data science such as Pandas, Matplotlib, and the Numpy of Data Management, Photos, and Compilations, and JVON Formal data.
if "TAVILY_API_KEY" not in os.environ:
os.environ["TAVILY_API_KEY"] = getpass.getpass("Enter Tavily API key: ")
if "GOOGLE_API_KEY" not in os.environ:
os.environ["GOOGLE_API_KEY"] = getpass.getpass("Enter Google API key: ")
import logging
logging.basicConfig(level=logging.INFO, format="%(asctime)s - %(name)s - %(levelname)s - %(message)s")
logger = logging.getLogger(__name__)We start safely of API keys with Tavily and Google Gemini by moving users only if they have not been set out in nature, confirming secure access and repeated access to external functions. It also fixes the limited setup of the Python logging module, which helps monitor the departure of suicides and error or error messages throughout the letter.
from langchain_community.retrievers import TavilySearchAPIRetriever
from langchain_community.vectorstores import Chroma
from langchain_core.documents import Document
from langchain_core.output_parsers import StrOutputParser, JsonOutputParser
from langchain_core.prompts import ChatPromptTemplate, SystemMessagePromptTemplate, HumanMessagePromptTemplate
from langchain_core.runnables import RunnablePassthrough, RunnableLambda
from langchain_google_genai import ChatGoogleGenerativeAI, GoogleGenerativeAIEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain.chains.summarize import load_summarize_chain
from langchain.memory import ConversationBufferMemoryWe import important nutrients from Langchain Ecosystem and its integration. It brings a real-time web search reference, the BROMA of the Vector storage, with ogleratatatatatives in the discussion and embedding models. Core Langchain modules like Chatprotttette, Runnabelambalambalambelambelambelambelambera, XsemoryMemory, and the issue of Parsers enables quick construction variables, responding to memory, and pipeline.
class SearchQueryError(Exception):
"""Exception raised for errors in the search query."""
pass
def format_docs(docs):
formatted_content = []
for i, doc in enumerate(docs):
metadata = doc.metadata
source = metadata.get('source', 'Unknown source')
title = metadata.get('title', 'Untitled')
score = metadata.get('score', 0)
formatted_content.append(
f"Document {i+1} [Score: {score:.2f}]:n"
f"Title: {title}n"
f"Source: {source}n"
f"Content: {doc.page_content}n"
)
return "nn".join(formatted_content)It describes two important components of search and scriptures. The Search class class creates a custom difference to manage wrong questions or fail to search kindly. Form_Docals Function processes a list of documents found by issuing a metadata such as the title, source, and a clean and readable string.
class SearchResultsParser:
def parse(self, text):
try:
if isinstance(text, str):
import re
import json
json_match = re.search(r'{.*}', text, re.DOTALL)
if json_match:
json_str = json_match.group(0)
return json.loads(json_str)
return {"answer": text, "sources": [], "confidence": 0.5}
elif hasattr(text, 'content'):
return {"answer": text.content, "sources": [], "confidence": 0.5}
else:
return {"answer": str(text), "sources": [], "confidence": 0.5}
except Exception as e:
logger.warning(f"Failed to parse JSON: {e}")
return {"answer": str(text), "sources": [], "confidence": 0.5}The SearchSultSultSultSSustrus section provides a firm way to remove the formal information from the llm Answers. Trying to combine JSON's rope – such as an outgoing model, returns to the obvious text format when Parsing fails. It treats the effects of the results and messaging, confirms low-fixing processing. In the event of errors, it is a warning and returns the fall answer containing a green response, empty sources, and automatic confidence score, to develop placement of programming.
class EnhancedTavilyRetriever:
def __init__(self, api_key=None, max_results=5, search_depth="advanced", include_domains=None, exclude_domains=None):
self.api_key = api_key
self.max_results = max_results
self.search_depth = search_depth
self.include_domains = include_domains or []
self.exclude_domains = exclude_domains or []
self.retriever = self._create_retriever()
self.previous_searches = []
def _create_retriever(self):
try:
return TavilySearchAPIRetriever(
api_key=self.api_key,
k=self.max_results,
search_depth=self.search_depth,
include_domains=self.include_domains,
exclude_domains=self.exclude_domains
)
except Exception as e:
logger.error(f"Failed to create Tavily retriever: {e}")
raise
def invoke(self, query, **kwargs):
if not query or not query.strip():
raise SearchQueryError("Empty search query")
try:
start_time = time.time()
results = self.retriever.invoke(query, **kwargs)
end_time = time.time()
search_record = {
"timestamp": datetime.now().isoformat(),
"query": query,
"num_results": len(results),
"response_time": end_time - start_time
}
self.previous_searches.append(search_record)
return results
except Exception as e:
logger.error(f"Search failed: {e}")
raise SearchQueryError(f"Failed to perform search: {str(e)}")
def get_search_history(self):
return self.previous_searchesThe developed phaseerverrievermantream is a custom wrapper around the tavilysebapiretriever, add a great variability, control, and tracking job searches. It supports advanced features such as reducing the depth of search, domain / releasing domain, and the calculation of prepared results. The Incokes Methodize a Web Search and track the Metadata of each Web metadata (time period, response, and statistical), the number of statistics have been kept.
class SearchCache:
def __init__(self):
self.embedding_function = GoogleGenerativeAIEmbeddings(model="models/embedding-001")
self.vector_store = None
self.text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
def add_documents(self, documents):
if not documents:
return
try:
if self.vector_store is None:
self.vector_store = Chroma.from_documents(
documents=documents,
embedding=self.embedding_function
)
else:
self.vector_store.add_documents(documents)
except Exception as e:
logger.error(f"Failed to add documents to cache: {e}")
def search(self, query, k=3):
if self.vector_store is None:
return []
try:
return self.vector_store.similarity_search(query, k=k)
except Exception as e:
logger.error(f"Vector search failed: {e}")
return []The Searchcache section uses the semantic layer using GooglegenativeInInivenativeInInives to convert documents into compact vector and keep them in the chro vector database. Add_doclews Mode begins or renewed the Vector Store, while searching is enabled the fastest saved text storage texts automatically preserved. This reduces the unwanted API calls and improves times to respond to repeated or related questions, which acts as a base of a hybrid memory in the pipeline assisting AI.
search_cache = SearchCache()
enhanced_retriever = EnhancedTavilyRetriever(max_results=5)
memory = ConversationBufferMemory(memory_key="chat_history", return_messages=True)
system_template = """You are a research assistant that provides accurate answers based on the search results provided.
Follow these guidelines:
1. Only use the context provided to answer the question
2. If the context doesn't contain the answer, say "I don't have sufficient information to answer this question."
3. Cite your sources by referencing the document numbers
4. Don't make up information
5. Keep the answer concise but complete
Context: {context}
Chat History: {chat_history}
"""
system_message = SystemMessagePromptTemplate.from_template(system_template)
human_template = "Question: {question}"
human_message = HumanMessagePromptTemplate.from_template(human_template)
prompt = ChatPromptTemplate.from_messages([system_message, human_message])
We start the main components of AI: Search Search Search, the improvement of the improvement, and negotferbsmomory maintenance to keep chat history repenting. It also describes the orderly immediately redoring using the TalchpromPTEMPTEMPLATE, directing the llm to make a research assistant. The Promptis is a strong accuracy of accurate accuracy, the use of the state, the quotation of the source, and short response, and responding to honest answers and bases.
def get_llm(model_name="gemini-2.0-flash-lite", temperature=0.2, response_mode="json"):
try:
return ChatGoogleGenerativeAI(
model=model_name,
temperature=temperature,
convert_system_message_to_human=True,
top_p=0.95,
top_k=40,
max_output_tokens=2048
)
except Exception as e:
logger.error(f"Failed to initialize LLM: {e}")
raise
output_parser = SearchResultsParser()
We define GET_LLM work, which starts the Modemo of Google Gemini with organized parameters such as model, temperature, and top_p settings, and higher tokens). It guarantees the stability of the error management of the model starting. Search Resultsparser and are designed to limit up to the llm's llm answers, allowing fixed processing of answers and Metadata.
def plot_search_metrics(search_history):
if not search_history:
print("No search history available")
return
df = pd.DataFrame(search_history)
plt.figure(figsize=(12, 6))
plt.subplot(1, 2, 1)
plt.plot(range(len(df)), df['response_time'], marker="o")
plt.title('Search Response Times')
plt.xlabel('Search Index')
plt.ylabel('Time (seconds)')
plt.grid(True)
plt.subplot(1, 2, 2)
plt.bar(range(len(df)), df['num_results'])
plt.title('Number of Results per Search')
plt.xlabel('Search Index')
plt.ylabel('Number of Results')
plt.grid(True)
plt.tight_layout()
plt.show()
PLOT_Search_metrics work appeared to work online tracks using matplotlib. It converts the search history to be data data and two companies: one indicating the response time by each search and indicating the amount of restored results. This helps to analyze the effectiveness of the program and quality of search later, to help improve retriev or point bottles in real world use.
def retrieve_with_fallback(query):
cached_results = search_cache.search(query)
if cached_results:
logger.info(f"Retrieved {len(cached_results)} documents from cache")
return cached_results
logger.info("No cache hit, performing web search")
search_results = enhanced_retriever.invoke(query)
search_cache.add_documents(search_results)
return search_results
def summarize_documents(documents, query):
llm = get_llm(temperature=0)
summarize_prompt = ChatPromptTemplate.from_template(
"""Create a concise summary of the following documents related to this query: {query}
{documents}
Provide a comprehensive summary that addresses the key points relevant to the query.
"""
)
chain = (
{"documents": lambda docs: format_docs(docs), "query": lambda _: query}
| summarize_prompt
| llm
| StrOutputParser()
)
return chain.invoke(documents)These two functions improve intelligence and efficiency of a helper. Returning Work – Refund Hybrid Retrieval Mechanism: first attempting to download the correct Chrodian cache documents and, if unsuccessful, fall back after new results. Currently, summarizi_docations The benefits of Gemini LLM production short summaries from the documents received, directed immediately to ensure that it is related to the question. Together, they empowered low, informal answers, practical conditions.
def advanced_chain(query_engine="enhanced", model="gemini-1.5-pro", include_history=True):
llm = get_llm(model_name=model)
if query_engine == "enhanced":
retriever = lambda query: retrieve_with_fallback(query)
else:
retriever = enhanced_retriever.invoke
def chain_with_history(input_dict):
query = input_dict["question"]
chat_history = memory.load_memory_variables({})["chat_history"] if include_history else []
docs = retriever(query)
context = format_docs(docs)
result = prompt.invoke({
"context": context,
"question": query,
"chat_history": chat_history
})
memory.save_context({"input": query}, {"output": result.content})
return llm.invoke(result)
return RunnableLambda(chain_with_history) | StrOutputParser()Advanced_Canced Work Explains the Work-related job, the full time answering users' questions using saved searches or actual time. Starts a specified Gemini model, selects a return strategy (temporary repairs or directive response), creates a pipeline attachment (if enabled), formats into context, and motivate the LLM using the program. The chain also penetrates the memory and returns the last response, combined with a clean text. The project empowers flexible assessment in models and retirement strategies while storing the agreement of the discussion.
qa_chain = advanced_chain()
def analyze_query(query):
llm = get_llm(temperature=0)
analysis_prompt = ChatPromptTemplate.from_template(
"""Analyze the following query and provide:
1. Main topic
2. Sentiment (positive, negative, neutral)
3. Key entities mentioned
4. Query type (factual, opinion, how-to, etc.)
Query: {query}
Return the analysis in JSON format with the following structure:
{{
"topic": "main topic",
"sentiment": "sentiment",
"entities": ["entity1", "entity2"],
"type": "query type"
}}
"""
)
chain = analysis_prompt | llm | output_parser
return chain.invoke({"query": query})
print("Advanced Tavily-Gemini Implementation")
print("="*50)
query = "what year was breath of the wild released and what was its reception?"
print(f"Query: {query}")We start the final components of a wise assistant. QA_CAIN is collenching college pupeline to process user questions using returns, memory, and Gemini-Based Reaction Generation. Analyzes of analyzes_query makes a Sound Systems for the question, issuing a large article, emotions, structures, and a type of question that uses the Gemini model. The question of example, about the respiratory respiration and acceptance of sanitation in the wild and receiving, show how the helper is caused and is prepared for full stack and full shelting. The printed article marks the beginning of effective execution.
try:
print("nSearching for answer...")
answer = qa_chain.invoke({"question": query})
print("nAnswer:")
print(answer)
print("nAnalyzing query...")
try:
query_analysis = analyze_query(query)
print("nQuery Analysis:")
print(json.dumps(query_analysis, indent=2))
except Exception as e:
print(f"Query analysis error (non-critical): {e}")
except Exception as e:
print(f"Error in search: {e}")
history = enhanced_retriever.get_search_history()
print("nSearch History:")
for i, h in enumerate(history):
print(f"{i+1}. Query: {h['query']} - Results: {h['num_results']} - Time: {h['response_time']:.2f}s")
print("nAdvanced search with domain filtering:")
specialized_retriever = EnhancedTavilyRetriever(
max_results=3,
search_depth="advanced",
include_domains=["nintendo.com", "zelda.com"],
exclude_domains=["reddit.com", "twitter.com"]
)
try:
specialized_results = specialized_retriever.invoke("breath of the wild sales")
print(f"Found {len(specialized_results)} specialized results")
summary = summarize_documents(specialized_results, "breath of the wild sales")
print("nSummary of specialized results:")
print(summary)
except Exception as e:
print(f"Error in specialized search: {e}")
print("nSearch Metrics:")
plot_search_metrics(history)
We show a complete functional pipe. It makes the search using Q_chain, displays the answer produced, and analyty of the question, topic, organizations, and kind. It also also printer is a history of support for each question, response time, and calculations from the result. Also, it works on a sized searches for the related sitendo sites, summarizing the results, and recognizes the performance of the search using Plot_search_metrics, which provides the full view of the actual use of use.
In conclusion, following this lesson provides users who use the complete system of skilled skills, and an incorrect RAG program that breaks the actual web intelligence AI. APILY SEARCH API allows users to pull new content and appropriate web content. Gemini LLM adds a stronger thinking of energy, and the Langchain Abstruction layer allows the orchesttions with no seamless orchestion between memory, embedding, and the results of the model. The implementation includes advanced features such as the specified sector, a question analysis (emotional anemia, and business strategies), and fall plans) using the Semantic Vector Cache formed by Chroma and GooglegerativeSheamsdings. Also, formal entry, error management, and analytics duties provide alarm systems and diagnostic.
Check the Colab booklet. All credit for this study goes to research for this project. Also, feel free to follow it Sane and don't forget to join ours 90k + ml subreddit.
Asphazzaq is a Markteach Media Inc. According to a View Business and Developer, Asifi is committed to integrating a good social intelligence. His latest attempt is launched by the launch of the chemistrylife plan for an intelligence, MarktechPost, a devastating intimate practice of a machine learning and deep learning issues that are clearly and easily understood. The platform is adhering to more than two million moon visits, indicating its popularity between the audience.
🚨 Build a GENAI you can trust them. ⭐️ Parliant is your open-sound engine of the Open-sound engine interacts controlled, compliant, and purpose AI – Star Parlont on Gitity! (Updated)
