How to Create Advanced Advanced Scraped Book Google Gemini Ai-Powered Data Extraction
In this lesson, we visit it to create an advanced web cleaning tool that will improve BrightdataThe powerful powerful proxy network next to the Google Gemini API for the restoration of intelligent data. You will see how you plan your Python project, enter and submit the required libraries, and you intend to align a clean, effective brightfatascrator components. Whether you look at the Amazon product pages, Bestseller list, or Linkedin Profiles, scrapers showing how to prepare for puppets, treating errors kindly, and returned the formal JSSon results. AGENT AGENT AGENT ACER OUGER OPTIONAL
!pip install langchain-brightdata langchain-google-genai langgraph langchain-core google-generativeaiWe include all the important libraries required for the lesson: Langchain-Brightdata Web scraping, Langchain-Ganai Google-Generatatue of Google Genai, and Langchain-Core of Langchain.
import os
import json
from typing import Dict, Any, Optional
from langchain_brightdata import BrightDataWebScraperAPI
from langchain_google_genai import ChatGoogleGenerativeAI
from langgraph.prebuilt import create_react_agentThis is enabled to prepare your environment and basic performance: OS system and Jonon Hanto Panduss and the type of data, while typing it gives a strategic plan. You bring the BrightDatwatwatatwatarda Scraping, ChatgoogleGengenativeia for the Granditiser for Gemini LLM, and create a_a_ent_Eent to Archestrate these parts of the styna.
class BrightDataScraper:
"""Enhanced web scraper using BrightData API"""
def __init__(self, api_key: str, google_api_key: Optional[str] = None):
"""Initialize scraper with API keys"""
self.api_key = api_key
self.scraper = BrightDataWebScraperAPI(bright_data_api_key=api_key)
if google_api_key:
self.llm = ChatGoogleGenerativeAI(
model="gemini-2.0-flash",
google_api_key=google_api_key
)
self.agent = create_react_agent(self.llm, [self.scraper])
def scrape_amazon_product(self, url: str, zipcode: str = "10001") -> Dict[str, Any]:
"""Scrape Amazon product data"""
try:
results = self.scraper.invoke({
"url": url,
"dataset_type": "amazon_product",
"zipcode": zipcode
})
return {"success": True, "data": results}
except Exception as e:
return {"success": False, "error": str(e)}
def scrape_amazon_bestsellers(self, region: str = "in") -> Dict[str, Any]:
"""Scrape Amazon bestsellers"""
try:
url = f"
results = self.scraper.invoke({
"url": url,
"dataset_type": "amazon_product"
})
return {"success": True, "data": results}
except Exception as e:
return {"success": False, "error": str(e)}
def scrape_linkedin_profile(self, url: str) -> Dict[str, Any]:
"""Scrape LinkedIn profile data"""
try:
results = self.scraper.invoke({
"url": url,
"dataset_type": "linkedin_person_profile"
})
return {"success": True, "data": results}
except Exception as e:
return {"success": False, "error": str(e)}
def run_agent_query(self, query: str) -> None:
"""Run AI agent with natural language query"""
if not hasattr(self, 'agent'):
print("Error: Google API key required for agent functionality")
return
try:
for step in self.agent.stream(
{"messages": query},
stream_mode="values"
):
step["messages"][-1].pretty_print()
except Exception as e:
print(f"Agent error: {e}")
def print_results(self, results: Dict[str, Any], title: str = "Results") -> None:
"""Pretty print results"""
print(f"n{'='*50}")
print(f"{title}")
print(f"{'='*50}")
if results["success"]:
print(json.dumps(results["data"], indent=2, ensure_ascii=False))
else:
print(f"Error: {results['error']}")
print()The BrightDatascraper is united all that brightdata Web-scraping logic and gemini-powered intelligence of one, practical. Its methods allow you to easily provide the product details of the Bestseller, with LinkedIn profiles, the APS telephone repairs, and the Figent-Fingling “.
def main():
"""Main execution function"""
BRIGHT_DATA_API_KEY = "Use Your Own API Key"
GOOGLE_API_KEY = "Use Your Own API Key"
scraper = BrightDataScraper(BRIGHT_DATA_API_KEY, GOOGLE_API_KEY)
print("🛍️ Scraping Amazon India Bestsellers...")
bestsellers = scraper.scrape_amazon_bestsellers("in")
scraper.print_results(bestsellers, "Amazon India Bestsellers")
print("📦 Scraping Amazon Product...")
product_url = "
product_data = scraper.scrape_amazon_product(product_url, "10001")
scraper.print_results(product_data, "Amazon Product Data")
print("👤 Scraping LinkedIn Profile...")
linkedin_url = "
linkedin_data = scraper.scrape_linkedin_profile(linkedin_url)
scraper.print_results(linkedin_data, "LinkedIn Profile Data")
print("🤖 Running AI Agent Query...")
agent_query = """
Scrape Amazon product data for
in New York (zipcode 10001) and summarize the key product details.
"""
scraper.run_agent_query(agent_query)
The main function () ties everything together by setting everything to your brightdata keys and Google API, and eventually runs Linked Results, and finally runs well-prepared results after each step.
if __name__ == "__main__":
print("Installing required packages...")
os.system("pip install -q langchain-brightdata langchain-google-genai langgraph")
os.environ["BRIGHT_DATA_API_KEY"] = "Use Your Own API Key"
main()
Finally, this inner block ensures that, when running as independent text, the required libraries are silenced, and the Brightdata API key is set to nature. Then the main work is done to start all the flow of the flow and performance.
In conclusion, at the end of this lesson, you will have a Python text that is ideal for default data for data collection activities, issuing the lower API info, and taps by choice in the Adventional Revenue Revenue. You can extend this basis by adding support to other types of data, including additional llms, or sending a scraper as part of a larger pipe or web site. In these construction blocks, you are now intended to collect, analytical, and effective web data, whether market research, competition, operational applications.
Look Notebook. All credit for this study goes to research for this project. Also, feel free to follow it Sane and don't forget to join ours 100K + ml subreddit Then sign up for Our newspaper.
Asphazzaq is a Markteach Media Inc. According to a View Business and Developer, Asifi is committed to integrating a good social intelligence. His latest attempt is launched by the launch of the chemistrylife plan for an intelligence, MarktechPost, a devastating intimate practice of a machine learning and deep learning issues that are clearly and easily understood. The platform is adhering to more than two million moon visits, indicating its popularity between the audience.
