Photo for Author | Chatgt
Obvious Introduction
To get real data for your data science projects usually is the hardest part. Toys are easy to find, but to find higher data or real time you usually need to use api or build custom suspecting pipes to issue information from the web.
In this article, I share my favorite 10 APIs – APIs used daily for data collection, data integration, and construction agents AI. These are five categories, reliable data, data scanning, and web search, and web search, to optimize the appropriate tool and from the details to Insight immediately.
Obvious The base of the foundation of the foundation
Basic data repository is a community-based platform where different organizations and open source donors share their information with a broad world. With a simple command, you can access these datasets for your project.
// 1. Kaggle API
Grandfather The most popular data when applying to data science projects. Instead of downloading manually, you can create a data pipe to download data data, wipe it out, and download it to your work environment.
This sale is allotted by the open source public for everyone to use. To get started, generate API key from your Kaggle account and set it as a flexibility. Then, you can conduct the following instructions in your forum. KaGgle and offers Pyython SDK, allows simple integration and code.
kaggle datasets download -d kingabzpro/world-vaccine-progress -p data --unzip// 2. Kissing a CLI face
It's like KaGle, Kisses face It is also a community science and a machine community where people share information, models, and demons. You can easily install Hugging Cebyi CLI and combine it in your work travel using CLI commands or Python code. Both options allow you to download datasets without requiring API key.
API key is required only when the dataset has been digested.
hf download kingabzpro/dermatology-qa-firecrawl-datasetObvious Web and Crawling Apis
Web contains various information. If you are unable to find the details you need in the above-mentioned platforms, you may need to change your data by threatening the web or using a Web Search API.
// 3. Firecrawl
Firecrawl It provides an API to remove content from websites and convert them into marking format to integrate with the simple AI. It also comes with API cleaning and extraction combined with llm (a large language model) for improved web options.
This API must have. I use daily to create data and combine it on my AI projects.
curl -s -X POST "
-H "Authorization: Bearer $FIRECRAWL_API_KEY"
-H "Content-Type: application/json"
-d '{
"url": "
"formats": ["markdown", "html"]
}'// 4. By the boat
For Monya It is an API for quick web support that offers 1,000 search applications for free. Accurate and faster. You can use the datastets, combine it in your AI projects, or use it as a simple api for searching your development needs.
curl --request POST
--url
--header "Authorization: Bearer "
--header "Content-Type: application/json"
--data '{
"query": "who is Leo Messi?",
"auto_parameters": false,
"topic": "general",
"search_depth": "basic",
"chunks_per_source": 3,
"max_results": 1,
"days": 7,
"include_answer": true,
"include_raw_content": true,
"include_images": false,
"include_image_descriptions": false,
"include_favicon": false,
"include_domains": [],
"exclude_domains": [],
"country": null
}' Obvious GEOSPATIAL AND WEATHER APIS
If you want the weather information and geospatial information, you will know that things continue to change. That is why you need real time for this data by API.
// 5. OPEWENEMAP
OPEWIEMAP Is the service that provides world weather data by API, including current conditions, predicting, historical records, and-by-by-by-by-by-by-by-by-by-by-by-by-by-by-by-by-by-the hyperlocol for predictors of heaven.
curl "// 6. OpenstreetMap
OpenStreetMap provides for the data of the land map, and Overpass API Does the database of the read only function work in the customs of OSM and can be asked by the best ql. An instance that is less than when carrying cafe areas inside a small London box.
curl -G "
--data-urlencode 'data=[out:json];node["amenity"="cafe"](51.50,-0.15,51.52,-0.10);out;'
Obvious Market Market Data Apis
Financial Market Market Apis Apis is very recommended when working on a financial project and requires real-time data in custody, CRYPTO, and other financial and financial information.
// 7. Alpha Vantage
Alpha Vantage Is the platform of financial data that provides free API of actual and historical data to custody. It also provides Chart-Ready Time series to Intraday, daily, weekly and monthly evolution.
curl "// 8. Yahoo Finance
Many beginners and doctors use yfnance API to find stock quotes, details of a series of memory, assignments and cracking, and basic metadata. This allows them to create data frames ready to process quick prototypes and project projects.
Yahoo final finances It provides free stock quotes, portfolios, portfolio tools, and international market coverage, enabling users to check the market details list without specific expenses.
import yfinance as yf
print(yf.download("AAPL", period="1y").head())Obvious Social and Community Data for data
If you work on a text analysis project and public discussions from the highest communication platforms, then these APIs provide easy access to real communication information.
// 9. Reddit
Red provides a wealthy, community-driven source, and Python Reddit API WRAPPER (PRW) It makes it easy to access the official reddit apple of tasks such as downloading posts, comments, and the Python Metadata.
Praw works by posting requests to reddit's API under the hood and is usually used in teaching and research to collect the chat tribs to be analyzed.
import praw
r = praw.Reddit(
client_id="ID",
client_secret="SECRET",
user_agent="myapp:ds-project:v1 (by u/yourname)"
)
print([s.title for s in r.subreddit("Python").hot(limit=5)])
// 10. X
X (Begad over that Twitter) provides an engineering platform with user restrooms and replacement of content, and the prevalence options for real-time data. Accessing often requires verification, adherence to measure restrictions and policy, and select the appropriate tier tier with your volume and volume use.
curl -H "Authorization: Bearer YOUR_BEARER_TOKEN"
"Obvious The last thoughts
These APIs provide free access to data that is usually difficult to find. They greatly improve your Web data ability or improve your web synthesis, which allow you to create customized datasets.
I am very recommending the icon to this article so I update when you need high quality data, real time from the web. By installing the Apis these APIs, you can open an important understanding that will help in your research and analysis.
Abid Awa (@ 1abidaswan) is a certified scientist for a scientist who likes the machine reading models. Currently, focus on the creation of the content and writing technical blogs in a machine learning and data scientific technology. Avid holds a Master degree in technical management and Bachelor degree in Telecommunication Engineering. His viewpoint builds AI product uses a Graph Neural network for students who strive to be ill.
