Reliable APIs for the website data
Project description
Web Transpose
Web Crawler & AI Web Scraper APIs for building new web experiences.
pip install webtranspose
Introduction · Installation · Docs
Introduction
In the near future, nobody will open websites. Instead, we will be directly served the information we are seeking. New web experiences will combine the information from many websites into a single, unified experience.
Web Transpose is a collection of API tools that allow building these new web experiences simple.
Crawl
import webtranspose as webt
import os
os.environ['WEBTRANSPOSE_API_KEY'] = "YOUR WEBT API KEY"
crawl = webt.Crawl(
"https://www.example.com",
max_pages=100,
render_js=True,
)
await crawl.crawl() # crawl.queue_crawl() for async
Scrape
import webtranspose as webt
import os
os.environ['WEBTRANSPOSE_API_KEY'] = "YOUR WEBT API KEY"
schema = {
"Merchant Name": "string",
"Title of Product": "string",
"Product Photo URL": "string",
}
scraper = webt.Scraper(
schema,
render_js=True,
)
out_json = scraper.scrape("https://www.example.com")
Web Search (AI SERP API)
import webtranspose as webt
import os
os.environ['WEBTRANSPOSE_API_KEY'] = "YOUR WEBT API KEY"
results = webt.search("what caused the fourth great ninja war?")
# results.keys()
# ['results']
# AI Filter
results = webt.search_filter("Paul Graham's Blog")
# results.keys()
# ['results', 'filtered_results']
Installation
Non-Python Users: 📄 API Docs.
This repo contains a local lite installation of Web Transpose. This is a good option if you want to run Web Transpose locally on your machine for quick use cases.
pip install webtranspose
However, if you wish to leverage the full tools of Web Transpose and use in production, you should add your API key to add the full version.
os.environ["WEBTRANSPOSE_API_KEY"] = "YOUR_API_KEY_HERE"
Enterprise Support
Web Transpose serves enterprises small and large. We partner with companies for the long term with hands-on support and custom solutions.
Please email me directly at mike@webtranspose.com for enquiries.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for webtranspose-0.3.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 0f6c90f887437468668d249a0b5456c81b3d3fa341117201c91461009b255aa6 |
|
MD5 | 89d3613b8a91518768a891bf672e4c1d |
|
BLAKE2b-256 | b9e4c7f568cb25b33cf133c3393ee28764b19d2176bc58411dbaed56bd45fa1e |