Utilities for spider.
Project description
Spider Utility
Utilities for spider.
Features
- Database connector
- Local file path generator
- Network session
- Data structure
- Log
- Exception
Installation
pip install spiderutil
Documentation
Still working...
Examples
Still working...
How to contribute
Open a new issue when you have questions, advices, or you encounter a bug.
Recommend packages
Parsers
- beautifulsoup4: parse the webpage like a tree.
pip install beautifulsoup4
- scrapy/parsel: Parse webpage with XPath or CSS selectors.
pip install parsel
Encoding and Decoding
- google/brotli: decode brotli encoding.
pip install brotli
Database
- SQLAlchemy: Process the SQL statement like object
pip install SQLAlchemy
- PyMySQL: Connect to MySQL
pip install pymysql
- mkleehammer/pyodbc: Connect to Microsoft SQL Server
pip install pyodbc
In addition to: level12/sqlalchemy_pyodbc_mssql.
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
spiderutil-0.1.7.tar.gz
(11.5 kB
view hashes)
Built Distribution
Close
Hashes for spiderutil-0.1.7-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | f98c208213ee463cc6a694bcdc7f354562fc9ca48ef78c16737431f9bd97dabc |
|
MD5 | 84235151847d5e7f418c1414ee62930a |
|
BLAKE2b-256 | f2c9bc109e3275ff074f92fc8931f2a26279b827cc8385bcc2ae49f8b4f60b1b |