Utilities for spider.
Project description
Spider Utility
Utilities for spider.
Features
- Database connector
- Local file path generator
- Network session
- Data structure
- Log
- Exception
Installation
pip install spiderutil
Documentation
Still working...
Examples
Still working...
How to contribute
Open a new issue when you have questions, advices, or you encounter a bug.
Recommend packages
Parsers
- beautifulsoup4: parse the webpage like a tree.
pip install beautifulsoup4
- scrapy/parsel: Parse webpage with XPath or CSS selectors.
pip install parsel
Encoding and Decoding
- google/brotli: decode brotli encoding.
pip install brotli
Database
- SQLAlchemy: Process the SQL statement like object
pip install SQLAlchemy
- PyMySQL: Connect to MySQL
pip install pymysql
- mkleehammer/pyodbc: Connect to Microsoft SQL Server
pip install pyodbc
In addition to: level12/sqlalchemy_pyodbc_mssql.
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
spiderutil-0.1.8.tar.gz
(11.5 kB
view details)
Built Distribution
Filter files by name, interpreter, ABI, and platform.
If you're not sure about the file name format, learn more about wheel file names.
Copy a direct link to the current filters
File details
Details for the file spiderutil-0.1.8.tar.gz.
File metadata
- Download URL: spiderutil-0.1.8.tar.gz
- Upload date:
- Size: 11.5 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.0.0 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.0.0.post20200309 requests-toolbelt/0.9.1 tqdm/4.42.1 CPython/3.7.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
dbdb0d2a0051ae84d1f505719f1651dd38893205859887e07bee011222aa9f6b
|
|
| MD5 |
3ba6a8ced323a0b6953e44eae9d291d4
|
|
| BLAKE2b-256 |
bc564483a917741a7f6e2c48398ba93fdb722a18a607eee5ba40cf5b6b3745bd
|
File details
Details for the file spiderutil-0.1.8-py2.py3-none-any.whl.
File metadata
- Download URL: spiderutil-0.1.8-py2.py3-none-any.whl
- Upload date:
- Size: 16.5 kB
- Tags: Python 2, Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.0.0 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.0.0.post20200309 requests-toolbelt/0.9.1 tqdm/4.42.1 CPython/3.7.3
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 |
baaf9954c018fb29f97d997b256f9a9d9287d4f324872c88965a8c80afb0ac68
|
|
| MD5 |
598b61296174564282cc38b7223a290f
|
|
| BLAKE2b-256 |
f8b875fbf6eac1ac4fe319521f0b4813effd2cdc9c0048231ff0109fc5cec6d4
|
