简易、好用的爬虫工具,减少重复代码与文件冗余
Project description
easy spider tool
在实际工作中,沉淀的一些简易、好用的爬虫工具,减少重复代码与文件冗余,希望一样能为使用者带来益处。如果您也想贡献好的代码片段,请将代码以及描述,通过邮箱( xinkonghan@gmail.com )发送给我。代码格式是遵循自我主观,如存在不足敬请指出!
安装
pip install easy_spider_tool
主要功能
时间相关
before_day
昨天日期(可用于时间递减)after_day
明天日期(可用于时间递增)between_day
两个日期之间current_date
当前时间timestamp
当前时间戳(支持精确到毫秒)date_parse
任意格式时间解析(支持时区转换,指定保留日期/时间(可设置默认值)部分)
json相关
format_json
漂亮美观的格式化输出jsonpath
任意多个json路径解析(支持设置默认值,选取首个匹配值)
hash摘要相关
md5
字符经md5编码
正则匹配相关
regex_match
条件匹配(支持多个不相关条件匹配,支持设置默认值,选取首个匹配值)for_to_regx_match
多个不相关条件匹配(兼容老版本保留)
数据清洗/转换相关
cookie_to_dic
cookie转换为字典(Dict)格式clear_value
清除列表(List)或字典(Dict)中的指定值(递归清除所有嵌套字典和列表中的指定值)
合法性验证相关
verify_ip_address
IP地址合法性验证verify_domain_name
域名合法性验证verify_port
端口合法性验证verify_url
URL合法性验证
通知相关
- 暂无
简单使用
from easy_spider_tool import format_json, jsonpath
data = {
"code": 200,
"data": [
{
"id": 1,
"username": "admin",
"level": "boss"
},
{
"id": 2,
"username": "user",
"level": "staff"
}
]
}
boss_name = jsonpath(data, '$.data[?(@.level=="boss")].username', first=True)
all_user_info = jsonpath(data, '$.data[*].username')
print(boss_name)
print(format_json(all_user_info))
链接
Github:https://github.com/hanxinkong/easy-spider-tool
在线文档:https://easy-spider-tool.xink.top/
注明
该工具借鉴作者【xingcweb】,根据主观新增部分功能
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
easy_spider_tool-1.0.17.tar.gz
(12.0 kB
view details)
Built Distribution
File details
Details for the file easy_spider_tool-1.0.17.tar.gz
.
File metadata
- Download URL: easy_spider_tool-1.0.17.tar.gz
- Upload date:
- Size: 12.0 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.7.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3f3a87298798de988ea43b5a11521fb72e0aebcee47b27c0fe010aecc7542124 |
|
MD5 | 55a31fb0ee76abdf3a771964d97d39e9 |
|
BLAKE2b-256 | 429384082ed8048078dde280543d2ce2915acbb6aa81a20a4f5e420232997d38 |
File details
Details for the file easy_spider_tool-1.0.17-py3-none-any.whl
.
File metadata
- Download URL: easy_spider_tool-1.0.17-py3-none-any.whl
- Upload date:
- Size: 13.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/4.0.2 CPython/3.7.6
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 6c485616689400fa6d0f6d5832c9884e90349406dc09a04d64e77acecae5ec7a |
|
MD5 | 2da9f2cf68073a1aa5e3d519020c22e8 |
|
BLAKE2b-256 | 68aabdbd9d768d8ec4179ba3f6316239f02fc927325e51f2eb426317bffa77af |