Skip to main content

download image from Baidu Image

Project description

BaiduImagesDownload

Python package codecov Codacy Badge

BaiduImagesDownload是一个快速、简单百度图片爬取工具

from BaiduImagesDownload.crawler import Crawler

net, num, urls = Crawler.get_images_url('二次元', 20)
Crawler.download_images(urls)

目录

安装

pip install BaiduImagesDownload

使用

基本

from BaiduImagesDownload.crawler import Crawler

net, num, urls = Crawler.get_images_url('二次元', 20)
Crawler.download_images(urls)

设置图片格式

from BaiduImagesDownload.crawler import Crawler

# rule默认为('.png', '.jpg')
net, num, urls = Crawler.get_images_url('二次元', 20)
Crawler.download_images(urls, rule=('.png', '.jpg'))

设置timeout

from BaiduImagesDownload.crawler import Crawler

# timeout默认为60(s)
net, num, urls = Crawler.get_images_url('二次元', 20, timeout=60)
Crawler.download_images(urls, rule=('.png', '.jpg'), timeout=60)

文档

get_images_url

class Crawler:

    @staticmethod
    def get_images_url(word: str, num: int, timeout: int = __CONCURRENT_TIMEOUT) -> (bool, bool, list):

参数

  • word: str: 搜索关键词
  • num: int: 搜索数量
  • timeout: int: 请求 timeout, 默认为60(s)

返回

  • net: bool: 网络连接是否成功,成功为 True,失败为 False
  • num: bool: 图片数量是否满足,满足为 True,不足为 False
  • urls: list: 获取的 urls,每项为一个dict,其中有两个键obj_urlfrom_urlobj_url为对应图片的urlfrom_urlReferer

download_images

class Crawler:

    @staticmethod
    def download_images(urls: list, rule: tuple = ('.png', '.jpg'),
                        path: str = 'download', timeout: int = __CONCURRENT_TIMEOUT,
                        concurrent: int = __CONCURRENT_NUM) -> (int, int):

参数

  • urls: list: 需要爬的图片列表,格式与get_images_url返回的相同
  • rule: tuple, optional: 允许下载的格式,默认为('.png', '.jpg')
  • path: str, optional: 图片下载的路径,默认为'download'
  • timeout: int, optional: 请求 timeout, 默认为60(s)
  • concurrent: int, optional: 并行下载的数量,默认为100

返回

  • success: int: 下载成功的数量
  • failed: int: 下载失败的数量

日志

可以设置日志的等级以及输出,具体请查看logging

import logging
from BaiduImagesDownload.crawler import logging

# 设置日志的等级为DEBUG
# 默认为INFO
logger.setLevel(logging.DEBUG)

# 设置输出到文件
file_handler = logging.FileHandler('~/BaiduImagesDownload.log')
file_handler.setFormatter(logging.Formatter(
    '[%(asctime)s] [%(levelname)s] %(message)s')) # 设置输出格式
logger.addHandler(file_handler)

许可

License: MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

BaiduImagesDownload-1.0.2.tar.gz (6.2 kB view hashes)

Uploaded source

Built Distribution

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page