Skip to main content
Join the official 2019 Python Developers SurveyStart the survey!

采集中文小说网站的爬虫

Project description

python 环境

  • python 2.7
  • mysql 5.7.9

项目安装

pip install novelSpider

MySQL 配置

# 加入 mysqld.cnf
# * Encode
init_connect='SET collation_connection = utf8_unicode_ci'
init_connect='SET NAMES utf8'
character-set-server=utf8
collation-server=utf8_unicode_ci
skip-character-set-client-handshake

项目运行

from novelSpider.task import Task

class Config(object):
    '''
    @desc:数据库配置
    @param:username 数据库用户名
    @param:password 数据库密码
    '''
    def __init__(self):
        self.username = 'root'
        self.password = 'root'
        self.database = 'novel'

# 创建任务实例
task = Task()

# 创建爬虫实例
spider = task.createDownloader(Config)

# 下载小说书目信息、章节列表
spider.getCharptList(novelNum=1)

# 下载小说书目信息、章节列表
spider.getCharptContent(novelId=0, charptNum=1)

版本日志

0.0.13 版本

# 支持下载小说书目信息、章节列表、章节内容

Project details


Release history Release notifications

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for novelSpider, version 0.0.13
Filename, size File type Python version Upload date Hashes
Filename, size novelSpider-0.0.13-py2-none-any.whl (8.2 kB) File type Wheel Python version py2 Upload date Hashes View hashes
Filename, size novelSpider-0.0.13.tar.gz (4.7 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN SignalFx SignalFx Supporter DigiCert DigiCert EV certificate StatusPage StatusPage Status page