scrapy-mysql-pipeline

scrapy-mysql-pipeline

Asynchronous mysql Scrapy item pipeline

Installation

pip install scrapy-mysql-pipeline

Configuration

Add pipeline

ITEM_PIPELINES = {
    'scrapy_mysql_pipeline.MySQLPipeline': 300,
}

Default values:

MYSQL_HOST = 'localhost'
MYSQL_PORT = 3306
MYSQL_USER = None
MYSQL_PASSWORD = ''
MYSQL_DB = None
MYSQL_TABLE = None
MYSQL_UPSERT = False
MYSQL_RETRIES = 3
MYSQL_CLOSE_ON_ERROR = True
MYSQL_CHARSET = 'utf8'

MYSQL_USER, MYSQL_PASSWORD, MYSQL_DB and MYSQL_TABLE, variables must be set in settings.py

Development Status
- 4 - Beta
Framework
- Scrapy
Intended Audience
- Developers
License
- OSI Approved :: MIT License
Operating System
- OS Independent
Programming Language
- Python
Topic
- Internet :: WWW/HTTP

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapy-mysql-pipeline-2019.7.19.tar.gz (4.6 kB view details)

Uploaded Jul 19, 2019 Source

File details

Details for the file scrapy-mysql-pipeline-2019.7.19.tar.gz.

File metadata

Download URL: scrapy-mysql-pipeline-2019.7.19.tar.gz
Upload date: Jul 19, 2019
Size: 4.6 kB
Tags: Source
Uploaded using Trusted Publishing? No
Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.32.1 CPython/3.7.3

File hashes

Hashes for scrapy-mysql-pipeline-2019.7.19.tar.gz
Algorithm	Hash digest
SHA256	`dc8a183727f2a79e3391461cec5970af62f4dbf44dd22d214860404f35853a4b`
MD5	`0a894b6b0fc757103c30be390c05c132`
BLAKE2b-256	`8cd0db760eaca71d6e9bb6e1859ce83402fe0f1b57ba6b00ffdb65e54b68df15`