Scrapy item exporter for the Algolia API
Project description
[![PyPI](https://img.shields.io/pypi/v/scrapy_algolia_exporter.svg)](https://pypi.python.org/pypi/scrapy_algolia_exporter) [![Build Status](https://travis-ci.org/clemfromspace/scrapy-algolia-exporter.svg?branch=master)](https://travis-ci.org/clemfromspace/scrapy-algolia-exporter)
## Scrapy Algolia Exporter
Directly populate a given **Algolia** index from a scrapy spider.
### Usage
:warning: **Your crawled items must contains a unique `ObjectID` key.** :warning:
Add the mandatory scrapy settings to configure the **Algolia** API:
```python
ALGOLIA_API_ID='my_algolia_api_id'
ALGOLIA_API_KEY='my_algolia_api_key'
ALGOLIA_INDEX_NAME='my_algolia_index_name'
```
Add the `AlgoliaItemPipeline` in the `ITEM_PIPELINES` scrapy setting:
```python
ITEM_PIPELINES = {
'scrapy_algolia_exporter.pipelines.AlgoliaItemPipeline': 10
}
```
The `ALGOLIA_ITEM_BULK_NBR` setting control how many items will be send to Algolia at the same time.
If not provided, the items will be send by group of `100`.
### Install
```
pip install scrapy_algolia_exporter
```
## Scrapy Algolia Exporter
Directly populate a given **Algolia** index from a scrapy spider.
### Usage
:warning: **Your crawled items must contains a unique `ObjectID` key.** :warning:
Add the mandatory scrapy settings to configure the **Algolia** API:
```python
ALGOLIA_API_ID='my_algolia_api_id'
ALGOLIA_API_KEY='my_algolia_api_key'
ALGOLIA_INDEX_NAME='my_algolia_index_name'
```
Add the `AlgoliaItemPipeline` in the `ITEM_PIPELINES` scrapy setting:
```python
ITEM_PIPELINES = {
'scrapy_algolia_exporter.pipelines.AlgoliaItemPipeline': 10
}
```
The `ALGOLIA_ITEM_BULK_NBR` setting control how many items will be send to Algolia at the same time.
If not provided, the items will be send by group of `100`.
### Install
```
pip install scrapy_algolia_exporter
```
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Close
Hashes for scrapy_algolia_exporter-0.0.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | e8853d8649dfb8925da6613eb592d1aca68edea339727ea2e2ae8dd0c0415f68 |
|
MD5 | 01d6a7bd85f4bcc45991862b815f759c |
|
BLAKE2b-256 | d8da507cfcb77325801cd7b6a5520e002f23c26ecc81f2ec4b1c10afe2a30c6d |
Close
Hashes for scrapy_algolia_exporter-0.0.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fc52d1c37510bf772a25f418a22ff2dfb302de047b3ad4a6ee356c8e1a4597ad |
|
MD5 | ccbdb039f5cc4fb5e41d418343b27197 |
|
BLAKE2b-256 | d7545de474ab4e3f24f1d9a32a066a7a8e47b470ff6da3712b7112226734f66a |