Skip to main content

Scrapy item exporter for the Algolia API

Project description

[![PyPI](https://img.shields.io/pypi/v/scrapy_algolia_exporter.svg)](https://pypi.python.org/pypi/scrapy_algolia_exporter) [![Build Status](https://travis-ci.org/clemfromspace/scrapy-algolia-exporter.svg?branch=master)](https://travis-ci.org/clemfromspace/scrapy-algolia-exporter)

## Scrapy Algolia Exporter

Directly populate a given **Algolia** index from a scrapy spider.

### Usage

:warning: **Your crawled items must contains a unique `ObjectID` key.** :warning:

Add the mandatory scrapy settings to configure the **Algolia** API:

```python
ALGOLIA_API_ID='my_algolia_api_id'
ALGOLIA_API_KEY='my_algolia_api_key'
ALGOLIA_INDEX_NAME='my_algolia_index_name'
```

Add the `AlgoliaItemPipeline` in the `ITEM_PIPELINES` scrapy setting:

```python
ITEM_PIPELINES = {
'scrapy_algolia_exporter.pipelines.AlgoliaItemPipeline': 10
}
```


The `ALGOLIA_ITEM_BULK_NBR` setting control how many items will be send to Algolia at the same time.
If not provided, the items will be send by group of `100`.

### Install
```
pip install scrapy_algolia_exporter
```








Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

scrapy_algolia_exporter-0.0.2.tar.gz (3.7 kB view details)

Uploaded Source

Built Distribution

scrapy_algolia_exporter-0.0.2-py3-none-any.whl (6.4 kB view details)

Uploaded Python 3

File details

Details for the file scrapy_algolia_exporter-0.0.2.tar.gz.

File metadata

File hashes

Hashes for scrapy_algolia_exporter-0.0.2.tar.gz
Algorithm Hash digest
SHA256 e8853d8649dfb8925da6613eb592d1aca68edea339727ea2e2ae8dd0c0415f68
MD5 01d6a7bd85f4bcc45991862b815f759c
BLAKE2b-256 d8da507cfcb77325801cd7b6a5520e002f23c26ecc81f2ec4b1c10afe2a30c6d

See more details on using hashes here.

File details

Details for the file scrapy_algolia_exporter-0.0.2-py3-none-any.whl.

File metadata

File hashes

Hashes for scrapy_algolia_exporter-0.0.2-py3-none-any.whl
Algorithm Hash digest
SHA256 fc52d1c37510bf772a25f418a22ff2dfb302de047b3ad4a6ee356c8e1a4597ad
MD5 ccbdb039f5cc4fb5e41d418343b27197
BLAKE2b-256 d7545de474ab4e3f24f1d9a32a066a7a8e47b470ff6da3712b7112226734f66a

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page