Client interface for Scrapinghub HubStorage
Project description
Overview
This is the HubStorage client library, which contains:
Full client api trough hubstorage.HubstorageClient
Requirements
requests: http://pypi.python.org/pypi/requests
Basic API
Example creating a new job:
>>> from hubstorage import HubstorageClient >>> hs = HubstorageClient(auth=apikey) >>> job = hs.push_job(projectid='1111111', spidername='foo') >>> job.key '1111111/1/1' >>> job.metadata['state'] 'pending' >>> job.items.write({'title': 'my first item'}) >>> job.logs.info('lorem impsum message are cool') >>> job.logs.error('but sometimes s**t happens') >>> job.finished()
Example getting job data later:
>> job = hs.get_job('1111111/1/1') >> job.metadata['state'] 'finished' >> list(job.items.list(count=1)) [{'title': 'my first item'}] ...
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
hubstorage-0.18.0.tar.gz
(13.1 kB
view hashes)
Built Distribution
Close
Hashes for hubstorage-0.18.0-py2-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 3b846c605f6175ff3added359e7ffc68590c2a528c3a0f94d47253e6cafbaeed |
|
MD5 | fce72b6daa2a26a8fa82eda679618bb0 |
|
BLAKE2b-256 | 5fa9f9c290d4259057944e29d6b712700482087cb8cd5b2fdbb7d538c8bf8e10 |