Skip to main content

Lightweight and expandable spider framework.

Project description

# charlotte

[![Build Status](https://travis-ci.com/Jack-Kingdom/charlotte.svg?branch=master)](https://travis-ci.com/Jack-Kingdom/charlotte)

Lightweight and expandable spider framework.

## Install
```shell
pip install charlotte
```

## Example

```python
import json
import asyncio
from charlotte.spider import BaseSpider


class Spider(BaseSpider):

async def on_start(self):
response = await self.fetch('https://blog.qiaohong.org/api/v1/articles?limit=999')
items = json.loads(response.body)

print(len(items))

await asyncio.wait(
[self.fetch('https://blog.qiaohong.org/api/v1/articles/' + item['slug'],
parser=self.parse_detail) for item in items])

def parse_detail(self, res):
print(res.body)


if __name__ == '__main__':
Spider().run()


```

## Documentation
Check [wiki](https://github.com/Jack-Kingdom/charlotte/wiki) for details.

## LICENSE
MIT

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

charlotte-0.2.1.linux-x86_64.tar.gz (9.8 kB view details)

Uploaded Source

File details

Details for the file charlotte-0.2.1.linux-x86_64.tar.gz.

File metadata

  • Download URL: charlotte-0.2.1.linux-x86_64.tar.gz
  • Upload date:
  • Size: 9.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/40.8.0 requests-toolbelt/0.9.1 tqdm/4.31.1 CPython/3.6.5

File hashes

Hashes for charlotte-0.2.1.linux-x86_64.tar.gz
Algorithm Hash digest
SHA256 15a24e5dcdb4e0a37f18c3e51dd56535fd86f1c782f2f3eabbc513139c227e62
MD5 f9da494744b6256933cd6c22ec3ea1d0
BLAKE2b-256 3b58204c12b0dba3976ba1df968a8793f1871e3626f53cc2eabd76f67120be98

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page