Python askfm crawler.
Project description
# askfmcrawler
[![PyPI Version](https://img.shields.io/pypi/v/askfmcrawler.svg)](https://pypi.python.org/pypi/askfmcrawler)
Python askfm crawler.
## Install
```
pipenv install askfmcrawler
```
## Requirements
- Python 3.6 or higher
- selenium
## Usage
```python
from selenium import webdriver
from askfmcrawler import Crawler
driver = webdriver.Chrome('./chromedriver')
crawler = Crawler(driver)
# Crawl user questions
for article in crawler.crawl_user_questions('uehara1414test', limit=20):
print(article.question, " => ", article.answer)
# 好きなデザートは? => ガトーショコラ
# 赤信号なのに道路を渡ることはある? => ある
# ...
# Get random users and crawl questions of them
for user in crawler.crawl_random_users():
for article in crawler.crawl_user_questions(user):
print(user, article.question, " => ", article.answer)
driver.quit()
```
## Test
### Setup
- Install requiremnts with `pipenv install --dev`
- Place chromedriver to project root
### Run tests
```sh
pipenv run nosetests
```
[![PyPI Version](https://img.shields.io/pypi/v/askfmcrawler.svg)](https://pypi.python.org/pypi/askfmcrawler)
Python askfm crawler.
## Install
```
pipenv install askfmcrawler
```
## Requirements
- Python 3.6 or higher
- selenium
## Usage
```python
from selenium import webdriver
from askfmcrawler import Crawler
driver = webdriver.Chrome('./chromedriver')
crawler = Crawler(driver)
# Crawl user questions
for article in crawler.crawl_user_questions('uehara1414test', limit=20):
print(article.question, " => ", article.answer)
# 好きなデザートは? => ガトーショコラ
# 赤信号なのに道路を渡ることはある? => ある
# ...
# Get random users and crawl questions of them
for user in crawler.crawl_random_users():
for article in crawler.crawl_user_questions(user):
print(user, article.question, " => ", article.answer)
driver.quit()
```
## Test
### Setup
- Install requiremnts with `pipenv install --dev`
- Place chromedriver to project root
### Run tests
```sh
pipenv run nosetests
```
Project details
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
askfmcrawler-0.1.4.tar.gz
(2.3 kB
view details)
File details
Details for the file askfmcrawler-0.1.4.tar.gz
.
File metadata
- Download URL: askfmcrawler-0.1.4.tar.gz
- Upload date:
- Size: 2.3 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/1.12.1 pkginfo/1.4.2 requests/2.20.1 setuptools/40.6.2 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.5
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 440aa698ddd0f6b618a99b6f1ddb55c666b1b1ebbcbe1586809d174143dc9931 |
|
MD5 | f0e0c4a76ea958865bdc903fca698006 |
|
BLAKE2b-256 | 9938f1517a3ac8149cdfab7e50c7f535fb2885e906c403fc06cb32e39bb08f78 |