Selenium Based Web Crawler
Project description
# Selenium Based Web Crawler
ReadTheDocs - http://browler.readthedocs.org/en/latest/
## Requirements
* redis `brew install redis`
## Install
```
pip install browler
```
## Uninstall
```
pip uninstall browler
```
## Configuration
### With Selenium Hub
```python
config = {
"browser": 'remote',
"remote": {
"url": 'http://localhost:49044/wd/hub',
'browser': 'firefox'
},
"url": "https://en.wikipedia.org/wiki/Main_Page",
"limit": 10,
"processes": 2
}
```
### Local Firefox
```python
config = {
"browser": 'firefox',
"url": "https://en.wikipedia.org/wiki/Main_Page",
"limit": 10,
"processes": 2
}
```
ReadTheDocs - http://browler.readthedocs.org/en/latest/
## Requirements
* redis `brew install redis`
## Install
```
pip install browler
```
## Uninstall
```
pip uninstall browler
```
## Configuration
### With Selenium Hub
```python
config = {
"browser": 'remote',
"remote": {
"url": 'http://localhost:49044/wd/hub',
'browser': 'firefox'
},
"url": "https://en.wikipedia.org/wiki/Main_Page",
"limit": 10,
"processes": 2
}
```
### Local Firefox
```python
config = {
"browser": 'firefox',
"url": "https://en.wikipedia.org/wiki/Main_Page",
"limit": 10,
"processes": 2
}
```
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
browler-0.1.3.tar.gz
(5.9 kB
view hashes)