Selenium Based Web Crawler
Project description
# Selenium Based Web Crawler
ReadTheDocs - http://browler.readthedocs.org/en/latest/
## Requirements
* redis `brew install redis`
## Install
```
pip install browler
```
## Uninstall
```
pip uninstall browler
```
## Configuration
### With Selenium Hub
```python
config = {
"browser": 'remote',
"remote": {
"url": 'http://localhost:49044/wd/hub',
'browser': 'firefox'
},
"url": "https://en.wikipedia.org/wiki/Main_Page",
"limit": 10,
"processes": 2
}
```
### Local Firefox
```python
config = {
"browser": 'firefox',
"url": "https://en.wikipedia.org/wiki/Main_Page",
"limit": 10,
"processes": 2
}
```
ReadTheDocs - http://browler.readthedocs.org/en/latest/
## Requirements
* redis `brew install redis`
## Install
```
pip install browler
```
## Uninstall
```
pip uninstall browler
```
## Configuration
### With Selenium Hub
```python
config = {
"browser": 'remote',
"remote": {
"url": 'http://localhost:49044/wd/hub',
'browser': 'firefox'
},
"url": "https://en.wikipedia.org/wiki/Main_Page",
"limit": 10,
"processes": 2
}
```
### Local Firefox
```python
config = {
"browser": 'firefox',
"url": "https://en.wikipedia.org/wiki/Main_Page",
"limit": 10,
"processes": 2
}
```
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
browler-0.1.3.tar.gz
(5.9 kB
view details)
File details
Details for the file browler-0.1.3.tar.gz
.
File metadata
- Download URL: browler-0.1.3.tar.gz
- Upload date:
- Size: 5.9 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 47220439ae6effe0d77a1b7950c7ab789f7b954c10e8ac9180506ca8d588564f |
|
MD5 | 6a710c063919ca73b1b6a947485625cf |
|
BLAKE2b-256 | 3e8d6d4d5e0840dabedba94e2b43000f0669208229121b07899dd5c86cd7298f |