Selenium package with requests integration and anti-bot detection measures
Project description
ak_selenium
Selenium package with requests integration and anti-bot detection measures
Documentation · Report Bug · Request Feature
Table of Contents
- 1. About the Project
- 2. Getting Started
- 3. Usage
- 4. Roadmap
- 5. License
- 6. Contact
- 7. Acknowledgements
1. About the Project
ak_selenium
is a Python package that provides an interface for automating browser tasks using Selenium. It comes with built-in functionalities for handling common tasks such as form filling, scrolling, and waiting for elements to load. Additionally, it has a built-in requests session that handles retries and timeouts, making it easier to send HTTP requests.
1.1. Features
- Chrome browser automation using Selenium WebDriver.
- Built-in methods for form filling, scrolling, and waiting for elements.
- Anti-bot detection measures
- Pass selenium headers/cookies to requests library
- Built-in requests session with retries and timeouts.
- Ability to use Chrome user data for browser automation.
- RAM optimization for browser options.
- Integrates Helium for easier automation
2. Getting Started
2.1. Installation
2.1.1. Production
Install with flit
pip install flit
flit install --deps production
Alternatively, you can use pip
pip install ak_selenium
2.1.2. Development
Install with flit
flit install --pth-file
3. Usage
from ak_selenium import Chrome, By, Keys
chrome = Chrome(headless=True) # Create a new Chrome browser instance
driver = chrome.driver #Get Chromedriver
chrome.get("https://example.com") # Navigate to a webpage
#Wait for element to load
locator = (By.TAG_NAME, "h1")
chrome.wait_for_locator(locator)
s = chrome.session # Pass selenium session to requests
s.get("https://www.iana.org/domains/reserved") # Get a website
# Get a list of websites
## Will randomize requests to not trigger bot detection
s.bulk_get(["https://www.iana.org/domains/reserved", "https://www.example.com"])
Integrated with Helium to make it easier to set up automation.
Helium methods and functions can be used as intended in the original documentation
Example:
import helium
helium.wait_until(helium.Button('Download').exists)
Alternatively, helium methods and classes have been collected into two classes Element
and Action
for convinience
Element
exposes the following classes: Alert
, Button
, CheckBox
, ComboBox
, Image
, Link
, ListItem
, RadioButton
, Text
, TextField
and the method find_all
Action
exposes the following methods: highlight
, wait_until
, refresh
, attach_file
, drag_file
, combobox_select
, hover
, write
.
Action
also incorporates a Mouse
sub-class that collect mouse-related methods.
Example:
from ak_selenium import Element, Action, Keys
import helium
chrome.get('https://google.com') #Go to website
Action.write('helium selenium github') #Enter text into text field
helium.press(Keys.ENTER) #Press Enter
Action.Mouse.click('mherrmann/helium') #Click
chrome.get('https://github.com/login') #Goto github
Action.write('username', into='Username') #Enter Username into Username field
Action.write('password', into='Password') #Enter Password into Password field
Action.Mouse.click('Sign in') #Click Sign-in
Action.Mouse.scroll(direction='down', num_pixels=100) #Scroll down 100px
helium.kill_browser() #Close the browser
3.1. Additional Options
# Selenium Overrides
## Overide default useragent
chrome.USERAGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) \
AppleWebKit/537.36 (KHTML, like Gecko) \
Chrome/83.0.4103.53 Safari/537.36'
## Override implicit and max wait times for selenium
chrome.IMPLICITLY_WAIT_TIME = 3 #seconds
chrome.MAX_WAIT_TIME = 5 #seconds
# Requests.Session Override
s.MIN_REQUEST_GAP = 0.9 #seconds between requests
4. Roadmap
- Add beautifulsoup integration
- Proxy
5. License
See LICENSE for more information.
6. Contact
Arun Kishore - @rpakishore
Project Link: https://github.com/rpakishore/ak_selenium
7. Acknowledgements
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ak_selenium-0.1.8-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c83bc26d6c73a3b8140c2d6cf56fd9e0d73e17c5be9a212ccf08ae8c6bd0e1b5 |
|
MD5 | 0a3f9623b58d9f50d0429d01ed239f2f |
|
BLAKE2b-256 | 90dc591947603e22eb6936e8ae5dfd87b18882bda0778dfd0be4863d364635ef |