Selenium package with requests integration and anti-bot detection measures
Project description
ak_selenium
Selenium package with requests integration and anti-bot detection measures
Documentation · Report Bug · Request Feature
Table of Contents
- 1. About the Project
- 2. Getting Started
- 3. Usage
- 4. Roadmap
- 5. License
- 6. Contact
- 7. Acknowledgements
1. About the Project
ak_selenium
is a Python package that provides an interface for automating browser tasks using Selenium. It comes with built-in functionalities for handling common tasks such as form filling, scrolling, and waiting for elements to load. Additionally, it has a built-in requests session that handles retries and timeouts, making it easier to send HTTP requests.
1.1. Features
- Chrome browser automation using Selenium WebDriver.
- Built-in methods for form filling, scrolling, and waiting for elements.
- Anti-bot detection measures
- Pass selenium headers/cookies to requests library
- Built-in requests session with retries and timeouts.
- Ability to use Chrome user data for browser automation.
- RAM optimization for browser options.
2. Getting Started
2.1. Installation
2.1.1. Production
Install with flit
pip install flit
flit install --deps production
Alternatively, you can use pip
pip install ak_selenium
2.1.2. Development
Install with flit
flit install --pth-file
3. Usage
from ak_selenium import Chrome, By, Keys
# Create a new Chrome browser instance
browser = Chrome(headless=True)
#Get Chromedriver
driver = chrome.driver
# Navigate to a webpage
driver.get("https://example.com")
# Wait for an element to load
#Wait for elements to load
locator = (By.TAG_NAME, "h1")
chrome.Wait_for_locator(locator)
# Fill a form
element = browser.driver.find_element_by_id("my-form")
browser.fill_userinput_form(element, "Hello, world!")
# Pass selenium session to requests
s = chrome.session
# Get a website
s.get("https://www.iana.org/domains/reserved")
# Get a list of websites
## Will randomize requests to not trigger bot detection
s.bulk_get(["https://www.iana.org/domains/reserved", "https://www.example.com"])
3.1. Additional Options
# Selenium Overrides
## Overide default useragent
chrome.USERAGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) \
AppleWebKit/537.36 (KHTML, like Gecko) \
Chrome/83.0.4103.53 Safari/537.36'
## Override implicit and max wait times for selenium
chrome.IMPLICITLY_WAIT_TIME = 3 #seconds
chrome.MAX_WAIT_TIME = 5 #seconds
# Requests.Session Override
s.MIN_REQUEST_GAP = 0.9 #seconds between requests
4. Roadmap
- Add beautifulsoup integration
- Proxy
5. License
See LICENSE for more information.
6. Contact
Arun Kishore - @rpakishore
Project Link: https://github.com/rpakishore/ak_selenium
7. Acknowledgements
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for ak_selenium-0.1.6-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | c5a38dc503d820c5ceb77821141d47ce80f8cd421a28c7d99819aacbad8e5a44 |
|
MD5 | d33a4698cfa3531d401ad05e3c8324db |
|
BLAKE2b-256 | 8a8ada143dd78692272a377e1d3f7cd31f0ce4163277f097484db4e47e2985d0 |