selenium-driverless 1.8.0.2

Undetected selenium without chromedriver usage (Non-commercial use only!)

Selenium-Driverless (Non-commercial use only!)

Documentation

• Use Selenium without chromedriver
• Currently passes Cloudflare, Bet365, Turnstile, and others
• Multiple tabs simultaneously
• Multiple Incognito-contexts with individual proxy & cookies
• Async (asyncio) and sync (experimental) support
• Proxy-auth support (experimental, example code)
• Network-interception
• headless supported

Questions?

Feel free to join the Driverless-Community on Discord:)

Also, see dev-branch for the latest implementations.

dev-installation (click to expand)

pip uninstall -y selenium-driverless
pip install https://github.com/kaliiiiiiiiii/Selenium-Driverless/archive/refs/heads/dev.zip

Still getting detected?

Feel free to give me a feedback!
You're a company and looking for another solution? Maybe undetect.io is smth for you

Also, feel free to

Dependencies

• Python >= 3.7
• Google-Chrome installed (Chromium not tested)

Installing

• Install Google-Chrome
• pip install selenium-driverless

Usage

with asyncio

from selenium_driverless import webdriver
from selenium_driverless.types.by import By
import asyncio


async def main():
    options = webdriver.ChromeOptions()
    async with webdriver.Chrome(options=options) as driver:
        await driver.get('http://nowsecure.nl#relax')
        await driver.sleep(0.5)
        await driver.wait_for_cdp("Page.domContentEventFired", timeout=15)
        
        # wait 10s for elem to exist
        elem = await driver.find_element(By.XPATH, '/html/body/div[2]/div/main/p[2]/a', timeout=10)
        await elem.click(move_to=True)

        alert = await driver.switch_to.alert
        print(alert.text)
        await alert.accept()

        print(await driver.title)


asyncio.run(main())

synchronous

asyncified, might be buggy

from selenium_driverless.sync import webdriver

options = webdriver.ChromeOptions()
with webdriver.Chrome(options=options) as driver:
    driver.get('http://nowsecure.nl#relax')
    driver.sleep(0.5)
    driver.wait_for_cdp("Page.domContentEventFired", timeout=15)

    title = driver.title
    url = driver.current_url
    source = driver.page_source
    print(title)

custom debugger address

from selenium_driverless import webdriver

options = webdriver.ChromeOptions()
options.debugger_address = "127.0.0.1:2005"

# specify if you don't want to run remote
# options.add_argument("--remote-debugging-port=2005")

async with webdriver.Chrome(options=options) as driver:
  await driver.get('http://nowsecure.nl#relax', wait_load=True)

Network-Interception

• use CDP events (see chrome-developer-protocoll for possible events)

Notes: synchronous might not work properly

Example Code (Click to expand)

warning: network interception with Fetch.enable might have issues with cross-domain iframes, maximum websocket message size or Font requests.
You might try using `Network.setRequestInterception (officially deprecated) or narrowing the pattern

import asyncio
import base64
import sys
import time
import traceback

from cdp_socket.exceptions import CDPError
from selenium_driverless import webdriver


async def on_request(params, global_conn):
    url = params["request"]["url"]
    _params = {"requestId": params['requestId']}
    if params.get('responseStatusCode') in [301, 302, 303, 307, 308]:
        # redirected request
        return await global_conn.execute_cdp_cmd("Fetch.continueResponse", _params)
    else:
        try:
            body = await global_conn.execute_cdp_cmd("Fetch.getResponseBody", _params, timeout=1)
        except CDPError as e:
            if e.code == -32000 and e.message == 'Can only get response body on requests captured after headers received.':
                print(params, "\n", file=sys.stderr)
                traceback.print_exc()
                await global_conn.execute_cdp_cmd("Fetch.continueResponse", _params)
            else:
                raise e
        else:
            start = time.perf_counter()
            body_decoded = base64.b64decode(body['body'])

            # modify body here

            body_modified = base64.b64encode(body_decoded).decode("ascii")
            fulfill_params = {"responseCode": 200, "body": body_modified, "responseHeaders": params["responseHeaders"]}
            fulfill_params.update(_params)
            if params["responseStatusText"] != "":
                # empty string throws "Invalid http status code or phrase"
                fulfill_params["responsePhrase"] = params["responseStatusText"]

            _time = time.perf_counter() - start
            if _time > 0.01:
                print(f"decoding took long: {_time} s")
            await global_conn.execute_cdp_cmd("Fetch.fulfillRequest", fulfill_params)
            print("Mocked response", url)


async def main():
    async with webdriver.Chrome(max_ws_size=2 ** 30) as driver:
        driver.base_target.socket.on_closed.append(lambda code, reason: print(f"chrome exited"))

        global_conn = driver.base_target
        await global_conn.execute_cdp_cmd("Fetch.enable",
                                          cmd_args={"patterns": [{"requestStage": "Response", "urlPattern": "*"}]})
        await global_conn.add_cdp_listener("Fetch.requestPaused", lambda data: on_request(data, global_conn))

        await driver.get("https://nowsecure.nl/#relax", timeout=60, wait_load=False)
        while True:
            # time.sleep(10) # no. cloudflare would hang
            await asyncio.sleep(10)


asyncio.run(main())

Multiple tabs simultaneously

Note: asyncio is recommended, threading only works on independent webdriver.Chrome instances.

Example Code (Click to expand)

from selenium_driverless.sync import webdriver
from selenium_driverless.utils.utils import read
from selenium_driverless import webdriver
import asyncio


async def target_1_handler(target):
    await target.get('https://abrahamjuliot.github.io/creepjs/')
    print(await target.title)


async def target_2_handler(target):
    await target.get("about:blank")
    await target.execute_script(script=read("/files/js/show_mousemove.js"))
    await target.pointer.move_to(500, 500, total_time=2)


async def main():
    options = webdriver.ChromeOptions()
    async with webdriver.Chrome(options=options) as driver:
        target_1 = await driver.current_target
        target_2 = await driver.new_window("tab", activate=False)
        await asyncio.gather(
            target_1_handler(target_1),
            target_2_handler(target_2)
        )
        await target_1.focus()
        input("press ENTER to exit")


asyncio.run(main())

Unique execution contexts

• execute javascript without getting detected

Example Code (Click to expand)

from selenium_driverless.sync import webdriver
from selenium_driverless import webdriver
import asyncio


async def main():
    options = webdriver.ChromeOptions()
    async with webdriver.Chrome(options=options) as driver:
        await driver.get('chrome://version')
        script = """
        const proxy = new Proxy(document.documentElement, {
          get(target, prop, receiver) {
            if(prop === "outerHTML"){
                console.log('detected access on "'+prop+'"', receiver)
                return "mocked value:)"
            }
            else{return Reflect.get(...arguments)}
          },
        });
        Object.defineProperty(document, "documentElement", {
          value: proxy
        })
        """
        await driver.execute_script(script)
        src = await driver.execute_script("return document.documentElement.outerHTML", unique_context=True)
        mocked = await driver.execute_script("return document.documentElement.outerHTML", unique_context=False)
        print(src, mocked)


asyncio.run(main())

Pointer Interaction

see @master/dev/show_mousemove.py for visualization

pointer = await driver.current_pointer
move_kwargs = {"total_time": 0.7, "accel": 2, "smooth_soft": 20}

await pointer.move_to(100, 500)
await pointer.click(500, 50, move_kwargs=move_kwargs, move_to=True)

Iframes

• switch and interact with iframes

iframes = await driver.find_elements(By.TAG_NAME, "iframe")
await asyncio.sleep(0.5)
iframe_document = await iframes[0].content_document
# iframe_document.find_elements(...)

use preferences

from selenium_driverless import webdriver
options = webdriver.ChromeOptions()

 # recommended usage
options.update_pref("download.prompt_for_download", False)
# or
options.prefs.update({"download": {"prompt_for_download": False}})

# supported
options.add_experimental_option("prefs", {"download.prompt_for_download": False})

Multiple Contexts

• different cookies for each context
• A context can have multiple windows and tabs within
• different proxy for each context
• opens as a window as incognito

Example Code (Click to expand)

from selenium_driverless import webdriver
import asyncio


async def main():
    options = webdriver.ChromeOptions()
    async with webdriver.Chrome(options=options) as driver:
        context_1 = driver.current_context
        
        await driver.set_auth("username", "password", "localhost:5000")
        # proxy not supported on windows due to https://bugs.chromium.org/p/chromium/issues/detail?id=1310057
        context_2 = await driver.new_context(proxy_bypass_list=["localhost"], proxy_server="http://localhost:5000")
        
        await context_1.current_target.get("https://examle.com")
        await context_2.get("https://examle.com")
        input("press ENTER to exit:)")


asyncio.run(main())

Custom exception handling

You can implement custom exception handling as following

import selenium_driverless
import sys
handler = (lambda e: print(f'Exception in event-handler:\n{e.__class__.__module__}.{e.__class__.__name__}: {e}',
                           file=sys.stderr))
sys.modules["selenium_driverless"].EXC_HANDLER = handler
sys.modules["cdp_socket"].EXC_HANDLER = handler

Help

You found a bug? Feel free to open an issue:) You've got other questions or proposials? feel free to join the Driverless-Community on Discord or open a discusion
Note: please check the todo's below at first!

Todo's

Click to expand

• implementations
  • WebElements
    • improve mid_location calculation
    • add WebElement.screenshot
  • Input
    • Mouse
      • mousemove
      • click
      • scroll
      • drag&drop
    • write
    • Touch
      • touchmove
      • TouchTap
      • scoll
      • pinch//zoom
    • KeyBoard
      • SendKeys
        • send files
  • support options.add_extension()
  • support prefs
• sync
  • move sync to threaded for allowing event_handlers
  • support multithreading with sync version
    • on independent driver instances
    • on same driver instance
• check Python 3.12.0

Authors

Copyright and Author:
Aurin Aegerter (aka Steve)

License

Shield:

Unless specified differently in a single file, this work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Disclaimer

I am not responsible what you use the code for!!! Also no warranty!

Acknowledgments

Inspiration, code snippets, etc.

• selenium_driverless/utils/find_chrome_executable
• cdp-socket
• jsobject
• pycdp/browser.py