ggwebextract 0.1.1

Add your description here

Installation

Install chromium

If chromium is not detected after installation, set CHROME_PATH environment variable to point to the chromium executable file

Linux

sudo apt update
sudo apt upgrade
sudo apt install chromium

Windows

You can install chromium from woolyss

Install venv to run tests

uv run python ## should automatically create venv and install dependencies

Usage

• method to get html content of a page
• or with a "tab" handle if you want to automate a flow

check tests for details on how to use.

Notes

Browser control

• -> selenium : old and battle tested
• pyppeteer : only chrome -> chrome debug protocol
• playwright : modern, support more browsers

How to set user agents

• The only working option found is the --user-agent chromium argument

Chromium started in its own process

• if vanilla selenium is used with patched driver, it does not work (detected as bot). The reason is that selenium starts chromium itself and that is a problem. Chromium has to be started independently.

References

• undetected-chromedriver

Improvements

• ?