Fetch using your browser.
Let the browser manage cookies for you.
⚠️ Incomplete. Not tested thoroughly. Consider using Playwright, especially for more complex scenarios.
Usage
You’ll run a Python script containing some code like this:
from asyncio import gather, new_event_loop
from browserfetch import fetch, run_server
async def main():
response1, response2, reponse3 = await gather(
fetch('https://example.com/path1', params={'a': 1}),
fetch('https://example.com/image.png'),
fetch('https://example.com/path2', data={'a': 1}, method='post'),
)
# do stuff with retrieved responses
loop = new_event_loop()
loop.create_task(start_server())
loop.run_until_complete(main())Open your browser, goto http://example.com (perhaps solve a captcha and log in).
Copy the contents of browserfetch.js file and paste it in browser’s console. (You can use a browser extensions like violentmonkey/tampermonkey to do this step for you.)
That’s it! Your Python script starts handling requests. The browser tab should remain open of-coarse.
The server can handle multiple websocket connections from different websites simultaneously.
How it works
browserfetch communicates with your browser using a websocket. The fetch function just passes the request to browser and it is the browser that handles the actual request. Response data is sent back to Python using the same WebSocket connection.
Motivations
browser_cookie3 stopped working on Chrome-based browsers. There is a workaround: ShadowCopy, but it requires admin privilege.
Another issue with browser_cookie’s approach is that it retrieves cookies from cookie files, but these files are not updated instantly. Thus, you might have to wait or retry a few times before you can successfully access newly set cookies.
ShadowCopying and File access are slow and inefficient operations.
Downsides
Setting up browserfetch is more cumbersome since it requires running a Python server and also injecting a small script into the webpage. Using browser_cookie3 might be a better choice if there are many websites that you need to communicate with.
1 maintainer
Metadata
1 maintainer
Release files for browserfetch 0.16.1
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| browserfetch-0.16.1.tar.gz | 8.4 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| browserfetch-0.16.1-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 19.9 kB
Release files / browserfetch-0.16.1.tar.gz
| Download URL | browserfetch-0.16.1.tar.gz |
|---|---|
| Size | 8.4 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
962ad6b40a1196beee71fdc09c1f81e2e2afaf05f76a1327d2fd5864fca0e9aa
|
|
BLAKE2b-256 checksum How to use checksums |
4d9415ed670bdc7c9e18c01be4ffb5352d296f055c6550e0d1c6a8c33ead6fad
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.19 {"installer":{"name":"uv","version":"0.12.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
Release files / browserfetch-0.16.1-py3-none-any.whl
| Download URL | browserfetch-0.16.1-py3-none-any.whl |
|---|---|
| Size | 11.5 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
0a0f1a6537f5bb6da8255d3192ce4969d48849e5d4ccec44ba473b5542a9a814
|
|
BLAKE2b-256 checksum How to use checksums |
1ca6e692b7f8c5302f31b16955860898a2b1ae972916e1d9be894a5bb9da55f4
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
uv/0.12.19 {"installer":{"name":"uv","version":"0.12.19","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":null,"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}
|
