WebArena benchmark for BrowserGym
Project description
WebArena benchmark for BrowserGym
This package provides browsergym.webarena, which is an unofficial port of the WebArena benchmark for BrowserGym.
Note: the original WebArena codebase has been slightly adapted to ensure compatibility.
Setup
- Install the package
pip install browsergym-webarena
- Download tokenizer ressources
python -c "import nltk; nltk.download('punkt_tab')"
- Setup the web servers (follow the webarena README).
BASE_URL=<YOUR_SERVER_URL_HERE>
- Setup the URLs as environment variables (note the
WA_prefix)
export WA_SHOPPING="$BASE_URL:7770/"
export WA_SHOPPING_ADMIN="$BASE_URL:7780/admin"
export WA_REDDIT="$BASE_URL:9999"
export WA_GITLAB="$BASE_URL:8023"
export WA_WIKIPEDIA="$BASE_URL:8888/wikipedia_en_all_maxi_2022-05/A/User:The_other_Kiwix_guy/Landing"
export WA_MAP="$BASE_URL:3000"
export WA_HOMEPAGE="$BASE_URL:4399"
- Setup an OpenAI API key
export OPENAI_API_KEY=...
NOTE: be mindful of costs, as WebArena will call GPT4 for certain evaluations (llm_fuzzy_match).
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file browsergym_webarena-0.13.1.tar.gz.
File metadata
- Download URL: browsergym_webarena-0.13.1.tar.gz
- Upload date:
- Size: 7.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/5.1.1 CPython/3.12.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | d1016b95b422eb0bde50ea7b4578f1128d7e20e0e41193ff91d7fe55e9d6eebc |
|
| MD5 | bc273c063ba2939ce15051ea9a4e6c7b |
|
| BLAKE2b-256 | 25d2df88dbdaaf338b3a78c0f34857e0d3196c8c7e1c273cf9312a5a93ea1872 |
Provenance
The following attestation bundles were made for browsergym_webarena-0.13.1.tar.gz:
Publisher:
pypi.yml on ServiceNow/BrowserGym
-
Statement type:
https://in-toto.io/Statement/v1- Predicate type:
https://docs.pypi.org/attestations/publish/v1 - Subject name:
browsergym_webarena-0.13.1.tar.gz - Subject digest:
d1016b95b422eb0bde50ea7b4578f1128d7e20e0e41193ff91d7fe55e9d6eebc - Sigstore transparency entry: 149138057
- Sigstore integration time:
- Predicate type:
File details
Details for the file browsergym_webarena-0.13.1-py3-none-any.whl.
File metadata
- Download URL: browsergym_webarena-0.13.1-py3-none-any.whl
- Upload date:
- Size: 7.6 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? Yes
- Uploaded via: twine/5.1.1 CPython/3.12.7
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | b50ce0fb270567eb1902baa1cb60643ce1e853c4e0c6bf731d6d73a29b4e946f |
|
| MD5 | 9e4f2a4705f92f370ccb76e311c0a5e2 |
|
| BLAKE2b-256 | d0faf1f4b6ff734c3d81930b4b18f206783dd176c032ad8f81f6c10627ca0504 |
Provenance
The following attestation bundles were made for browsergym_webarena-0.13.1-py3-none-any.whl:
Publisher:
pypi.yml on ServiceNow/BrowserGym
-
Statement type:
https://in-toto.io/Statement/v1- Predicate type:
https://docs.pypi.org/attestations/publish/v1 - Subject name:
browsergym_webarena-0.13.1-py3-none-any.whl - Subject digest:
b50ce0fb270567eb1902baa1cb60643ce1e853c4e0c6bf731d6d73a29b4e946f - Sigstore transparency entry: 149138103
- Sigstore integration time:
- Predicate type:
