Skip to main content

The official SWE-bench package - a benchmark for evaluating LMs on software engineering

Project description

SWE-bench PyPI Package

WebsiteGitHubPaper

Code repository that holds the latest version of the SWE-bench PyPI package.

To create/update the PyPI package, follow these steps:

  1. Edit create_src.py such that PATH_TO_SWE_BENCH and PATH_TO_PYPI point to the correct SWE-bench and PyPI repositories on your local machine.
  2. Run create_src.py to update src/swebench with the latest code from princeton-nlp/SWE-bench and creates empty __init__.py files in each folder.
  3. Update __version__ + edit the src/swebench/__init__.py if you would like to expose functions from SWE-bench that were not available in the previous version.
  4. Edit import statements as needed to resolve dependencies within SWE-bench (add swebench.<file>. prefix).
  5. Run ./build_deploy.sh to build the PyPI distribution and update PyPI with the latest package.

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

swebench-0.4.8.tar.gz (38.4 kB view hashes)

Uploaded Source

Built Distribution

swebench-0.4.8-py3-none-any.whl (46.6 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page