pantsbuild.pants 2.17.0rc5

The ergonomic and hermetic software build system for Python, Java, Scala, Go, and Shell. Pants lets you fearlessly scale up your codebase.

Pants is an Apache2 licensed build tool written in Python and Rust.

The latest documentation can be found at pantsbuild.org.

2.17.x Release Series

What's New

Individuals and companies can now sponsor Pants financially.

Pants is an open-source project that is not owned or controlled by any one company or organization, and does incur some expenses. These expenses are managed by Pants Build, a non-profit that was established for this purpose. This non-profit's only source of revenue is sponsorship by individuals and companies that use Pants.

We offer formal sponsorship tiers for companies, as well as individual sponsorships via GitHub.

Overall

The pants launcher binary (aka scie-pants) is now the recommended way to run Pants. Use of a ./pants shell script in a repository is deprecated.

The pants launcher has numerous benefits, such as:

• automatically download and maintain a Python distribution for running Pants
• run pants from any directory within a repository and not just the build root
• far less overhead when starting commands (up to 4-5×)

See the Pants installation instructions for additional information on how to migrate to the pants launcher binary.

Related to this, pants' distribution model is changing, and 2.17 will be the last version where:

• the pantsbuild.pants package is published to PyPI. Use the pants_requirements target for installing the requisite packages for plugins, rather than python_requirement or similar.
• the universal (cross-platform) pants PEX is published to GitHub Releases. For future releases, smaller per-platform PEXes will be attached instead.

Performance

As mentioned above, the new launcher binary reduces start-up overhead.

Large files are now cached as standalone files on disk, rather than stored directly in the internal database pants uses for cache (but still under the ~/.cache/pants/lmdb_store directory). This improves performance for manipulating these files, especially when the cache directory and sandbox temporary directory are on the same file system and thus allow hard-linking.

Backends

Python

Python dependencies are now analyzed via an intrinsic rule implemented in Rust for increased performance over the prior Python implementation of the rule. See #18854 for discusion of the performance improvements.

Exported mutable virtualenvs can now include PEP-660 editable installs of python_distribution targets. To enable this feature for a resolve, add that resolve's name to the [export].py_editables_in_resolves list in pants.toml. See #18639 for details on Pants' PEP-660 implementation.

The default pip Pants uses under the hood has changed from 20.3.4 to 23.1.2. This newer pip has better dependency resolution performance in many cases, but may give different results than the earlier pip. Of course those results will still be correct in the sense that they will be compatible with your requirements and constraints.

The python_awslambda or python_google_cloud_function targets now support a new 'zip' layout, as recommended by the cloud vendors. This layout gives smaller packages and faster cold starts than the existing Lambdex layout, and will become the default in 2.18. See the docs for python_awslambda and for python_google_cloud_function.

The default value for the [python].interpreter_constraints option was deprecated in 2.16 and is now removed: pants.toml must provide this option. We recommend constraining to a single interpreter minor version if you can, for instance: interpreter_constraints = ['==3.11.*']. See the Interpreter Compatibility docs for more details.

Docker

The Docker backend now supports authenticating with registries via the DOCKER_HOST, DOCKER_CONFIG and DOCKER_CERT_PATH environment variables.

New: Javascript (experimental!)

Pants now has very early experimental support for Javascript thanks to ongoing efforts by Tobias Nilsson and Theo Ribeiro. Please note that the Javascript backend is still incomplete, is being actively developed, and probably has critical bugs. Despite that, the Pants maintainers would appreciate any feedback from the community to help guide our development efforts.

Supported goals are:

• test: Letting you run tests via runners installed with the package manager of your choice.
• package goal, either to run a customized package.json script that produces an artifact via node_build_script, or to pack a tarball for npm-registry publication via the npm_distribution target.
• tailor: Generates build file targets for *.js, *.test.js and package.json files.
• generate-lockfile: Creates the lockfile in the format matching a projects package manager.

The backend supports all package managers provided by corepack. yarn@v2 and PlugNPlay is not supported.

Enable the pants.backend.experimental.javascript backend to try out this support. Please file issues for any issues encountered, and follow along the stabilization ticket on github.

New: Taplo TOML Formatter

Pants now supports the Taplo TOML formatter. Enable the pants.backend.tools.taplo backend to add this support.

Plugin API Changes

Rules should now request output types which do not need an input type via the new one-argument form of Get. For example, rules can now write await Get(ChosenLocalEnvironmentName). Certain request types which exists only to work around the previous lack of such synax are now deprecated in favor of the one argument Get() form.

The PythonBinary type is now deprecated, use PythonBuildStandalone instead.

2.17.0rc5 (Aug 26, 2023)

Bug Fixes

• Remove tensorboard mapping from our default mapping (Cherry-pick of #19673) (#19677)

Documentation

• Backport to 2.17: docs: set EnvironmentBehavior in plugin Goal examples (#19590) (#19664)

2.17.0rc4 (Aug 23, 2023)

User API Changes

• Require scie-pants 0.9.2 or newer, for new distribution model (Cherry-pick of #19654) (#19655)

• Add Python 3.12 to the default interpreter universe (Cherry-pick of #19641) (#19651)

Bug Fixes

• Have Pants create the immutable_inputs base parent(s) (Cherry-pick of #19646) (#19652)

• Use hardlinking when possible for mypy cache (Cherry-pick of #19633) (#19650)

• Undo pyenv install marking files as RO (Cherry-pick of #19645) (#19648)

• Make mypy cache move truly atomic (Cherry-pick of #19610) (#19615)

• Use named cache for Docker-env PBS (Cherry-pick of #19611) (#19616)

• Add path to cache key for Rust dep inference, for relative imports (cherry-pick of #19630) (#19640)

Documentation

• Backport to 2.17: Fix dead doc links (#19546) (#19659)

• Additional mentions and tweaks in 2.17's "What's new" (Cherry-pick of #19629) (#19642)

• docs: fix code examples for setting-up-ide and interpreter-compatibility docs (Cherry-pick of #19624) (#19639)

• docs: fix markdown inconsistencies (Cherry pick of #19363) (#19622)

• docs: add 2.16 entry to changelog (Cherry-pick of #19589) (#19597)

2.17.0rc3 (Aug 13, 2023)

User API Changes

• Upgrade the default pip to v23.1.2. (Cherry-pick of #19538) (#19541)

Bug Fixes

• Scala: support Enums in dependency inference (Cherry pick of #19588) (#19593)

• Fix the new tool lockfile deprecation message. (#19565)

• Fix golangci-lint runner script (Cherry-pick of #19015) (#19473)

Documentation

• Initial draft of What's New for v2.17.x (Cherry-pick of #19168) (#19595)

• Add ruff, adjust pyugprade, autoflake in docs (Cherry-pick of #19555) (#19576)

• Catch google artifact registry gotcha (Cherry-pick of #19568) (#19570)

• Stop referring to ~/.cache/pants/setup in CI docs (Cherry-pick of #19554) (#19561)

• Visibility: documentation fixes and clarifications. (Cherry-pick of #19438) (#19494)

2.17.0rc2 (Jul 14, 2023)

Bug Fixes

• Visibility: fix issue with using target type in verbose rule selector. (Cherry-pick of #19437) (#19456)

• Use the "package_name" variable to fix an UnboundLocalError. (Cherry-pick of #19433) (#19453)

• Fix helm documentation example (Cherry-pick of #19271) (#19313)

Documentation

• Tweak getting-help docs (Cherry-pick of #19418) (#19419)

• Refine run_shell_command docs (Cherry-pick of #19413) (#19421)

• Add a sponsorships page to the docsite. (Cherry-pick of #19412) (#19416)

• Get doc titles from the in-repo markdown files (Cherry-pick of #19311) (#19321)

• docs: layout tweak of caching info, remove toolchain :( and additional info about bazel-remote-cache (Cherry-pick of #19299) (#19329)

2.17.0rc1 (Jun 08, 2023)

Plugin API Changes

• Deprecate using PythonBinary (Cherry-pick of #19209) (#19222)

Bug Fixes

• Improve handling of additional files in Helm unit tests (Cherry-pick of #19263) (#19268)

• Add taplo to the release (Cherry-pick of #19258) (#19259)

• Handle from foo import * wildcard imports in Rust dep inference parser (Cherry-pick of #19249) (#19255)

• Fix secondary ownership warning semantics (Cherry-pick of #19191) (#19224)

2.17.0rc0 (Jun 01, 2023)

User API Changes

• Deprecate the version/extra_requirements options on python tools. (Cherry-pick of #19204) (#19211)

• Terraform: Hashes for MacOS binaries have changed (Cherry-pick of #19004) (#19136)

• Implement layout="zip" for Lambda/GCF, deprecating lambdex (Cherry-pick of #19076) (#19120)

• Deprecate [mypy].extra_type_stubs and its lockfile. (Cherry-pick of #19084) (#19087)

• Bring the tool --export option deprecation forward. (Cherry-pick of #19079) (#19083)

Bug Fixes

• Discover Scala REPL main class and coordinates (Cherry-pick of #19189) (#19203)

• Scrub the Pyenv install dir before attempting an install. (Cherry-pick of #19193) (#19197)

• Avoid extra . in from .. import parent imports in Rust Python dep inference (Cherry-pick of #19175) (#19177)

• Don't try and garbage collect tempfiles (Cherry-pick of #19103) (#19113)

• If a shebang already exists when adding a preamble, preserve it. (Cherry-pick of #19133) (#19138)

• Workaround botocore bug in S3 URL Handler backend (Cherry-pick of #19056) (#19111)

Documentation

• Add tobni to Maintainers (Cherry-pick of #19195) (#19207)

• Adjust FaaS docs for layout = "zip" (Cherry-pick of #19180) (#19198)

• Add botocore-a-la-carte reference to s3 URL Handler (Cherry-pick of #19055) (#19109)

• add bazel-remote to list of compatible remote caches (Cherry-pick of #19041) (#19115)

• Fix docs link to pants_from_sources script. (Cherry-pick of #19088) (#19091)

2.17.0a1 (May 19, 2023)

Due to infrastructure issues, 2.17.0a1 is a second attempt at publishing 2.17.0a0.

2.17.0a0 (May 18, 2023)

NOTE: 2.17.0a0 was not released to PyPI due to infrastructure issues.

New Features

• Add a workunit logging plugin (#18984)

• javascript: Add yarn@v1 support (#18928)

• javascript: Support Node.js subpath imports (#18934)

• Add Taplo toml formatter (#18865)

• Include the Pants native client in released wheels (#18957)

• Extract Python dependencies in an intrinsic (#18854)

User API Changes

• Do not publish wheels for all SHAs, and publish to PyPI from Github Actions. (#19028)

• Support Python requirement target addrs in tool requirements. (#19014)

• javascript: Add an explicit npm_distribution target instead of coupling to node_package (#18925)

• Shorten the deprecation for export without --resolve. (#19000)

• Upgrade Pex to 2.1.137. (#19003)

• Remove the underlying implementation of anonymous-telemetry (#18952)

Plugin API Changes

• Port Django migrations inference away from PythonDependencyVisitorRequest (#19008)

• fmtrequest -> abstractfmtrequest (#18935)

Bug Fixes

• Use {bin_name()} run as $0 in run_shell_command (#19020)

• Make DigestSubset symlink-aware (#18963)

• javascript: Do not add literal {chroot} to pnpm cache path (#18921)

Documentation

• Clarify how to set $0 properly in run_in_shell_command (#19019)

• Docs tweaks for 2.16.x (#19009)

• Fix example first_party_dependency_version_scheme in docs (#18993)

2.17.0.dev5 (May 08, 2023)

New Features

• javascript: Implement dependency inference opt-out and improve resillience (#18931)

• javascript: Add support for the pnpm package manager (#18864)

• Eagerly clean running nodes rather than interrupting them. (#18855)

• javascript: Nodejs tests batch support (#18742)

• New API type: CurrentExecutingGoals. (#18788)

User API Changes

• javascript: Support package goal in node_build_script target (#18926)

• Deprecate running Pants outside scie-pants. (#18833)

• Upgrade Pex to 2.1.135. (#18876)

Plugin API Changes

• Remove redundant PythonToolBase/JvmToolBase features. (#18805)

Bug Fixes

• Ensure packaged artefacts are fully replaced in dist/ (#18930)

• Use a set when calculating resolve names (#18939)

• Use a set when calculating resolve names (#18939)

• javascript: Handle package.json placed at the root of the repository (#18919)

• Actually run deprecated targets fixer (#18860)

• Remove existing entry if any when materialising symlink (#18873)

• Ensure non-ambiguous args/env vars injection into PEXes (#18861)

• Handle workdir="."/default properly in run_shell_command (#18840)

• Write adhoc_tool(stdout/stderr="...") relative to workdir, support absolute paths (#18814)

• Patch StreamingWorkunitContext.get_expanded_specs(). (#18713)

• help backends: Only list enabled experimental backends unless advanced help. (#18821)

• Use build root as workdir for shell_command(workdir="") (#18813)

• Include pants.backend.url_handlers.s3 in pants distribution. (#18826)

Performance

• Optimize Target and FieldSet operations (#18917)

• Triage some noqa: PNT30 await-in-loops (#18831)

Documentation

• Update the identity of the response team for CoC violation reporting. (#16410)

• Fix adhoc output_... example, document run_shell_command workdir changes (#18894)

• Add missing backtick in help string (#18907)

• Add Salesforce to the users list (#18891)

• update troubleshooting document about self-hosted action runner (#18900)

• Credit the Works on ARM program for our CI resources. (#18892)

• feat: update installation.md (#18836)

• Remove reference to readme.com as the way to edit documentation (#18863)

• Fix library name typo in third party example (#18862)

• Update mypy and pytest documentation to refer to install_from_resolve (#18791)

• Include symbols as topic in usage help. (#18825)

• docs: reorder contributor steps, rustup is required for git hooks install to work smoothly (#18828)

2.17.0.dev4 (Apr 23, 2023)

New Features

• javascript: Manage npm version with corepack (#18748)

• python_distribution editable installs in exports (#18639)

User API Changes

• Default to using the entire tool lockfile. (#18793)

• Upgrade Pex to 2.1.134. (#18785)

• Use the resolve's ICs for tools. (#18776)

• Use Self backport in BinaryPath factory method. (#18763)

• Deprecate "Secondary Ownership" semantics (#18737)

• Expand versions range for ruff + upgrade it (#18707)

Plugin API Changes

• Allow BinaryPaths to close over their type. (#18760)

• Set a central default ICs for tools. (#18714)

Bug Fixes

• The OverridesField.compute_value() must return a hashable value. (#18787)

• Dedup graph errors. (#18770)

• Flush metadata to disk before renaming immutable store files (#18768)

• Clear FSDB materialization cache when removeing a file (#18747)

• Use address as stable key when batching field sets in lint/test (#18725)

• Fix repr for Fields that don't define default. (#18719)

• Add ruff lint rules (#18716)

• javascript: Use binary shims in nodejs sandboxes (#18682)

Documentation

• Improve error location for invalid targets/target fields. (#18366)

• Add info on how to configure Docker Desktop for using docker_environment (#18786)

• Update linter documentation to refer to install_from_resolve (#18781)

• docs: Fix typo on requirements-overrides code snippet (#18750)

• Edit Python publishing docs for clarity. (#18718)

2.17.0.dev3 (Apr 10, 2023)

New Features

• Helm: Add support for DNS lookups in Chart templates (#18670)

• Include global Git ignore file and .git/info/exclude in pants_ignore to ignore for file watching (#18649)

User API Changes

• Remove deprecated default ICs. (#18627)

Plugin API Changes

• Move/Rename: goals/setup_py.py -> {util_rules,goals}/package_dists.py + subsystems/setup_py_generation.py (#18702)

• Deprecate some Request types in favor of Get() with only one arg (#18655)

• Add support for Get(MyType) syntax with no input args (#18648)

Bug fixes

• Adds a fully-qualified path to a Python interpreter for pex_binary RunRequests (#18699)

• Remove implicit conversion from std::io::Error to StoreError (#18696)

• Add timeout to remote store calls, and adjust name of cache timeout. (#18695)

• Makes adhoc_tool actually respect the extra_env_vars field; adds tests (#18692)

• doc: Describe relevant error modes during rule compilation (#18663)

• Bump minimum supported version of Pex to v2.1.129. (#18678)

• Fix JVM resource JAR creation on older Mac OSes (#18673)

• javascript: Do not read none mapped package.json files in javascript rules (#18523)

Documentation

• Add PEP 621 pyproject.toml file docs (#18672)

• Document AWS CDK issue that multiple users have encountered. (#18656)

• Error when goals used without relevant backends activated (#18420)

• Improve lockfile consumption error message. (#18638)

• Remove a python-specific tip from the general prereqs page. (#18637)

• Make UUID message clearer (#18561)

2.17.0.dev2 (Mar 31, 2023)

New Features

• javascript: Test goal with package manager installed test runner support (#18554)

• expose PANTS_VERSION and make it string-comparable (#18573)

User API Changes

• Upgrade Pex to 2.1.131. (#18626)

Bug fixes

• Ensure that sandboxed processes exit before their sandboxes are cleaned up (#18632)

• Plumb entire lockfile/internal only code through create_pex_from_targets (#18622)

• narrow pex_binary files warning (#18619)

• Fix PexPEX invocations (#18610)

• Improve debug information for docker auth, and expose an environment variable. (#18599)

Performance

• Only materialize immutable files once per process (#18600)

Documentation

• Fix a string that should have been an f-string (#18608)

• Fix ./pants -> pants in some docs and help strings. (#18596)

2.17.0.dev1 (Mar 24, 2023)

New Features

• engine: add an option to clear the destination directory when writing digest to disk (#18534)

• javascript: NodeJS bootstrapping via binary paths, PATH, asdf or nvm (#18520)

• javascript: Initial NPM support (#18326)

• Always provide Python-for-Pants-scripts (#18433)

User API Changes

• Upgrade Pex to 2.1.130. (#18576)

• Deprecate old-style custom tool lockfiles. (#18494)

• Let a user resolve shadow a tool lockfile of the same name (#18481)

Bug fixes

• Ensure that a symlinked exported venv exists and is valid. (#18575)

• Fail fast for attempts to use test --debug with a docker environment (#18560)

• Declare ruff outputs as files. (#18551)

• Add support for Docker registry auth (#18541)

• Fix Poetry req synthesis for URLs with markers. (#18535)

• Re-enable hardlinking using fclonefileat on macOS. (#18501)

Performance

• Materialize "large" files in a new store location and hardlink them in sandboxes (#18153)

• Finish parallelizing materialization of Process inputs (#18469)

Documentation

• First pass at adhoc_tool documentation (#18531)

• Add Tobias Nilsson to team page (#18571)

• docs: fix an f-string in help (#18539)

• Clarify that experimental_ shell functions are deprecated and replaced (#18529)

• Update docs on running pants from sources with scie-pants. (#18513)

• Add ExoFlare to user list. (#18496)

• Fix malformed pantsbuild/example-python link in documentation (#18484)

• update scie-jump cache directory reference (#18485)

• Expand lockfile documentation to its own page. (#18471)

2.17.0.dev0 (Mar 10, 2023)

New Features

• Expose environment information on process metadata (#18458)

• Support running pants from anywhere in the project. (#18412)

User API Changes

• Audit and update requirements for Python tools. (#18431)

• Upgrade Pex to 2.1.128. (#18453)

Bug fixes

• Preserve environment variables in pantsd to allow Docker auth. (#18465)

• Only expose the equiv of from typing import * in prelude files (#18452)

• Support type hints in prelude files. (#18442)

• Synthesized lockfile targets should never err on missing lockfiles. (#18406)

• Fixes to visualisations for xdot (#18421)