Skip to main content

gdexws

Service tools for data processing and management.

Installation

Release version (PyPI)

Install the latest published release:

pip install gdexws

Upgrade an existing install to the newest release:

pip install --upgrade gdexws

Runtime dependencies (netCDF4, httpx) are installed automatically.

Developer version (GitHub)

Clone the repository and install gdexws in editable mode, so changes to the source take effect without reinstalling:

git clone https://github.com/NCAR/gdex-web-services.git
cd gdex-web-services
pip install -e ./gdexws

The version is derived from the repository's git tags, so install from a git clone (not a copied folder) to get a meaningful version number.

HPC setup and continuous update

The HPC environment that PBS jobs use lives at /glade/u/home/gdexdata/gdexwsenv. The packages it needs are listed in requirements_hpc.txt.

  • Python: 3.11 from ncarenv/25.10 (/glade/u/apps/casper/25.10/opt/view/bin/python3).
  • User: run every command below as gdexdata (sudo -iu gdexdata).
  • System tools: setfacl/getfacl (OS package acl) must be on PATH.

1. Create the virtual environment

python3 -m venv /glade/u/home/gdexdata/gdexwsenv
source /glade/u/home/gdexdata/gdexwsenv/bin/activate        # bash/zsh
source /glade/u/home/gdexdata/gdexwsenv/bin/activate.csh    # csh/tcsh

2. Install the packages

/glade/u/home/gdexdata/gdexwsenv/bin/python -m pip install -r https://raw.githubusercontent.com/NCAR/gdex-web-services/main/gdexws/requirements_hpc.txt

3. Install the setuid wrappers

These let specialists run tools such as gdexcp as gdexdata, so copied files get the right owner and permissions.

pywrapper-install -c                              # compile pywrapper (owned by gdexdata, setuid, mode 4750)
pywrapper-install --pgstart --username gdexdata   # create pgstart_gdexdata
pywrapper-install -l all                          # create the dsarch/gdexcp symlinks through pgstart_gdexdata

Daily automatic upgrade (gdexws only)

Add this to the gdexdata crontab to upgrade gdexws every day at 03:00:

0 3 * * * (date; /glade/u/home/gdexdata/gdexwsenv/bin/python -m pip install --upgrade --disable-pip-version-check "gdexws>=0.1.1") >> /glade/u/home/gdexdata/gdexwsenv-upgrade.log 2>&1

Check /glade/u/home/gdexdata/gdexwsenv-upgrade.log for the result of each run.

Manual upgrade (all packages)

When the other packages (rda-python-*) need upgrading, run as gdexdata (sudo -iu gdexdata):

# 1. Upgrade every package to the versions required in requirements_hpc.txt
/glade/u/home/gdexdata/gdexwsenv/bin/python -m pip install --upgrade -r https://raw.githubusercontent.com/NCAR/gdex-web-services/main/gdexws/requirements_hpc.txt

# 2. Rebuild the setuid wrappers (no sudo needed again)
/glade/u/home/gdexdata/gdexwsenv/bin/pywrapper-install -u       # recompile pywrapper and reinstall all pgstart_USER binaries
/glade/u/home/gdexdata/gdexwsenv/bin/pywrapper-install -l all   # link any new setuid_* program the upgrade added

CLI Commands

All tools run through the single gdexws command: gdexws <tool> [options]. Run gdexws --help to list tools and gdexws <tool> --help for a tool's options. -d/--debug is available on every tool.

Adding a tool: drop a module into tools/ or composers/ that defines DESCRIPTION, add_arguments(parser) and run(args); it is discovered automatically as gdexws <module-name-with-hyphens>.

add-global-meta

Add a global attribute to a netCDF file.

gdexws add-global-meta -f /path/to/file.nc -n "attribute_name" -v "attribute_value"

create-exchange-dir

Create a directory directly under /glade/campaign/collections/gdex/data/exchange and grant an HPC user read/write access (rwx, plus a default ACL so new contents inherit it) via POSIX ACLs. The directory's group (decs, which includes gdexdata) also gets rwx the same way, so copy-files/gdexcp can write into it; the directory is setgid (mode 2775) so new contents stay in that group, and others can read (r-x). Requires setfacl/getfacl.

gdexws create-exchange-dir --username <username> --dirname <dirname>

The directory name must be a single path component ([A-Za-z0-9._-], starting with a letter or digit), the user must exist on the machine (and not be uid 0), and the command fails if the directory already exists. If the ACL cannot be applied and verified, the new directory is removed.

Development

From the repository root, with the developer version installed:

pip install pytest
pytest tests/gdexws_tests -v

Metadata

Release files for gdexws 0.1.8

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for gdexws 0.1.8
File Size Uploaded Provenance Actions
gdexws-0.1.8.tar.gz 33.9 kB With provenance Details for gdexws-0.1.8.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for gdexws 0.1.8
File Interpreter ABI Platform Actions
gdexws-0.1.8-py3-none-any.whl Python 3 none any Details for gdexws-0.1.8-py3-none-any.whl

Total release size: 67.0 kB

Release files / gdexws-0.1.8.tar.gz

Download URL gdexws-0.1.8.tar.gz
Size 33.9 kB
Tags Source
SHA-256 checksum
How to use checksums
c508a1377fab919bb60f8e9be41711f4e7be0dda727007a80be0aab1167a621f
BLAKE2b-256 checksum
How to use checksums
8d5562edb60ef02b5bb628a8ac97e7ca553f4f113eb518e5b4981ffcf4b29711
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

Release files / gdexws-0.1.8-py3-none-any.whl

Download URL gdexws-0.1.8-py3-none-any.whl
Size 33.1 kB
Tags Python 3
SHA-256 checksum
How to use checksums
2ddc100fc0acf0219985494ee7f9379d345c3d30b41b6ccda17dbc76c91a607c
BLAKE2b-256 checksum
How to use checksums
71b3b6eb217db8f872e8c0f8b24a9021ce0019139aaf50dd7ffc61da18ce28aa
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
Yes
Uploaded via twine/7.0.0 CPython/3.13.16

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

Security for gdexws 0.1.8

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

Full provenance

Both files in this release have been uploaded with provenance metadata.

Select from 2 files

Attestation for gdexws-0.1.8.tar.gz

PyPI Publish Attestation

PyPI verified that this artifact, at this checksum, originated from the publisher listed below.

Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.

Transparency log

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page