gdexws
Service tools for data processing and management.
Installation
Release version (PyPI)
Install the latest published release:
pip install gdexws
Upgrade an existing install to the newest release:
pip install --upgrade gdexws
Runtime dependencies (netCDF4, httpx) are installed automatically.
Developer version (GitHub)
Clone the repository and install gdexws in editable mode, so changes to the source take effect without reinstalling:
git clone https://github.com/NCAR/gdex-web-services.git
cd gdex-web-services
pip install -e ./gdexws
The version is derived from the repository's git tags, so install from a git clone (not a copied folder) to get a meaningful version number.
HPC setup and continuous update
The HPC environment that PBS jobs use lives at /glade/u/home/gdexdata/gdexwsenv. The packages it needs are listed in requirements_hpc.txt.
- Python: 3.11 from
ncarenv/25.10(/glade/u/apps/casper/25.10/opt/view/bin/python3). - User: run every command below as
gdexdata(sudo -iu gdexdata). - System tools:
setfacl/getfacl(OS packageacl) must be onPATH.
1. Create the virtual environment
python3 -m venv /glade/u/home/gdexdata/gdexwsenv
source /glade/u/home/gdexdata/gdexwsenv/bin/activate # bash/zsh
source /glade/u/home/gdexdata/gdexwsenv/bin/activate.csh # csh/tcsh
2. Install the packages
/glade/u/home/gdexdata/gdexwsenv/bin/python -m pip install -r https://raw.githubusercontent.com/NCAR/gdex-web-services/main/gdexws/requirements_hpc.txt
3. Install the setuid wrappers
These let specialists run tools such as gdexcp as gdexdata, so copied files get the right owner and permissions.
pywrapper-install -c # compile pywrapper (owned by gdexdata, setuid, mode 4750)
pywrapper-install --pgstart --username gdexdata # create pgstart_gdexdata
pywrapper-install -l all # create the dsarch/gdexcp symlinks through pgstart_gdexdata
Daily automatic upgrade (gdexws only)
Add this to the gdexdata crontab to upgrade gdexws every day at 03:00:
0 3 * * * (date; /glade/u/home/gdexdata/gdexwsenv/bin/python -m pip install --upgrade --disable-pip-version-check "gdexws>=0.1.1") >> /glade/u/home/gdexdata/gdexwsenv-upgrade.log 2>&1
Check /glade/u/home/gdexdata/gdexwsenv-upgrade.log for the result of each run.
Manual upgrade (all packages)
When the other packages (rda-python-*) need upgrading, run as gdexdata (sudo -iu gdexdata):
# 1. Upgrade every package to the versions required in requirements_hpc.txt
/glade/u/home/gdexdata/gdexwsenv/bin/python -m pip install --upgrade -r https://raw.githubusercontent.com/NCAR/gdex-web-services/main/gdexws/requirements_hpc.txt
# 2. Rebuild the setuid wrappers (no sudo needed again)
/glade/u/home/gdexdata/gdexwsenv/bin/pywrapper-install -u # recompile pywrapper and reinstall all pgstart_USER binaries
/glade/u/home/gdexdata/gdexwsenv/bin/pywrapper-install -l all # link any new setuid_* program the upgrade added
CLI Commands
All tools run through the single gdexws command: gdexws <tool> [options]. Run gdexws --help to list tools and gdexws <tool> --help for a tool's options. -d/--debug is available on every tool.
Adding a tool: drop a module into tools/ or composers/ that defines DESCRIPTION, add_arguments(parser) and run(args); it is discovered automatically as gdexws <module-name-with-hyphens>.
add-global-meta
Add a global attribute to a netCDF file.
gdexws add-global-meta -f /path/to/file.nc -n "attribute_name" -v "attribute_value"
create-exchange-dir
Create a directory directly under /glade/campaign/collections/gdex/data/exchange and grant an HPC user read/write access (rwx, plus a default ACL so new contents inherit it) via POSIX ACLs. The directory's group (decs, which includes gdexdata) also gets rwx the same way, so copy-files/gdexcp can write into it; the directory is setgid (mode 2775) so new contents stay in that group, and others can read (r-x). Requires setfacl/getfacl.
gdexws create-exchange-dir --username <username> --dirname <dirname>
The directory name must be a single path component ([A-Za-z0-9._-], starting with a letter or digit), the user must exist on the machine (and not be uid 0), and the command fails if the directory already exists. If the ACL cannot be applied and verified, the new directory is removed.
Development
From the repository root, with the developer version installed:
pip install pytest
pytest tests/gdexws_tests -v
Metadata
Release files for gdexws 0.1.8
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| gdexws-0.1.8.tar.gz | 33.9 kB | With provenance | Details for gdexws-0.1.8.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| gdexws-0.1.8-py3-none-any.whl | Python 3 | none | any | Details for gdexws-0.1.8-py3-none-any.whl |
Total release size: 67.0 kB
Release files / gdexws-0.1.8.tar.gz
| Download URL | gdexws-0.1.8.tar.gz |
|---|---|
| Size | 33.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
c508a1377fab919bb60f8e9be41711f4e7be0dda727007a80be0aab1167a621f
|
|
BLAKE2b-256 checksum How to use checksums |
8d5562edb60ef02b5bb628a8ac97e7ca553f4f113eb518e5b4981ffcf4b29711
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.
Transparency logRelease files / gdexws-0.1.8-py3-none-any.whl
| Download URL | gdexws-0.1.8-py3-none-any.whl |
|---|---|
| Size | 33.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
2ddc100fc0acf0219985494ee7f9379d345c3d30b41b6ccda17dbc76c91a607c
|
|
BLAKE2b-256 checksum How to use checksums |
71b3b6eb217db8f872e8c0f8b24a9021ce0019139aaf50dd7ffc61da18ce28aa
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.
Transparency logSecurity for gdexws 0.1.8
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for gdexws-0.1.8.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 18, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.