Skip to main content

Pandamonium

Cause panda and rucio don't work too good

This tells you if your jobs are done. And stuff like that.

  • No login required
  • Colors! (when you want them)
  • Gets input / output dataset names
  • Works with piping, other unix nice things

Like badges? We have the best badges:

GitHub Project DOI

PyPI version Supported Python versions

pre-commit.ci status

Installation

Install from PyPI

You can install pandamonium from PyPI into any Python environment by running

python -m pip install pandamonium

Install oldschool way

If you can't get pip working you can just put the scripts you need in your path. Note that this is deprecated (but still works for now).

  1. Clone the repository
git clone git@github.com:dguest/pandamonium.git
  1. Add the directory to your PATH.

Maybe with something like the following in your .bashrc

# Add pandamonium to PATH
if [ -d "your/path/stuff/goes/here/pandamonium" ]; then
    PATH="your/path/stuff/goes/here/pandamonium${PATH:+:${PATH}}"
    export PATH
fi

Use

  1. Run pandamon
pandamon [user.<your user name>]
  1. See the output of your current GRID jobs with pretty colors!

You can add more of the task name if you want, and use wildcards (*). Wildcards are automatically appended to names that don't end in * or /.

Without any arguments the task name defaults to user.$RUCIO_ACCOUNT*.

Also try pandamon -h.

Other tricks

Get input/output dataset names

> pandamon -s IN <task name>
> pandamon -s OUT <task name>

Filter by user name

This is useful if you're running with group privileges. Set the environment variable GRID_USER_NAME to your full user name (the one that shows up on the top of the bigpanda page). Or specify one with --user.

Find input datasets for jobs in the broken state

You can do more useful stuff by piping through standard Unix utilities

> pandamon your.tasks > tasks.txt
> cat tasks.txt | awk '$1 ~ /broken/ {print $2}' | pandamon - -s IN

or (faster)

pandamon your.tasks -i broken -s IN

Filter by taskid range

Use to only display jobs in a specific range. This is useful for when you inevitably submit jobs with wrong parameters that you don't want to retry.

pandamon -r 12000-12100

Read the job user metadata

Now panda supports a userMetadata.json file for additional information in your job. Print it with

pandamon your.tasks -m

See this JIRA ticket where they plan to make it faster.

Additional Technical Information

Deprecation Warning

You can currently just clone the repository and have master work the same way as v0.1 on LXPLUS or ATLAS Connect, but this will be deprecated in the future in favor of installing pandamonium as a Python library.

The motivation for this is that pandamonium does have hard requirements on other libraries, and it is better to fully contain them through the installation of the library through PyPI.

If you really need the old behaviour forever, you can always use

git clone git@github.com:dguest/pandamonium.git --branch v0.1

Install development release from TestPyPI

You can install the latest development release of pandamonium from TestPyPI into any Python virtual environment by running

python -m pip install --extra-index-url https://test.pypi.org/simple/ --pre pandamonium

Note: This adds TestPyPI as an additional package index to search when installing pandamonium specifically. PyPI will still be the default package index pip will attempt to install from for all dependencies.

Notes if working on a remote server

If you are working from a remote server where you do not have control over your Python runtimes (e.g. LXPLUS, ALTAS Connect login nodes) it is recommended that you bootstrap virtualenv and a default Python virtual environment by adding the following to your .bashrc or .bashrc_user

# Ensure local virtualenv setup
if [ ! -f "${HOME}/opt/venv/bin/virtualenv" ]; then
    curl -sL --location --output /tmp/virtualenv.pyz https://bootstrap.pypa.io/virtualenv.pyz
    python /tmp/virtualenv.pyz ~/opt/venv # Change this to python3 if available
    ~/opt/venv/bin/pip install --upgrade pip
    ~/opt/venv/bin/pip install virtualenv
    mkdir -p ~/bin  # Ensure exists if new machine
    ln -s ~/opt/venv/bin/virtualenv ~/bin/virtualenv
fi

# default venv from `virtualenv "${HOME}/.venvs/base"`
if [ -d "${HOME}/.venvs/base" ]; then
    source "${HOME}/.venvs/base/bin/activate"
fi

After that source your .profile or .bash_profile and then if you want to create a default Python virtual environment run

virtualenv "${HOME}/.venvs/base"

You will now be dropped into a virtual environment named base each time you login. The virtual environment is not special in anyway, so you should treat it as you would any other.

Testimonials

"I like colors" -- Chase Schimmin

"I found a bug" -- Danny Antrim (Fixed! Thanks Danny!)

"I tried to use it but it's python 3" -- also Chase

(I added Python 2 support, but pandamonium is also Python 3 compliant)

"I made a merge request. It was approved!" -- Alex

"It needs to use Code style: black" -- Matthew Feickert

(Hey man, whatever floats your boat!)

I'll add other stuff too, if you want.

Metadata

Release files for pandamonium 0.3.1

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for pandamonium 0.3.1
File Size Uploaded Provenance Actions
pandamonium-0.3.1.tar.gz 15.3 kB Without provenance Details for pandamonium-0.3.1.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for pandamonium 0.3.1
File Interpreter ABI Platform Actions
pandamonium-0.3.1-py2.py3-none-any.whl Python 3, Python 2 none any Details for pandamonium-0.3.1-py2.py3-none-any.whl

Total release size: 29.2 kB

Release files / pandamonium-0.3.1.tar.gz

Download URL pandamonium-0.3.1.tar.gz
Size 15.3 kB
Tags Source
SHA-256 checksum
How to use checksums
2ef055113916e85b44a26fd8fb264c0808b74ba1f18a120ecd52856883b8649b
BLAKE2b-256 checksum
How to use checksums
01598944c97586e65d6cbdb13f460bea28cc3053d527e85d38206f3080a81354
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.1 CPython/3.9.13

Release files / pandamonium-0.3.1-py2.py3-none-any.whl

Download URL pandamonium-0.3.1-py2.py3-none-any.whl
Size 14.0 kB
Tags Python 2 Python 3
SHA-256 checksum
How to use checksums
ee298f096b0a39941a04f99f8beab0dc875e33f665d4f2345ca325be3e3cec2c
BLAKE2b-256 checksum
How to use checksums
d5c6b526b54ee507acec4a3fda5f8894200bb0e20178a77356726d24e43bad2a
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.1 CPython/3.9.13

Security for pandamonium 0.3.1

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page