DMWM client
Hacks and scripts to use in CMS DMWM dev/ops.
Release installation
pip3 install dmwmclient
Usage
Three ways to interact with this package are envisaged:
- python script
dmwmcommand-line interface- jupyter notebook
CLI example
Here we use the dmwm shell command to launch an IPython shell with the various API clients instantiated.
We make a few simple requests to datasvc and reqmgr.
$ dmwm shell
Python 3.7.6 (default, Dec 30 2019, 19:38:36)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.10.2 -- An enhanced Interactive Python. Type '?' for help.
Local variables: client (<dmwmclient.Client object at 0x111172810>)
In [1]: df = await client.datasvc.blockreplicas(dataset='/EGamma/Run2018D-22Jan2019-v2/AOD')
In [2]: df
Out[2]:
replica.bytes replica.files replica.node replica.time_create ... files name id is_open
0 859056433305 135 T1_US_FNAL_Disk 2019-03-16 15:12:37 ... 135 /EGamma/Run2018D-22Jan2019-v2/AOD#57b25659-3e2... 10992784 n
1 859056433305 135 T1_RU_JINR_Disk 2019-03-02 12:54:55 ... 135 /EGamma/Run2018D-22Jan2019-v2/AOD#57b25659-3e2... 10992784 n
2 859056433305 135 T1_DE_KIT_Disk 2019-03-09 12:05:02 ... 135 /EGamma/Run2018D-22Jan2019-v2/AOD#57b25659-3e2... 10992784 n
3 859056433305 135 T1_US_FNAL_MSS 2019-03-29 23:29:13 ... 135 /EGamma/Run2018D-22Jan2019-v2/AOD#57b25659-3e2... 10992784 n
4 859056433305 135 T1_US_FNAL_Buffer 2019-04-01 10:02:04 ... 135 /EGamma/Run2018D-22Jan2019-v2/AOD#57b25659-3e2... 10992784 n
... ... ... ... ... ... ... ... ... ...
1495 1249881194222 201 T1_RU_JINR_Disk 2019-03-12 01:04:44 ... 201 /EGamma/Run2018D-22Jan2019-v2/AOD#f5d3628e-1c7... 11021144 n
1496 1249881194222 201 T1_US_FNAL_Disk 2019-03-16 15:12:37 ... 201 /EGamma/Run2018D-22Jan2019-v2/AOD#f5d3628e-1c7... 11021144 n
1497 1249881194222 201 T1_DE_KIT_Disk 2019-03-08 00:38:51 ... 201 /EGamma/Run2018D-22Jan2019-v2/AOD#f5d3628e-1c7... 11021144 n
1498 1249881194222 201 T1_US_FNAL_MSS 2019-03-29 23:29:13 ... 201 /EGamma/Run2018D-22Jan2019-v2/AOD#f5d3628e-1c7... 11021144 n
1499 1249881194222 201 T1_US_FNAL_Buffer 2019-04-01 06:08:03 ... 201 /EGamma/Run2018D-22Jan2019-v2/AOD#f5d3628e-1c7... 11021144 n
[1500 rows x 16 columns]
In [3]: df.groupby('replica.node').sum()
Out[3]:
replica.bytes replica.files replica.node_id
replica.node
T1_DE_KIT_Disk 215484127440685 33550 546300
T1_RU_JINR_Disk 215484127440685 33550 523500
T1_US_FNAL_Buffer 215484127440685 33550 2700
T1_US_FNAL_Disk 215484127440685 33550 534300
T1_US_FNAL_MSS 215484127440685 33550 3000
In [4]: df = await client.reqmgr.transitions(outputdataset='/EGamma/Run2018D-22Jan2019-v2/AOD')
In [5]: df[df['current']]
Out[5]:
requestname current mask ... Status DN UpdateTime
7 vlimant_ACDC0_Run2018D-v1-EGamma-22Jan2019_102... True RequestTransition ... closed-out /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=vl... 2019-03-27 13:14:10
17 prebello_Run2018D-v1-EGamma-22Jan2019_1025p1_1... True RequestTransition ... normal-archived /DC=ch/DC=cern/OU=computers/CN=dmwm/cmsweb.cer... 2019-04-08 14:34:20
[2 rows x 7 columns]
Developer installation:
git clone git@github.com:nsmith-/dmwmclient.git
cd dmwmclient
pip3 install -e .[dev]
# do some development
black dmwmclient
flake8 dmwmclient
Please run the flake8 linter and black code formatter before committing.
Metadata
Release files for dmwmclient 0.0.4
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| dmwmclient-0.0.4.tar.gz | 13.7 kB | Without provenance | Details for dmwmclient-0.0.4.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| dmwmclient-0.0.4-py3-none-any.whl | Python 3 | none | any | Details for dmwmclient-0.0.4-py3-none-any.whl |
Total release size: 30.7 kB
Release files / dmwmclient-0.0.4.tar.gz
| Download URL | dmwmclient-0.0.4.tar.gz |
|---|---|
| Size | 13.7 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
27eefe226c80005ca2c41af1eba847dd2edf751e951fe957994198f2057a7af8
|
|
BLAKE2b-256 checksum How to use checksums |
34866c02e618bd41845c6a2352ca30fbaa8a6874ecd387d64775f325ac3ae5f0
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.7.6
|
Release files / dmwmclient-0.0.4-py3-none-any.whl
| Download URL | dmwmclient-0.0.4-py3-none-any.whl |
|---|---|
| Size | 17.0 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
55c145643436ec75f801aa54373e4ef4bf3cd8dcea4d4fdcc10dac2d02633613
|
|
BLAKE2b-256 checksum How to use checksums |
f8dd3d02f8cfd14caba320b4959b9797fabee1cb125a3b495be454f00a0175a1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/3.1.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.7.6
|
Security for dmwmclient 0.0.4
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.