dmwmclient

DMWM client

Hacks and scripts to use in CMS DMWM dev/ops.

Release installation

pip3 install dmwmclient

Usage

Three ways to interact with this package are envisaged:

python script
dmwm command-line interface
jupyter notebook

CLI example

Here we use the dmwm shell command to launch an IPython shell with the various API clients instantiated. We make a few simple requests to datasvc and reqmgr.

$ dmwm shell
Python 3.7.6 (default, Dec 30 2019, 19:38:36)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.10.2 -- An enhanced Interactive Python. Type '?' for help.


Local variables: client (<dmwmclient.Client object at 0x111172810>)

In [1]: df = await client.datasvc.blockreplicas(dataset='/EGamma/Run2018D-22Jan2019-v2/AOD')

In [2]: df
Out[2]:
      replica.bytes  replica.files       replica.node replica.time_create  ... files                                               name        id is_open
0      859056433305            135    T1_US_FNAL_Disk 2019-03-16 15:12:37  ...   135  /EGamma/Run2018D-22Jan2019-v2/AOD#57b25659-3e2...  10992784       n
1      859056433305            135    T1_RU_JINR_Disk 2019-03-02 12:54:55  ...   135  /EGamma/Run2018D-22Jan2019-v2/AOD#57b25659-3e2...  10992784       n
2      859056433305            135     T1_DE_KIT_Disk 2019-03-09 12:05:02  ...   135  /EGamma/Run2018D-22Jan2019-v2/AOD#57b25659-3e2...  10992784       n
3      859056433305            135     T1_US_FNAL_MSS 2019-03-29 23:29:13  ...   135  /EGamma/Run2018D-22Jan2019-v2/AOD#57b25659-3e2...  10992784       n
4      859056433305            135  T1_US_FNAL_Buffer 2019-04-01 10:02:04  ...   135  /EGamma/Run2018D-22Jan2019-v2/AOD#57b25659-3e2...  10992784       n
...             ...            ...                ...                 ...  ...   ...                                                ...       ...     ...
1495  1249881194222            201    T1_RU_JINR_Disk 2019-03-12 01:04:44  ...   201  /EGamma/Run2018D-22Jan2019-v2/AOD#f5d3628e-1c7...  11021144       n
1496  1249881194222            201    T1_US_FNAL_Disk 2019-03-16 15:12:37  ...   201  /EGamma/Run2018D-22Jan2019-v2/AOD#f5d3628e-1c7...  11021144       n
1497  1249881194222            201     T1_DE_KIT_Disk 2019-03-08 00:38:51  ...   201  /EGamma/Run2018D-22Jan2019-v2/AOD#f5d3628e-1c7...  11021144       n
1498  1249881194222            201     T1_US_FNAL_MSS 2019-03-29 23:29:13  ...   201  /EGamma/Run2018D-22Jan2019-v2/AOD#f5d3628e-1c7...  11021144       n
1499  1249881194222            201  T1_US_FNAL_Buffer 2019-04-01 06:08:03  ...   201  /EGamma/Run2018D-22Jan2019-v2/AOD#f5d3628e-1c7...  11021144       n

[1500 rows x 16 columns]

In [3]: df.groupby('replica.node').sum()
Out[3]:
                     replica.bytes  replica.files  replica.node_id
replica.node
T1_DE_KIT_Disk     215484127440685          33550           546300
T1_RU_JINR_Disk    215484127440685          33550           523500
T1_US_FNAL_Buffer  215484127440685          33550             2700
T1_US_FNAL_Disk    215484127440685          33550           534300
T1_US_FNAL_MSS     215484127440685          33550             3000

In [4]: df = await client.reqmgr.transitions(outputdataset='/EGamma/Run2018D-22Jan2019-v2/AOD')

In [5]: df[df['current']]
Out[5]:
                                          requestname  current               mask  ...           Status                                                 DN          UpdateTime
7   vlimant_ACDC0_Run2018D-v1-EGamma-22Jan2019_102...     True  RequestTransition  ...       closed-out  /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=vl... 2019-03-27 13:14:10
17  prebello_Run2018D-v1-EGamma-22Jan2019_1025p1_1...     True  RequestTransition  ...  normal-archived  /DC=ch/DC=cern/OU=computers/CN=dmwm/cmsweb.cer... 2019-04-08 14:34:20

[2 rows x 7 columns]

Developer installation:

git clone git@github.com:nsmith-/dmwmclient.git
cd dmwmclient
pip3 install -e .[dev]
# do some development
black dmwmclient
flake8 dmwmclient

Please run the flake8 linter and black code formatter before committing.

Metadata

Release files for dmwmclient 0.0.4

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for dmwmclient 0.0.4
File	Size	Uploaded
dmwmclient-0.0.4.tar.gz	13.7 kB	Apr 6, 2020	Details

Built distribution (wheel)

Table of built distributions (wheels) for dmwmclient 0.0.4
File	Interpreter	ABI	Platform	Reset
dmwmclient-0.0.4-py3-none-any.whl 17.0 kB Apr 6, 2020	Python 3	none	any	Details

Total release size: 30.7 kB

Release files / dmwmclient-0.0.4.tar.gz

Download URL	dmwmclient-0.0.4.tar.gz
Size	13.7 kB
Tags	Source
SHA-256 checksum How to use checksums	`27eefe226c80005ca2c41af1eba847dd2edf751e951fe957994198f2057a7af8`
BLAKE2b-256 checksum How to use checksums	`34866c02e618bd41845c6a2352ca30fbaa8a6874ecd387d64775f325ac3ae5f0`
Upload date	Apr 6, 2020
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/3.1.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.7.6`

Release files / dmwmclient-0.0.4-py3-none-any.whl

Download URL	dmwmclient-0.0.4-py3-none-any.whl
Size	17.0 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`55c145643436ec75f801aa54373e4ef4bf3cd8dcea4d4fdcc10dac2d02633613`
BLAKE2b-256 checksum How to use checksums	`f8dd3d02f8cfd14caba320b4959b9797fabee1cb125a3b495be454f00a0175a1`
Upload date	Apr 6, 2020
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/3.1.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.7.6`