Skip to main content

client and scripts

Project description

DMWM client

Hacks and scripts to use in CMS DMWM dev/ops.

Release installation

pip3 install dmwmclient

Usage

Three ways to interact with this package are envisaged:

  • python script
  • dmwm command-line interface
  • jupyter notebook

CLI example

Here we use the dmwm shell command to launch an IPython shell with the various API clients instantiated. We make a few simple requests to datasvc and reqmgr.

$ dmwm shell
Python 3.7.6 (default, Dec 30 2019, 19:38:36)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.10.2 -- An enhanced Interactive Python. Type '?' for help.


Local variables: client (<dmwmclient.Client object at 0x111172810>)

In [1]: df = await client.datasvc.blockreplicas(dataset='/EGamma/Run2018D-22Jan2019-v2/AOD')

In [2]: df
Out[2]:
      replica.bytes  replica.files       replica.node replica.time_create  ... files                                               name        id is_open
0      859056433305            135    T1_US_FNAL_Disk 2019-03-16 15:12:37  ...   135  /EGamma/Run2018D-22Jan2019-v2/AOD#57b25659-3e2...  10992784       n
1      859056433305            135    T1_RU_JINR_Disk 2019-03-02 12:54:55  ...   135  /EGamma/Run2018D-22Jan2019-v2/AOD#57b25659-3e2...  10992784       n
2      859056433305            135     T1_DE_KIT_Disk 2019-03-09 12:05:02  ...   135  /EGamma/Run2018D-22Jan2019-v2/AOD#57b25659-3e2...  10992784       n
3      859056433305            135     T1_US_FNAL_MSS 2019-03-29 23:29:13  ...   135  /EGamma/Run2018D-22Jan2019-v2/AOD#57b25659-3e2...  10992784       n
4      859056433305            135  T1_US_FNAL_Buffer 2019-04-01 10:02:04  ...   135  /EGamma/Run2018D-22Jan2019-v2/AOD#57b25659-3e2...  10992784       n
...             ...            ...                ...                 ...  ...   ...                                                ...       ...     ...
1495  1249881194222            201    T1_RU_JINR_Disk 2019-03-12 01:04:44  ...   201  /EGamma/Run2018D-22Jan2019-v2/AOD#f5d3628e-1c7...  11021144       n
1496  1249881194222            201    T1_US_FNAL_Disk 2019-03-16 15:12:37  ...   201  /EGamma/Run2018D-22Jan2019-v2/AOD#f5d3628e-1c7...  11021144       n
1497  1249881194222            201     T1_DE_KIT_Disk 2019-03-08 00:38:51  ...   201  /EGamma/Run2018D-22Jan2019-v2/AOD#f5d3628e-1c7...  11021144       n
1498  1249881194222            201     T1_US_FNAL_MSS 2019-03-29 23:29:13  ...   201  /EGamma/Run2018D-22Jan2019-v2/AOD#f5d3628e-1c7...  11021144       n
1499  1249881194222            201  T1_US_FNAL_Buffer 2019-04-01 06:08:03  ...   201  /EGamma/Run2018D-22Jan2019-v2/AOD#f5d3628e-1c7...  11021144       n

[1500 rows x 16 columns]

In [3]: df.groupby('replica.node').sum()
Out[3]:
                     replica.bytes  replica.files  replica.node_id
replica.node
T1_DE_KIT_Disk     215484127440685          33550           546300
T1_RU_JINR_Disk    215484127440685          33550           523500
T1_US_FNAL_Buffer  215484127440685          33550             2700
T1_US_FNAL_Disk    215484127440685          33550           534300
T1_US_FNAL_MSS     215484127440685          33550             3000

In [4]: df = await client.reqmgr.transitions(outputdataset='/EGamma/Run2018D-22Jan2019-v2/AOD')

In [5]: df[df['current']]
Out[5]:
                                          requestname  current               mask  ...           Status                                                 DN          UpdateTime
7   vlimant_ACDC0_Run2018D-v1-EGamma-22Jan2019_102...     True  RequestTransition  ...       closed-out  /DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=vl... 2019-03-27 13:14:10
17  prebello_Run2018D-v1-EGamma-22Jan2019_1025p1_1...     True  RequestTransition  ...  normal-archived  /DC=ch/DC=cern/OU=computers/CN=dmwm/cmsweb.cer... 2019-04-08 14:34:20

[2 rows x 7 columns]

Developer installation:

git clone git@github.com:nsmith-/dmwmclient.git
cd dmwmclient
pip3 install -e .[dev]
# do some development
black dmwmclient
flake8 dmwmclient

Please run the flake8 linter and black code formatter before committing.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

dmwmclient-0.0.4.tar.gz (13.7 kB view details)

Uploaded Source

Built Distribution

dmwmclient-0.0.4-py3-none-any.whl (17.0 kB view details)

Uploaded Python 3

File details

Details for the file dmwmclient-0.0.4.tar.gz.

File metadata

  • Download URL: dmwmclient-0.0.4.tar.gz
  • Upload date:
  • Size: 13.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.7.6

File hashes

Hashes for dmwmclient-0.0.4.tar.gz
Algorithm Hash digest
SHA256 27eefe226c80005ca2c41af1eba847dd2edf751e951fe957994198f2057a7af8
MD5 664b75175d1a059be00d1246bedf1704
BLAKE2b-256 34866c02e618bd41845c6a2352ca30fbaa8a6874ecd387d64775f325ac3ae5f0

See more details on using hashes here.

File details

Details for the file dmwmclient-0.0.4-py3-none-any.whl.

File metadata

  • Download URL: dmwmclient-0.0.4-py3-none-any.whl
  • Upload date:
  • Size: 17.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.21.0 setuptools/46.1.3 requests-toolbelt/0.9.1 tqdm/4.36.1 CPython/3.7.6

File hashes

Hashes for dmwmclient-0.0.4-py3-none-any.whl
Algorithm Hash digest
SHA256 55c145643436ec75f801aa54373e4ef4bf3cd8dcea4d4fdcc10dac2d02633613
MD5 0b121a9badbba94e42de9b798d6da993
BLAKE2b-256 f8dd3d02f8cfd14caba320b4959b9797fabee1cb125a3b495be454f00a0175a1

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page