Utility tool to upload files and datasets from local machine to rucio
Project description
User DataManagement Tools
Requirements
- initialise rucio (installs rucio_client) -
source /cvmfs/cms.cern.ch/rucio/setup-py3.sh
- initalise proxy -
voms-proxy-init -voms cms -rfc -valid 192:00
- export RUCIO_ACCOUNT=$USER
Additionally
pip3 install -r requirements.txt
How to
Show list of available commands: python3 upload.py --help
Show help related to a specific commands (cmd): python3 upload.py <cmd> --help
To upload a local file and create a rule on it:
-
Copy template from
lib/templates/file-upload.yml
-
Specify parameters:
filePath
: location of local filelfn
: lfn (logical file name) for the file -store/user/rucio/<username>/<filename>
tempRSE
: temp RSE to use for the intermediate hop (can be left to template value)rse
: destination RSE/RSE Expression on which rule should be createdlifetime
: life of rulecopies
: number of copies required
-
Execute:
python3 upload.py upload-file-yaml <yaml-file-path>
To upload a dataset create a rule on it:
-
Copy template from
lib/templates/dataset-upload.yml
-
Specify parameters:
datasetPath
: location of folder to use as datasetdatasetName
: name of dataset conforming to CMS naming conventionstempRSE
: temp RSE to use for the intermediate hop (can be left to template value)rse
: destination RSE/RSE Expression on which rule should be createdlifetime
: life of rulecopies
: number of copies required
-
Execute:
python3 upload.py upload-dataset-yaml <yaml-file-path>
Build Instructions
Prerequisite
We will be using poetry to build the package.
Create a wheel package
cd CMSRucio/UserDMTools/cmsrucio_import
poetry build
Installing from the wheel package
pip3 install --user dist/cmsrucio_import-0.1.0-py3-none-any.whl
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for cmsrucio_import-0.1.1-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | fbbd1215aed0358de39f0e14576c68ce4f84e0c1e0398541ab3aef67b8868705 |
|
MD5 | af02c4968c140561dbd82123c8ad5d8a |
|
BLAKE2b-256 | b0ee6f97115d315c0701bf4d58ffe2c5b698dbdb228b3f9b49e1013dfa0aca9e |