Skip to main content

Python Client for European XFEL Calibration Catalogue Web App available at https://in.xfel.eu/calibration

Project description

Calcat is the Web App design for Calibration Constants Data Management at European XFEL.

This library (calibration_client) is a client for the RESTful APIs exposed by the European XFEL Calibration Constants Catalogue Web Application - calCat (https://in.xfel.eu/calibration).

Repository:

Dependencies:

Installation

Python project

  1. Install requirements, if never done before

1.1. For OS X distributions:

1.1.1. Homebrew

      brew install python3

1.1.2 Port

      sudo port install python36

      sudo port select --set python3 python36

      sudo port install py36-pip
      sudo port select --set pip pip36

1.2. For Linux distributions:

sudo apt-get update
sudo apt-get install python3.9
  1. Make calibration_client library available in your python environment

2.1. Install it via pip:

# Install dependencies from local wheels files
pip install . --no-index --find-links ./external_dependencies/

# Install dependencies from the pypi
pip install .

# Force re-installation of packages
pip install --ignore-installed

Installing it will place two folders under the current Python installation site-packages folder:

  • calibration_client with the sources;
  • calibration_client-11.0.0.dist-info/ with Wheels configuration files.

To identify your Python site-packages folder run:

python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())"

Usage

To use this project you need to import it:

from calibration_client import CalibrationClient
  1. Connection to the MdC (Metadata Catalog):

    from calibration_client import CalibrationClient
    
    # Necessary configuration variables to establish a connection
    # Go to https://in.xfel.eu/calibration/oauth/applications to make a token for
    # the calibration catalogue.
    user_id = 'PUT_HERE_YOUR_CLIENT_KEY'
    user_secret = 'PUT_HERE_YOUR_SECRET_KEY'
    user_email = 'luis.maia@xfel.eu'
    #
    metadata_web_app_url = 'https://in.xfel.eu/calibration'
    token_url = 'https://in.xfel.eu/calibration/oauth/token'
    refresh_url = 'https://in.xfel.eu/calibration/oauth/token'
    auth_url = 'https://in.xfel.eu/calibration/oauth/authorize'
    scope = ''
    base_api_url = 'https://in.xfel.eu/calibration/api/'
    
    # Authentication via OAUTH2
    # Generate the connection (example with minimum parameter options)
    client_conn = CalibrationClient(client_id=user_id,
                                    client_secret=user_secret,
                                    user_email=user_email,
                                    token_url=token_url,
                                    refresh_url=refresh_url,
                                    auth_url=auth_url,
                                    scope=scope,
                                    base_api_url=base_api_url)
    
    # Authentication via OAUTH2
    # Generate the connection (example with all parameter options)
    client_conn = CalibrationClient(use_oauth2=True
                                    client_id=user_id,
                                    client_secret=user_secret,
                                    user_email=user_email,
                                    token_url=token_url,
                                    refresh_url=refresh_url,
                                    auth_url=auth_url,
                                    scope=scope,
                                    base_api_url=base_api_url,
                                    session_token=None,
                                    max_retries=3,
                                    timeout=12,
                                    ssl_verify=True)
    
    # Authentication via Proxy Service
    # Generate the connection (example with minimum parameter options)
    client_conn = CalibrationClient(use_oauth2=False,
                                    base_api_url="http://HOSTNAME:PORT/api/")
    
  2. Interaction with the CalCat (Calibration Catalog):

2.1 Example data_group_types:

params_h = { 'detector_identifier': 'TEST_DET_CI-2',
             'snapshot_at': '' }

resp = client_conn.get_all_phy_det_units_from_detector(params_h)

resp
# >>> {'success': True,
#      'data': [ { "id":-1,
#                  "physical_name":"PHYSICAL_DETECTOR_UNIT-1_DO_NOT_DELETE",
#                  "karabo_da":"TEST_DAQ_DA_01",
#                  "virtual_device_name":"Q1M1",
#                  "uuid":1000,
#                  "float_uuid":4.94e-321,
#                  "detector_type_id":-1,
#                  "detector_id":-2,
#                  "flg_available":true,
#                  "description":"None",
#                  "detector":{
#                     "id":-2,
#                     "name":"DET_TEST-2_DO_NOT_DELETE",
#                     "identifier":"TEST_DET_CI-2",
#                     "karabo_name":"TEST_DET_CI_-2",
#                     "karabo_id_control":"TEST_DET_CI-2_CTRL00",
#                     "flg_available":true,
#                     "description":"None"
#                  },
#                  "detector_type":{
#                     "id":-1,
#                     "name":"UNIT_TEST_DETECTOR_TYPE-1_DO_NOT_DELETE",
#                     "flg_available":true,
#                     "description":"None"
#                  }
#                },
#                { ... },
#                { ... } ],
#      'app_info': {},
#      'info': 'Got physical_detector_unit successfully'}

resp['success']
# >>> True

resp['data'][0]['karabo_da']
# >>> 'TEST_DAQ_DA_01'

For additional examples, please take a look in the tests/ folder.

Development & Testing

When developing, and before commit changes, please validate that:

  1. All tests continue passing successfully (to validate that run pytest):

    # Go to the source code directory
    cd calibration_client
    
    # Upgrade package and all its required packages
    pip install . -U --upgrade-strategy eager
    
    # Install test dependencies
    pip install '.[test]' -U --upgrade-strategy eager
    
    # Run all tests using pytest
    pytest
    
    # When running all tests against the standard http application
    OAUTHLIB_INSECURE_TRANSPORT=1 pytest
    
    # Run all tests and get information about coverage for all files inside calibration_client package
    pytest --cov calibration_client --cov-report term-missing
    
  2. Code keeps respecting pycodestyle code conventions (to validate that run pycodestyle):

    pycodestyle .
    pycodestyle . --exclude venv
    
  3. To generate all the wheels files for the dependencies, execute:

    # Generate Wheels to itself and dependencies
    pip wheel --wheel-dir=./external_dependencies .
    pip wheel --wheel-dir=./external_dependencies --find-links=./external_dependencies .
    
  4. Check that you have the desired dependency versions in external_dependencies folder, since no versions are now set in setup.py.

Registering library on https://pypi.org

To register this python library, the following steps are necessary:

# Install twine
python -m pip install --upgrade twine

# Generates source distribution (.tar.gz) and wheel (.whl) files in the dist/ folder
python setup.py sdist
python setup.py bdist_wheel

# Upload new version .egg and .whl files
twine upload dist/*

# In case a test is necessary, it is possible to test it against test.pypi.org
twine upload --repository-url https://test.pypi.org/legacy/ dist/* --verbose

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

calibration_client-11.0.0.tar.gz (49.4 kB view hashes)

Uploaded source

Built Distribution

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Huawei Huawei PSF Sponsor Microsoft Microsoft PSF Sponsor NVIDIA NVIDIA PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page