lhcsmapi 1.2.39

A package with an API for signal access and processing for the LHC Signal Monitoring project.

LHC Signal Monitoring API Package

This is a package with an API for signal access and processing for the LHC Signal Monitoring project.

Prerequisites

In order to use the API has to be installed with a python package installer as

pip install --user lhcsmapi

Check the latest version at https://pypi.org/project/lhcsmapi/

The API relies on external python packages which have to be installed in a similar manner. The list of packages is stored in the <u>requirements.txt</u> file.

The documentation for the API is stored at http://cern.ch/lhc-sm-api. Below please find an excerpt of the code structure and snippets of the most frequently used functions.

Project structure

• cals
  • CalsDbSignal.py
  • DetectBmodePattern.py
• dbsignal
  • DbSignal.py
  • Signal.py
  • SignalElectrical.py
  • SignalProcessing.py
  • SignalUtilities.py
• influx
  • InfluxDbSignal.py
• post_mortem
  • PmDbSignal.py
  • PmDbRequest.py
• metadata
  • Metadata.py
  • QpsMetadata.json
  • FgcMetadata.json
  • Ipqd2Metadata.json
  • Ipqd4Metadata.json
  • Ipqd8Metadata.json
  • QpsMetadata.json
  • RbMetadata.json
  • RqMetadata.json
• Time.py
• Timer.py

Time

Time class encapsulates provides a set of methods for conversion between various time formats. In addition it provides a generator for days between given periods. Main methods are:

• Time.to_unix_timestamp(input, unit='ns')
• Time.to_string(input, unit='ns')
• Time.to_datetime(input, unit='ns')
• Time.to_pandas_timestamp(input, unit='ns')

from lhcsmapi.Time import Time
stringDate = "2018-05-01 00:00:00"
Time.to_unix_timestamp(stringDate, unit = "ns")
>>> 1525125600000000000

from lhcsmapi.Time import Time
from datetime import datetime
dateTime = datetime(2017, 7, 2, 1, 23, 27, 0)
Time.to_string(stringDate, unit = "ns")
>>> '2017-07-02 01:23:27'

from lhcsmapi.Time import Time
stringDate = "2019-01-02 23:59:11"
Time.to_datetime(stringDate, unit = "ns")
>>> datetime.datetime(2019, 1, 2, 23, 59, 11, tzinfo=<DstTzInfo 'Europe/Zurich' CET+1:00:00 STD>)

from lhcsmapi.Time import Time
stringDate = "2019-07-10 10:23:27.013"
Time.to_pandas_timestamp(stringDate, unit = "ns")
>>> Timestamp('2019-07-10 10:23:27.013000+0200', tz='Europe/Zurich')

• Time.get_query_period_in_unix_time()

from lhcsmapi.Time import Time
startTimeData = 1426220459491000000
endTimeData =   1426220479491000000
Time.get_query_period_in_unix_time(startTimeData, endTimeData, None, tz='Europe/Zurich')
>>> (1426220459491000000, 1426220479491000000)

from lhcsmapi.Time import Time
startTimeData = 1426220459491000000
durationData =  [(20, 's')]
Time.get_query_period_in_unix_time(startTimeData, None, durationData, tz='Europe/Zurich')
>>> (1426220459491000000, 1426220479491000000)

from lhcsmapi.Time import Time
startTimeData = 1426220469491000000
durationData =  [(10, 's'), (10, 's')]
Time.get_query_period_in_unix_time(startTimeData, None, durationData, tz='Europe/Zurich')
>>> (1426220459491000000, 1426220479491000000)

• Time.daterange(startDate, endDate)

from lhcsmapi.Time import Time
from datetime import date
startDate = date(2014, 1, 1)
endDate = date(2014, 1, 3)
for singleDate in Time.daterange(startDate, endDate):
    print(singleDate)
>>> 2014-01-01
>>> 2014-01-02

from datetime import date
startDate = date(2014, 1, 5)
endDate = date(2014, 1, 3)
for singleDate in Time.daterange(startDate, endDate):
    print(singleDate)
>>>

Metadata

Metadata class stores information about signal and circuit names.

from lhcsmapi.metadata.Metadata import Metadata
magnetType = "MB"
circuitType = "RB"
component = "QPS"
magnetTypeToSignalMetadata = Metadata.getMetadata(component, magnetType)
magnetTypeToSignalMetadata[magnetType]
>>> {'className': 'DQAMCNMB_PMHSU',
>>> 'sourcePrefix': 'MB_',
>>> 'system': 'QPS',
>>> 'source': '*',
>>> 'magnetNameRegExp': '[ABC]{1}\\d+[LR]{1}[1-8]{1}',
>>> 'uNames': ['U_HDS_1', 'U_HDS_2', 'U_HDS_3', 'U_HDS_4'],
>>> 'iNames': ['I_HDS_1', 'I_HDS_2', 'I_HDS_3', 'I_HDS_4'],
>>> 'rNames': ['R_HDS_1', 'R_HDS_2', 'R_HDS_3', 'R_HDS_4'],
>>> 'dRNames': ['dR_HDS_1', 'dR_HDS_2', 'dR_HDS_3', 'dR_HDS_4'],
>>> 'filterNames': ['U_HDS_1',
>>>  'U_HDS_2',
>>>  'U_HDS_3',
>>>  'U_HDS_4',
>>>  'I_HDS_1',
>>>  'I_HDS_2',
>>>  'I_HDS_3',
>>>  'I_HDS_4'],
>>> 'uNamesCals': ['MB.{}:U_HDS_1',
>>>  'MB.{}:U_HDS_2',
>>>  'MB.{}:U_HDS_3',
>>>  'MB.{}:U_HDS_4'],
>>>'iNamesCals': ['MB.{}:I_HDS_1',
>>>  'MB.{}:I_HDS_2',
>>>  'MB.{}:I_HDS_3',
>>>  'MB.{}:I_HDS_4']}

PmDbQuery

PmDbRequest class encapsulates PM database queries in order to simplify the query creation as well as browsing of PM events for a given period.

• Expected Success Response

from source.post_mortem.PmDbRequest import PmDbRequest
source = "*"
system = "QPS"
className = "DQAMCNMB_PMHSU"
selectedDayTimeStamp = 1426201200000000000
eventDurationSeconds = 24*60*60

PmDbRequest.find_events(source, system, className,
    eventTime = selectedDayTimeStamp, duration = [(eventDurationSeconds, 's')])
>>> [('B20L5', 1426220469491000000),
>>> ...

• Expected Fail Response::

from source.post_mortem.PmDbRequest import PmDbRequest
source = "*"
system = "QPS"
className = "DQAMCNMB_PMHSU"
selectedDayTimeStamp = 1426201200000000000
eventDurationSeconds = 1
PmDbRequest.find_events(source, system, className,
    eventTime = selectedDayTimeStamp, duration = [(eventDurationSeconds, 's')])
>>> UserWarning: Querying Post Mortem failed using the following query:
                http://pm-api-pro/v2/pmdata/header/within/duration?system=QPS&className=DQAMCNMB_PMHSU&source=*&fromTimestampInNanos=1426201200000000000&durationInNanos=1000000000 returned no data so an empty string is being returned

Signal

Signal class encapsulates behavior for working with signal databases. It provides read and write methods for accessing Post Mortem, CALS and InfluxDB APIs.

• CALS Success Response

from lhcsmapi.dbsignal.Signal import Signal
import pytimber
ldb = pytimber.LoggingDB()
arcCurrentName = "RPTE.UA47.RB.A45:I_MEAS"
startTime = 1426220459491000000
endTime =   1426220479491000000
I_MEAS = Signal().read('cals', name=arcCurrentName, startTime = startTime, endTime = endTime, ldb = ldb)
print(I_MEAS.head())
>>>               RPTE.UA47.RB.A45:I_MEAS
>>> 1.426220e+09                  10071.5
>>> 1.426220e+09                  10076.5
>>> 1.426220e+09                  10081.5
>>> 1.426220e+09                  10086.5
>>> 1.426220e+09                  10091.5

from lhcsmapi.dbsignal.Signal import Signal
import pytimber
ldb = pytimber.LoggingDB()
arcCurrentName = "RPTE.UA47.RB.A45:I_MEAS"
startTime = 1426220459491000000
duration = [(20, 's')]
I_MEAS = Signal().read('cals', name=arcCurrentName, startTime = startTime, duration = duration, ldb = ldb)
print(I_MEAS.head())
>>>               RPTE.UA47.RB.A45:I_MEAS
>>> 1.426220e+09                  10071.5
>>> 1.426220e+09                  10076.5
>>> 1.426220e+09                  10081.5
>>> 1.426220e+09                  10086.5
>>> 1.426220e+09                  10091.5

from lhcsmapi.dbsignal.Signal import Signal
import pytimber
ldb = pytimber.LoggingDB()
arcCurrentName = "RPTE.UA47.RB.A45:I_MEAS"
startTime = 1426220469491000000
duration = [(10, 's'), (10, 's')]
I_MEAS = Signal().read('cals', name=arcCurrentName, startTime = startTime, duration = duration, ldb = ldb)
print(I_MEAS.head())
>>>               RPTE.UA47.RB.A45:I_MEAS
>>> 1.426220e+09                  10071.5
>>> 1.426220e+09                  10076.5
>>> 1.426220e+09                  10081.5
>>> 1.426220e+09                  10086.5
>>> 1.426220e+09                  10091.5

• CALS Fail Response

from lhcsmapi.dbsignal.Signal import Signal
import pytimber
ldb = pytimber.LoggingDB()
arcCurrentName = "RPTE.UA47.RB.A45:I_MEAS"
startTime = 1426220469491000000
duration = [(10, 's'), (10, 's')]
I_MEAS = Signal().read('cals', name=arcCurrentName, startTime = startTime, duration = duration)
print(I_MEAS.head())
>>> Exception: You have to provide a instance of pytimber.LoggingDB via the ldb keyword

from lhcsmapi.dbsignal.Signal import Signal
import pytimber
ldb = pytimber.LoggingDB()
arcCurrentName = "RPTE.UA47.RB.A45:I_MEA"
startTime = 1426220469491000000
duration = [(10, 's'), (10, 's')]
I_MEAS = Signal().read('cals', name=arcCurrentName, startTime = startTime, duration = duration, ldb = ldb)
print(I_MEAS.head())
>>> WARNING:pytimber.pytimber:No variables found.
>>> Empty DataFrame
>>> Columns: []
>>> Index: []
>>> UserWarning: Querying CALS with the parameters name: RPTE.UA47.RB.A45:I_MEA, start time: 1426220459.4910002 and end time: 1426220479.4910002 returned no data.
>>> An empty DataFrame was returned.
>>> ___________________________________________________________________________________
>>> Reproduce the error using the raw pytimber API:

>>> import pytimber
>>> ldb = pytimber.LoggingDB()
>>> ldb.get(RPTE.UA47.RB.A45:I_MEA, 1426220459.4910002, 1426220479.49100

• PM Success Response

from lhcsmapi.dbsignal.Signal import Signal
system = "QPS"
className = "DQAMCNMB_PMHSU"
magnet = "B20L5"
signal = "I_HDS_4"
timestamp = 1426220469491000000
I_HDS_4 = Signal().read('pm', system=system, className=className, source=magnet,
                              signal=signal, eventTime=timestamp)
print(I_HDS_4.head())
>>>                       I_HDS_4
>>> 1426220469491020832  0.089648
>>> 1426220469491026040  0.091555
>>> 1426220469491031248  0.091555
>>> 1426220469491036456  0.093463
>>> 1426220469491041664  0.091555

• PM Fail Response

from lhcsmapi.dbsignal.Signal import Signal
system = "QPS"
className = "DQAMCNMB_PMHSU"
magnet = "B20L5"
signal = "I_HDS_4"
timestamp = 1426220469491000000
I_HDS_4 = Signal().read('pm', system=system, className=className, source=magnet,
                              signal=signal, eventTime=timestamp)
print(I_HDS_4.head())
>>> Querying Post Mortem failed using the following query:
>>> http://pm-api-pro/v2/pmdata/signal?system=QPS&className=DQAMCNMB_PMHS&source=B20L5&timestampInNanos=1426220469491000000&signal=I_HDS_4