func-adl-xAOD 1.5b4

Functional Analysis Description Language backend for accessing ATLAS xAOD files.

func_adl_xAOD

Backend that converts qastle to run on an ATLAS xAOD backend.

Introduction

This allows you to query hierarchical data stored in a root file that has been written using the ATLAS xAOD format. This code allows you to query that.

Features

A short list of some of the features that are supported by the xAOD C++ translator follows.

Python

Many, but not all, parts of the python language are supported. As a general rule, anything that is a statement or flow control is not supported. So no if or while or for statements, for example. Assignment isn't supported, which may sound limiting - but this is a functional implementation so it is less to than one might think.

What follows are the parts of the language that are covered:

• Function calls, method calls, property references, and lambda calls (and lambda functions), with some limitations.
• Integer indexing into arrays
• Limited tuple support as a means of collecting information together, or as an output to a ROOT file.
• Limited list support (in same way as above). In particular, the append method is not supported as that modifies the list, rather than creating a new one.
• Unary, Binary, and comparison operations. Only 2 argument comparisons are supported (e.g. a > b and not a > b > c).
• Using and and or to combine conditional expressions. Note that this is written as & and | when writing an expression due to the fact python demands a bool return from and and or when written in code.
• The conditional if expression (10 if a > 10 else 20)
• Floating point numbers, integers, and strings.

xAOD Functions

You can call the functions that are supported by the C++ objects as long as the required arguments are primitive types. Listed below are special extra functions attached to various objects in the ATLAS xAOD data model.

The Event

The event object has the following special functions to access collections:

• Jets, Tracks, EventInfo, TruthParticles, Electrons, Muons, and MissingET. Each function takes a single argument, the name of the bank in the xAOD. For example, for the electrons one can pass "Electrons".

Adding new collections is fairly easy.

The Jet Object

Template functions don't make sense yet in python.

• getAttribute - this function is templated, so must be called as either getAttributeFloat or getAttributeVectorFloat.

Math

• Math Operators: +, -, *, /, %, **
• Comparison Operators: <, <=, >, >=, ==, !=
• Unary Operators: +, -, not
• Math functions are pulled from the C++ cmath library: sin, cos, tan, acos, asin, atan, atan2, sinh, cosh, tanh, asinh, acosh, atanh, exp, ldexp, log, ln, log10, exp2, expm1, ilogb, log1p, log2, scalbn, scalbln, pow, sqrt, cbrt, hypot, erf, erfc, tgamma, lgamma, ceil, floor, fmod, trunc, round, rint, nearbyint, remainder, remquo, copysign, nan, nextafter, nexttoward, fdim, fmax, fmin, fabs, abs, fma.
• Do not use math.sin in a call. However sin is just fine. If you do, you'll get an exception during resolution that it doesn't know how to translate math.
• for things like sum, min, max, etc., use the Sum, Min, Max LINQ predicates.

Output Formats

The xAOD code only renders the func_adl expression as a ROOT file. The ROOT file contains a simple TTree in its root directory.

• If AsROOTTTree is the top level func_adl node, then the tree name and file name are taken from that expression. Only a sequence of python tuples or a single item can be understood by AsROOTTTree.
• If a Select sequence of int or double is the last func_adl expression, then a file called xaod_output.root will be generated, and it will contain a TTree called atlas_xaod_tree with a single column, called col1.
• If a Select sequence of a tuple is the last func_adl expression, then a file called xaod_output.root will be generated, and it will contain a TTree called atlas_xaod_tree with a columns named col1, col2, etc.
• If a Select sequence of dictionary's is the last func_adl expression, then a file called xaod_output.root will be generated, and it will contain a TTree called atlas_xaod_tree, with column names taken from the dictionary keys.

ServiceX (and the servicex frontend package) can convert from ROOT to other formats like a pandas.DataFrame or an awkward array.

Testing and Development

Setting up the development environment:

• After creating a virtual environment, do a setup-in-place: pip install -e .[test]

To run tests:

• pytest -m "not atlas_xaod_runner and not cms_aod_runner" will run the fast tests.
• pytest -m "atlas_xaod_runner" and pytest -m "cms_aod_runner" will run the slow tests for ATLAS and CMS respectively that require docker installed on your command line. docker is involved via pythons os.system - so it needs to be available to the test runner.
• The CI on github is setup to run tests against python 3.7, 3.8, and 3.9 (only the non-xaod-runner tests).

Contributing:

• Develop in another repo or on a branch
• Submit a PR against the master branch.

In general, the master branch should pass all tests all the time. Releases are made by tagging on the master branch.

Publishing to PyPi:

• Automated by declaring a new release (or pre-release) in github's web interface