PIDCalib2

A set of software tools for estimating LHCb PID efficiencies.

The package includes several user-callable modules:

• make_eff_hists creates histograms that can be used to estimate the PID efficiency of a user's sample
• ref_calib calculates the LHCb PID efficiency of a user reference sample
• merge_trees merges two ROOT files with compatible TTrees
• plot_calib_distributions allows you to plot distributions of variables in the calibration datasets
• pklhisto2root converts Pickled boost-histograms to ROOT histograms

The term "reference dataset/sample" refers to the user's dataset to which they want to assign PID efficiencies. The "calibration datasets/samples" are the special, centrally managed samples used internally by PIDCalib for PID efficiency estimation. The --sample argument always concerns these calibration samples.

Slides with additional information, example output, and plots are available on Indico.

For a summary of the selections and coverage of the Run 3 samples see this talk.

Setup

When working on a computer where the LHCb software stack is available (LXPLUS, university cluster, etc.), one can setup PIDCalib2 by running

lb-conda pidcalib bash

After this, the following commands will be available

pidcalib2.make_eff_hists
pidcalib2.ref_calib
pidcalib2.merge_trees
pidcalib2.plot_calib_distributions
pidcalib2.pklhisto2root

You can skip the bash invocation and join the setup and run phases into a single command

lb-conda pidcalib pidcalib2.make_eff_hists

To run make_eff_hists, you will need access to CERN EOS. You don't need to do anything special on LXPLUS. On other machines, you will usually need to obtain a Kerberos ticket by running

kinit [username]@CERN.CH

Installing from PyPI

The PIDCalib2 package is available on PyPI. It can be installed on any computer via pip simply by running (preferably in a virtual environment; see venv)

pip install pidcalib2

Note that this will install the xrootd Python bindings. One also has to install XRootD itself for the bindings to work. See this page for XRootD releases and instructions.

make_eff_hists

This module creates histograms that can be used to estimate the PID efficiency of a user's sample.

Reading all the relevant calibration files can take a long time. When running a configuration for the first time, we recommend using the --max-files 1 option. This will limit PIDCalib2 to reading just a single calibration file. Such a test will reveal any problems with, e.g., missing variables quickly. Keep in mind that you might get a warning about empty bins in the total histogram as you are reading a small subset of the calibration data. For the purposes of a quick test, this warning can be safely ignored.

Options

To get a usage message listing all the options, their descriptions, and default values, type

pidcalib2.make_eff_hists --help

The calibration files to be processed are determined by the sample, magnet, and particle options. All the valid combinations can be listed by running

pidcalib2.make_eff_hists --list configs

Aliases for standard variables are defined to simplify the commands. We recommend users use the aliases when specifying variables. When you use a name that isn't an alias, a warning message like the following will show up in the log.

'probe_PIDK' is not a known PID variable alias, using raw variable

All aliases can be listed by running

pidcalib2.make_eff_hists --list aliases

Note that there are many more variables than there are aliases. If you want to find a variable for which no alias exists, you can check one of the calibration files yourself. The paths to the calibration files are printed when the --verbose option is specified. Alternatively, you can simply guess the name - if it doesn't exist, PIDCalib2 will let you know and might provide a list of similar names that do exist.

A file with alternative binnings can be specified using --binning-file. The file must contain valid JSON specifying bin edges. For example, two-bin binnings for particle Pi, variables P and PT can be defined as

{"Pi": {"P": [10000, 15000, 30000], "PT": [6000, 10000, 20000]}}

An arbitrary number of binnings can be defined in a single file.

Complex cut expressions can be created by chaining simpler expressions using & (logical and) and | (logical or). One can also use standard mathematical symbols, like *, /, +, -, (, ). Whitespace does not matter.

Examples

• Create a single 3D efficiency histogram for a single PID cut

  pidcalib2.make_eff_hists --sample Turbo18 --magnet up --particle Pi --pid-cut "DLLK > 4" --bin-var P --bin-var ETA --bin-var nSPDhits --output-dir pidcalib_output
  

• Create multiple histograms in one run (most of the time is spent reading in data, so specifying multiple cuts is much faster than running make_eff_hists sequentially)

  pidcalib2.make_eff_hists --sample Turbo16 --magnet up --particle Pi --pid-cut "DLLK > 0" --pid-cut "DLLK > 4" --pid-cut "DLLK > 6" --bin-var P --bin-var ETA --bin-var nSPDhits --output-dir pidcalib_output
  

• Create a single efficiency histogram for complex cuts using only negatively charged tracks

  pidcalib2.make_eff_hists --sample Turbo18 --magnet up --particle Pi --pid-cut "MC15TuneV1_ProbNNp*(1-MC15TuneV1_ProbNNpi)*(1-MC15TuneV1_ProbNNk) < 0.5 & DLLK < 3" --cut "IsMuon==0 & Brunel_PT>250 & trackcharge==-1" --bin-var P --bin-var ETA --bin-var nSPDhits --output-dir pidcalib_output
  

Caveats

Not all datasets have all the variables, and in some cases, the same variable is named differently (e.g., probe_Brunel_IPCHI2 is named probe_Brunel_MINIPCHI2 in certain electron samples). The aliases correspond to the most common names, but you might need to check the calibration files if PIDCalib2 can't find the variable you need.

ref_calib

This module uses the histograms created by make_eff_hists to assign efficiency to events in a reference sample supplied by the user. Adding efficiency to the user-supplied file requires PyROOT and is optional.

The module works in two steps:

1. Calculate the efficiency and save it as a TTree in a separate file.
2. Optionally copy the efficiency TTree to the reference file and make it a friend of the user's TTree. The user must request the step by specifying --merge on the command line.

Be aware that --merge will modify your file. Use with caution.

Options

The sample and magnet options are used solely to select the correct PID efficiency histograms. They should therefore mirror the options used when running make_eff_hists.

bin-vars must be a dictionary that relates the binning variables (or aliases) used to make the efficiency histograms with the variables in the reference sample. We assume that the reference sample branch names have the format [ParticleName]_[VariableName]. E.g., D0_K_calcETA, corresponds to a particle named D0_K and variable calcETA. If the user wants to estimate PID efficiency of their sample using 1D binning, where calcETA corresponds to the ETA binning variable alias of the calibration sample, they should specify --bin-vars '{"ETA": "calcETA"}'.

ref-file is the user's reference file to which they want to assign PID efficiencies. The parameter can be a local file or a remote file, e.g., on EOS (--ref-file root://eoslhcb.cern.ch//eos/lhcb/user/a/anonymous/tuple.root).

ref-pars must be a dictionary of particles from the reference sample to apply cuts to. The keys represent the particle branch name prefix (D0_K in the previous example), and the values passed are a list containing particle type and PID cut, e.g. '{"D0_K" : ["K", "DLLK > 4"], "D0_Pi" : ["Pi", "DLLK < 4"]}'.

The --merge option will copy the PID efficiency tree to your input file and make the PID efficiency tree a "Friend" of your input tree. Then you can treat your input tree as if it had the PID efficiency branches itself. E.g., input_tree->Draw("PIDCalibEff") should work. ROOT's "Friend" mechanism is an efficient way to add branches from one tree to another. Take a look here if you would like to know more.

Examples

• Evaluate efficiency of a single PID cut and save it to user_ntuple_PID_eff.root without adding it to user_ntuple.root

  pidcalib2.ref_calib --sample Turbo18 --magnet up --ref-file data/user_ntuple.root --histo-dir pidcalib_output --bin-vars '{"P": "mom", "ETA": "Eta", "nSPDHits": "nSPDhits"}' --ref-pars '{"Bach": ["K", "DLLK > 4"]}' --output-file user_ntuple_PID_eff.root
  

• Evaluate efficiency of a single PID cut and add it to the reference file user_ntuple.root

  pidcalib2.ref_calib --sample Turbo18 --magnet up --ref-file data/user_ntuple.root --histo-dir pidcalib_output --bin-vars '{"P": "mom", "ETA": "Eta", "nSPDHits": "nSPDhits"}' --ref-pars '{"Bach": ["K", "DLLK > 4"]}' --output-file user_ntuple_PID_eff.root --merge
  

• Evaluate efficiency of multiple PID cuts and add them to the reference file

  pidcalib2.ref_calib --sample Turbo18 --magnet up --ref-file data/user_ntuple.root --histo-dir pidcalib_output --bin-vars '{"P": "P", "ETA": "ETA", "nSPDHits": "nSPDHits"}' --ref-pars '{"Bach": ["K", "DLLK > 4"], "SPi": ["Pi", "DLLK < 0"]}' --output-file user_ntuple_PID_eff.root --merge
  

Irregular Binning Schemes

The default binning schemes for PIDCalib2 are regular, but users can also make use of irregular binning schemes, i.e. a different number of bins in momentum depending on the ETA bin. The method is to split the irregular binning scheme into several binning json files which are regular and loop over these, producing a new output file per binning file. An example is shown in notebooks/irregular_binning_run.sh using binning jsons defined in binnings.

Caveats

You might notice that some of the events in your reference sample are assigned PIDCalibEff, PIDCalibErr, or both of -999.

• PIDCalibEff is -999 when for at least one track
  • The event is out of binning range
  • The relevant bin in the efficiency histogram has no events whatsoever
  • The efficiency is negative
• PIDCalibErr is -999 when for at least one track
  • The event is out of binning range
  • The relevant bin in the efficiency histogram has no events whatsoever
  • The relevant bin in the efficiency histogram has no events passing PID cuts
  • The efficiency is negative

Because of double → float conversion in the original PIDCalib, tiny discrepancies (<1e−3 relative difference) in the efficiencies and/or uncertainties are to be expected.

A bug in the original PIDCalib caused the electron calibration datasets to be read twice, resulting in incorrect efficiency map uncertainties.

The original PIDCalib didn't apply the correct cuts to Omega samples (K_Omega and K_DD), leading to non-sensical efficiency maps.

Electrons

To use the efficiency tables for electrons in 2024 and 2025, one should run a command similar to:

pidcalib2.ref_calib  --sample 2024_WithUT_block1_Tables_with_brem --magnet up --ref-file "data/user_ntuple.root" --histo-dir /eos/lhcb/wg/rta/WP4/PIDCalib2_ElectronTables --bin-vars "{'PT' : 'PT', 'ETA': 'ETA'}" --ref-pars '{"eprobe": ["e", "ProbNNe > 0.2"]}' --output-file user_ntuple_PID_eff.root -v

The important differences here are:

• The --sample must include the bremsstrahlung category in the end, with_brem or without_brem. The data blocks available are: block1-8, MC_W3134, MC_W3537, MC_W3739, MC_W4042, s25c1MagDown, s25c1MagUp, s25c2MagDown, s25c3MagUp, s25c4MagDown, s25c4MagUp, s26c1MagDown, s26c1MagUp, s26c2MagDown and s26c2MagUp. (The four original brem categories 0brem, 1brem_tag, 1brem_probe and 2brem are availible for Block 1, 5 and 7, in case someone was already using those, but we recommend using the new naming scheme)
• The --magnet is labeled as up but it actually includes MagUp and MagDown.
• The --histo-dir must always be /eos/lhcb/wg/rta/WP4/PIDCalib2_ElectronTables, as that is where the efficiency tables are stored.
• The --bin-vars must always be "{'PT' : 'PT', 'ETA': 'ETA'}". The binning used to compute the efficiencies is {"e": {"PT": [500,700,900,1150,1500,2000,2750,4200,20000], "ETA": [1.5,2.5,2.85,3.2,3.6,5.25]}}. (For the orignal brem categories, use "{'P' : 'P'}", with the binning {"e": {"P": [0,4375,8750,13125,17500,20625,23750,26875,30000,35000,40000,45000,50000,62500,75000,87500,100000]}})
• The --ref-pars have to match the values of the computed tables, that is: "{DLLe: 0, 2, 3, 5}" and "{ProbNNe: 0.2, 0.8}".

Electron samples based on sWeights are also available. These use sWeights derived from a mass fit with Gaussian-constrained parameters from the simulated sample, and are evaluated in 40 bi-dimensional bins of PT and ETA. The samples are produced without preliminary INECAL and HASRICH requirements.

To generate an efficiency table using the sWeights, run pidcalib2.make_eff_hists command in a similar way:

pidcalib2.make_eff_hists --sample 2024_WithUT_block1_with_brem_v1 --magnet up --particle e --pid-cut "DLLe > 2" --bin-var PT --bin-var ETA --binning-file my_e_binning.json --output-dir pidcalib_output

• The --sample must include the bremsstrahlung category at the end, with_brem or without_brem, followed by _v1. The samples available are: 2024_WithUT_block1_v1-2024_WithUT_block8_v1, 2024_ppref_v1 (MagUp only), 2025_c1_v1-2025_c4_v1, and 2026_c1_v1-2026_c2_v1.
• The --magnet here does need to match the actual polarity of the data, unlike for the Tables samples above.

Always use the _v1-suffixed samples above: they are the current, actively maintained sWeight production. The older _1Dbin/_2Dbin-suffixed samples should not be used for new analyses.

plot_calib_distributions

This tool allows you to plot distributions of variables in the calibration datasets. You can supply the same cuts and custom binnings that you would use for make_eff_hists. If you wish to plot a variable for which no binning exists, a uniform binning with 50 bins will be used. You can change the number of bins using --bins and force a uniform binning even if another binning is defined via --force-uniform.

A plot for every requested variable will be created in the --output-dir directory. The format of the plots can be controlled by --format. Furthermore, plot_calib_distributions.pkl will be saved in the same directory, containing all the histograms, should the user want to make the plots manually.

Examples

• Create plots of the variables DLLK and P using 1 calibration file

  pidcalib2.plot_calib_distributions --sample Turbo18 --magnet up --particle Pi --bin-var DLLK --bin-var P --output-dir pidcalib_output --max-files 1
  

• Create PDF plots of variable P with 95 uniform bins

  pidcalib2.plot_calib_distributions --sample Turbo18 --magnet up --particle Pi --bin-var P --output-dir pidcalib_output --max-files 1 --format pdf --force-uniform --bins 95
  

• Create plots of variable P using custom binning

  pidcalib2.plot_calib_distributions --sample Turbo18 --magnet up --particle Pi --bin-var P --output-dir pidcalib_output --max-files 1 --format png --binning-file my_binning.json
  

optibinscheme

This module derives a data-driven binning scheme from Bayesian statistics for make_eff_hists/ref_calib, instead of requiring the user to pick bin edges by hand. It reads the calibration data (same sample/magnet/particle/PID-cut machinery as make_eff_hists), builds a fine sWeighted efficiency grid over the requested variables and runs a Bayesian beam search to find the partition of that grid the data support best scoring every candidate bin with its Bayesian evidence and penalising extra bins with a structural prior.

The scheme is irregular meaning that any region can be split along any variable at any fine edge independently of the others so the momentum binning can differ from one eta region to the next. Every candidate bin is scored on the joint statistics of all its fine cells so the correlations between the binning variables drive the splits. The bins are then grouped into maximal regular grids (blocks) each written as an individually-regular binning_scheme-block<i>.json file directly usable with make_eff_hists --binning-file, the way binning.py/notebooks/irregular_binning_run.sh expect. A scheme that happens to be regular comes out as a single file.

Along with the scheme the module reports per bin an efficiency, its statistical uncertainty, and a binning systematic answering how much the efficiency would move under the alternative binnings of that region the data do not disfavour. It also reports for every boundary a support rho (the posterior probability of keeping it rather than merging the two bins it separates) and the list of admissible alternative binnings with the log-posterior each gives up. These are diagnostics on the binning itself.

Options

To get a usage message listing all the options, their descriptions and default values type

pidcalib2.optibinscheme --help

--variables defines the initial fine grid for every binning variable as [min, max, n_step]. The optimizer works on a grid twice as fine as n_step in each dimension and places bin edges on it so n_step should be set generously. Over-binning the starting grid costs runtime, under-binning it caps the finest boundary the search can find. It also sets the smallest boundary move the systematic considers.

--structural-prior sets the penalty per bin (bic is the Bayesian Information Criterion and scales automatically with the effective sample size, uniform applies no explicit penalty, exponential requires an explicit --alpha).

--beams is the number of partitions the search expands at each level and --patience how many levels without improvement it tolerates before stopping. More beams and more patience explore more binnings at the cost of runtime.

--syst-window is the log-posterior distance to the chosen binning within which an alternative counts as admissible and --max-alternatives caps how many are kept. Widen the window if too few alternatives are found.

-o/--output-dir always writes the binning scheme files. Passing it with no argument, or omitting it entirely, uses output/, supplying it switches on the full diagnostic output as well (uncertainties-block<i>.json with the efficiencies and uncertainties of each block, splits.json, alternatives.json and a manifest.json tying the whole scheme together).

As with make_eff_hists, -n/--max-files is recommended for a first quick smoke test of a new configuration before reading the full sample.

Choosing --local-prior

--local-prior sets the Beta prior placed on each bin's true efficiency p before its evidence is computed. It only affects how the optimizer scores partitions (and downstream the efficiencies it reports). It does not affect make_eff_hists's own efficiency estimate on the final scheme.

• jeffreys (Beta(0.5, 0.5), the default) is the reference non-informative prior for a binomial-type rate. It pulls bins away from the boundary less aggressively than a flat prior when a bin's raw efficiency is near 0 or 1 with limited data which is the regime low-statistics bins (eg sparsely populated high-ETA or high-P bins) tend to fall into. Recommended default for most use cases especially whenever any bin in the fine grid is likely to be low-statistics.
• uniform (Beta(1, 1), flat) treats every value of p in [0, 1] as equally likely a priori. This is simpler and lets the data speak when every fine bin already has enough statistics that the prior has little influence.

Both integrals are computed exactly. In closed form for the uniform prior and by Gauss-Jacobi quadrature for Jeffreys whose singularities at 0 and 1 are removed by the substitution p = sin^2(t).

Examples

• Optimize a (P, ETA) binning scheme

  pidcalib2.optibinscheme --magnet up --particle K --pid-cut "DLLK > 6" --variables '{"ETA": [1.5, 5, 20], "P": [1000, 80000, 50]}' --output-dir pidcalib_binning_output
  

• Same with a flat efficiency prior and a fixed exponential structural penalty instead of BIC

  pidcalib2.optibinscheme --magnet up --particle K --pid-cut "DLLK > 6" --variables '{"ETA": [1.5, 5, 20], "P": [1000, 80000, 50]}' --local-prior uniform --structural-prior exponential --alpha 5 --output-dir pidcalib_binning_output
  

• Accept alternatives further from the best binning for a better-sampled systematic

  pidcalib2.optibinscheme --magnet up --particle K --pid-cut "DLLK > 6" --variables '{"ETA": [1.5, 5, 20], "P": [1000, 80000, 50]}' --beams 32 --patience 4 --syst-window 5 --output-dir pidcalib_binning_output
  

• Quick smoke test on a few calibration files before committing to a full run

  pidcalib2.optibinscheme --magnet up --particle K --pid-cut "DLLK > 6" --variables '{"ETA": [1.5, 5, 20], "P": [1000, 80000, 50]}' --max-files 1 --output-dir pidcalib_binning_output
  

Caveats

The binning systematic is the spread of the efficiency over the admissible alternative binnings of a bin's own region weighted by their posterior relative to each other. It answers "how much would this bin move under another plausible binning of this region". It does not vanish just because the chosen model would be strongly preferred overall. The alternative_cost written next to it says how much log-posterior the best alternative of that bin gives up so a large systematic with a large cost should be read as an upper bound. Despite being stable, part of it could be the statistical noise of finer alternatives so do not add it in quadrature to make_eff_hists's statistical uncertainty without checking it first on your own sample. Checks on toys will be tested in a future version but the current numbers are still safe and informative. Its magnitude obviously depends on the fine grid since one fine bin is the smallest boundary move considered. A coarser n_step mechanically gives a larger systematic. Choose n_step so that one fine bin is the resolution at which you consider two boundaries equivalent.

Irregularity costs binning files : a scheme of 130 bins typically comes out as 50 to 70 blocks hence as many make_eff_hists outputs. Use a stronger structural prior if that is impractical or you could automatize the reading of the files with a small dedicated macro.

Any binning variable not covered by pid_data's known-alias table is used as a literal branch name exactly as with make_eff_hists's --bin-var. If that literal name doesn't correspond to an actual branch on the sample/run you're using it can fail silently rather than raising an error and histogramdd simply drops every event falling outside the (nonsensical) values it reads which can silently zero out the entire optimization. Always check the log for '<var>' is not a known alias, using raw variable and on a new sample/run do a --max-files 1 sanity run first.

Bins with no calibration yield have no efficiency and no systematic. They are written as null in the JSON output. If the log reports bins that no alternative changes or no admissible alternative at all, widen --syst-window or raise --beams.

The efficiencies computed by optibinscheme and make_eff_hists are consistent so optibinscheme's schemes can be fed into make_eff_hists safely but beware that the efficiencies optibinscheme reports are diagnostics with make_eff_hists remaining the reference.

Future development

The current optimization is based on calibration samples, not on user's reference sample. It is planned to be a feature in a next version.

pklhisto2root

This tool converts pickled PIDCalib2 histograms to TH*D and saves them in a ROOT file. It can be used on histograms produced by make_eff_hists or plot_calib_distributions. Note that ROOT supports only 1-, 2-, and 3-dimensional histograms; attempting to convert higher-dimensional histograms will fail.

Example

• Convert pickled boost_histograms from make_eff_hists to ROOT

  pidcalib2.pklhisto2root "pidcalib_output/effhists-Turbo18-up-Pi-DLLK>4-P.ETA.nSPDhits.pkl"
  

  This will translate the histograms and save them to pidcalib_output/effhists-Turbo18-up-Pi-DLLK>4-P.ETA.nSPDhits.root.

Matching sWeights to calibration samples

In PIDCalib2, the sWeights are not appended to the calibration tuples to save size on eos. Instead, the sweight files on eos only have two branches: the sWeight branch and an extra branch to enable matching between the sWeight branch and the calibration tuples, called order_index.

If you wish to match the order between the sWeight branch and the tuples from the calibration samples, then for each file, you should sort the tuples using the ordering_variables, accessible within src/pidcalib2/data/samples.json in ascending order sequentially through the list. After sorting, the row number (starting from 0) corresponds to the order_index in the matching sWeight file. You can copy the match_sweights function in notebooks/sweight_calib_matching.ipynb to your own code if you wish to have the sWeight branches appended to calibration tuples. The exact logic of how PIDCalib2 does matching is manifest via its implementation at src/pidcalib2/utils.py lines 523-671.

Development

With lb-conda

On machines where lb-conda is available, you may use the pidcalib environment for PIDCalib2 development. This is mainly useful for small modifications and only if you don't need to add any new dependencies.

1. Clone the repository from GitLab
2. Enter the PIDCalib2 directory

   cd pidcalib2
   

3. Start a new BASH shell within the pidcalib environment

lb-conda pidcalib bash

3. Run your local PIDCalib2 code

cd src
python -m pidcalib2.make_eff_hists -h

Without lb-conda

This is a more versatile (if convoluted) method. It gives you full control of the dev environment and the ability to use IDEs, etc.

1. Clone the repository from GitLab
2. Enter the PIDCalib2 directory

   cd pidcalib2
   

3. (Optional) Set up a virtual environment

   python3 -m venv .venv
   source .venv/bin/activate
   

4. Install pinned dependencies

   pip install -r requirements-dev.txt
   

5. Install xrootd (possibly manually; see this issue)
6. Run the tests

   pytest
   

7. Run the modules

   cd src
   python3 -m pidcalib2.make_eff_hists -h
   

Tips

Certain tests can be excluded using markers like this

pytest -m "not xrootd"

See available markers by running pytest --markers (the list will start with PIDCalib2 custom markers, then it will include all the pytest built-in markers).

Links

• PIDGen2 - a tool to resample MC PID variables based on distributions from data calibration samples