nrtqclib
nrtqclib is a Python library that runs Near-Real Time Quality Control (NRT QC) on CTD (Conductivity, Temperature, Depth) ocean profiles. It applies the automated real-time QC tests (Argo/CTD RTQC tests) to temperature and salinity, writes one flag column per test and a final NRT flag per variable, and can compare the new flags with the ones already in the data. It is part of the AIQC project, alongside aiqclib, which implements the QC tests and uses their flags as training features for its machine-learning models.
QC Tests Run by nrtqclib
| Item | Test | Level | Catches |
|---|---|---|---|
impossible_date |
RTQC2 | profile | a timestamp that is missing, too early or in the future |
impossible_location |
RTQC3 | profile | a latitude or longitude that is missing or out of bounds |
position_on_land |
RTQC4 | profile | a position that is not in the ocean |
global_range |
RTQC6 | observation | a value no ocean water can have |
regional_range |
RTQC7 | observation | a value outside what the region allows |
pressure_increasing |
RTQC8 | observation | pressure that stalls or goes back up the profile |
spike |
RTQC9 | observation | a single value sticking out from both neighbours |
gradient |
RTQC11 | observation | too steep a change between neighbouring levels |
digit_rollover |
RTQC12 | observation | a jump caused by a sensor counter wrapping around |
stuck_value |
RTQC13 | profile | a sensor reporting the same value all the way down |
density_inversion |
RTQC14 | observation | lighter water under heavier water |
temp_to_psal |
rule | observation | salinity computed from a bad temperature |
Flags follow the IOC/Argo scheme: 1 (good), 3 (probably bad) and 4 (bad).
Installation
The package is available on PyPI and needs Python 3.12 or newer. It installs aiqclib as a dependency.
Using pip:
pip install nrtqclib
Using uv:
uv pip install nrtqclib
nrtqclib is not on conda-forge; in a conda environment, install it with pip.
Documentation
Project documentation is hosted on Read the Docs. It includes a Getting Started tutorial that builds the input file with the companion tools ctddump (NetCDF to Parquet) and seastamp (sea floor depth), and a page per QC test.
Core Concepts
A run reads one Parquet file of CTD observations (one row per observation) and goes through four steps:
- input: Read and validate the input file.
- qc: Run every configured QC item, producing one flag column per item and variable.
- concat: Combine the item flags into the final
temp_nrt_flagandpsal_nrt_flag(the most severe item flag) and write the output file. - compare: Compare the new flags with the flags already in the input, when the configuration names them.
Each run is controlled by a YAML configuration file, allowing you to define and reproduce your QC runs with ease. Per item, fail_flag: 3 softens a failure to "probably bad", and include_in_final_flag: false keeps the item's column while leaving it out of the final flag.
The QC tests themselves live in aiqclib, as qc_* features; nrtqclib is the pipeline that runs them on their own. It was aiqclib.run_nrt_qc up to aiqclib 0.14.0.
Usage
The workflow follows the same steps as in aiqclib:
- Generate a Configuration Template: Create a starter YAML file.
- Customize the Configuration: Edit the YAML file to specify paths, the input file and the regional ranges.
- Run NRT QC: Load the configuration and run the pipeline.
Step 1: Generate a configuration template.
import nrtqclib as nq
nq.write_config_template("/path/to/nrt_qc_config.yaml")
Step 2: Customize nrt_qc_config.yaml.
You must edit the file to set the correct input/output paths, the input file name and the regional ranges. See the Configuration section for details.
Step 3: Run NRT QC.
import nrtqclib as nq
config = nq.read_config("/path/to/nrt_qc_config.yaml")
nq.run_nrt_qc(config)
This generates the following output folders:
- qc: The flag columns of every QC item.
- nrt_qc: The input data with the item flags and the final NRT flags.
- compare: One report per variable comparing the existing and new flags (contingency table, agreement metrics, per-item breakdown).
Running several datasets at once
run_batch runs NRT QC over a table of datasets (column nrt_qc_set_name), returning a summary of every run:
import nrtqclib as nq
summary = nq.run_batch(
"datasets.txt", # table of set names per dataset
nrt_qc_config="nrt_qc_config.yaml",
verbose=True,
)
See the batch guide in the documentation for the table format and options.
Migrating from aiqclib
aq.run_nrt_qc, aq.read_config, aq.read_config_template and aq.write_config_template become nq.run_nrt_qc, nq.read_config, nq.read_config_template and nq.write_config_template with the same arguments, minus stage; aq.run_batch(..., mode="nrt_qc") becomes nq.run_batch(...) without mode. The pipeline, configuration format and outputs are unchanged.
Configuration
Configuration is managed via a YAML file. The write_config_template function provides a starting point that you must customize.
To inspect the defaults without writing a file, read_config_template returns the configuration object instead:
print(nq.read_config_template())
The configuration requires you to modify these sections:
-
path_info_sets: Defines the location of input and output data.path_info_sets: - name: nrt_qc_path_1 common: base_path: /path/to/data # EDIT: Root output directory input: base_path: /path/to/input # EDIT: Directory with input files step_folder_name: "" concat: step_folder_name: nrt_qc # EDIT: Directory with NRT QC results
-
nrt_qc_sets: Defines a specific dataset to be checked.nrt_qc_sets: - name: nrt_qc_0001 # EDIT: Your NRT QC set name dataset_folder_name: nrt_qc_0001 # EDIT: Your output folder input_file_name: nrt_cora_bo_4.parquet # EDIT: Your input filename
-
qc_item_sets: Lists the QC items and their parameters. The regional range test has no defaults: replace the template's example ranges (Mediterranean) with the ranges of your region.qc_item_sets: - name: qc_item_set_1 items: - name: regional_range # EDIT: ranges of this file's region params: { temp: { min: 10.0, max: 40.0 }, psal: { min: 2.0, max: 40.0 } }
-
qc_variable_sets(optional): Names the existing flag column of each variable, to compare against.qc_variable_sets: - name: qc_variable_set_1 variables: - name: temp flag: temp_qc # Optional: existing NRT QC flag column for comparison pos_flag_values: [ 4, 6, 7 ] # Optional: for agreement metrics neg_flag_values: [ 1 ]
Contributing & Development
We welcome contributions! Development setup (uv environment), running tests, and code style are documented in CONTRIBUTING.md.
Releasing & Deployment (for Maintainers)
The release process (versioning checklist), building the docs, and deployment to PyPI are documented in RELEASING.md.
License
MIT
Metadata
Release files for nrtqclib 0.1.5
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | Actions |
|---|---|---|---|---|
| nrtqclib-0.1.5.tar.gz | 302.9 kB | With provenance | Details for nrtqclib-0.1.5.tar.gz |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Actions Reset |
|---|---|---|---|---|
| nrtqclib-0.1.5-py3-none-any.whl | Python 3 | none | any | Details for nrtqclib-0.1.5-py3-none-any.whl |
Total release size: 335.5 kB
Release files / nrtqclib-0.1.5.tar.gz
| Download URL | nrtqclib-0.1.5.tar.gz |
|---|---|
| Size | 302.9 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
8c3117780358c5ddf584de07792f264bc148f619f940e5e18125b69f4e2fd6a4
|
|
BLAKE2b-256 checksum How to use checksums |
7ac6388a2b934a811fdf9dec77d2aeccb49f5c4e3247658804ad1343d021708f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 6, 2026.
Transparency logRelease files / nrtqclib-0.1.5-py3-none-any.whl
| Download URL | nrtqclib-0.1.5-py3-none-any.whl |
|---|---|
| Size | 32.7 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
b0cb781541a6f5650aa51e922fec9e06be3c6995f108d8971f1b4e5411c7bb49
|
|
BLAKE2b-256 checksum How to use checksums |
8cd46a3d41af18dce1b2946b2623397ef1c2df1c6a3dc90a45060522b171c67f
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
Yes |
| Uploaded via |
twine/7.0.0 CPython/3.13.16
|
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 6, 2026.
Transparency logSecurity for nrtqclib 0.1.5
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
Full provenance
Both files in this release have been uploaded with provenance metadata.
Attestation for nrtqclib-0.1.5.tar.gz
PyPI Publish Attestation
PyPI verified that this artifact, at this checksum, originated from the publisher listed below.
Signed by GitHub Actions, verified by PyPI on Nov 6, 2026.
Transparency logA note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.