Skip to main content

Utilities for reading Atop files natively in Python

Project description

os: linux python: 3.10+ python style: google imports: isort code style: black code style: pycodestyle doc style: pydocstyle static typing: mypy linting: pylint testing: pytest security: bandit license: MIT

Atoparser

Atoparser is a zero dependency Atop log processing library written in Python. The library supports reading binary C data directly from compressed or uncompressed Atop log files, without the need to install Atop or call a subprocess. The converted data contains structured Python objects, that can then be used for JSON, CSV, or other types of output, storage, and analysis.

For full information on the amazing performance monitoring software that creates these files, known as "Atop", refer to:
Atop - The one stop shop for all your tops

Table Of Contents

Compatibility

  • Supports Python 3.10+
  • Supports Atop 1.26 and 2.3 through 2.10.

Getting Started

Installation

Install Atoparser via pip:

pip install atoparser

Or via git clone:

git clone <path to fork>
cd atoparser
pip install .

Or build and install from wheel:

# Build locally.
git clone <path to fork>
cd atoparser
make wheel

# Push dist/atoparser*.tar.gz to environment where it will be installed.
pip install dist/atoparser*.tar.gz

How Tos

Read an Atop log with the example JSON command:

atoparser ~/atop.log -P CPU --pretty

Iterate over the C structs as Python objects:

from atoparser import atop_helpers

with open(file, 'rb') as raw_file:
    header = atop_helpers.get_header(raw_file)
    for record, sstat, tstat in atop_helpers.generate_statistics(raw_file, header):
        total_cycles = record.interval * sstat.cpu.nrcpu * header.hertz
        usage = 1 - sstat.cpu.all.itime / total_cycles
        print(f'CPU usage was {usage:.02%}')

Convert the C structs into JSON compatible objects:

import json
from atoparser import atop_helpers

with open(file, 'rb') as raw_file:
    header = atop_helpers.get_header(raw_file)
    print(json.dumps(atop_helpers.struct_to_dict(header), indent=2))

Add a new version

  1. Copy the previous struct definition in atoparser/atop_structs/ to a new file by the <major>_<minor> version.

  2. Update any individual struct definitions as needed. Additional guidelines for this process outlined in the files.

  3. Update the atoparser/atop_helpers.py file to include the new version in the imports and _VERSIONS list.

  4. Use the utils/build_atop.sh script to generate a new sample Atop log file, and place in atoparser/test/files/.

  5. Compress the new log with gzip to reduce storage overhead.

  6. Add new tests to atoparser/test/test_atoparser.py to ensure the new version is parsed correctly.

  7. Update the README.md file to include the new supported version.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

atoparser-3.0.0.tar.gz (23.0 kB view hashes)

Uploaded Source

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page