olog-tool 0.2.0

A command-line tool for backing up, restoring, and generating reports from Olog servers

Olog Tool

A command-line tool for backing up, restoring, and generating reports from Olog servers using the Olog REST API.

Features

• Backup: Complete backup of logs and attachments from an Olog server
• Restore: Restore logs and attachments from a backup
• Metadata Backup: Backup of tags, properties, logbooks, levels, and templates
• Metadata Restore: Restore metadata entities from a backup
• Individual Creation: Create templates, properties, and logbooks individually
• Markdown Reports: Generate Markdown documentation of logs with custom titles
• PDF Reports: Generate PDF reports with logs, metadata, and embedded images with custom titles
• HTML Reports: Generate HTML reports with logs, metadata, and embedded images with custom titles

Understanding Olog Properties

Properties are structured metadata fields that can be attached to log entries and templates. They provide a way to add custom, structured information beyond the basic title, description, tags, and logbooks.

Property Structure

Each property has:

• Name: Unique identifier (e.g., "resource", "experiment", "shift")
• Attributes: Key-value pairs that define the property's data fields
• State: "Active" or "Inactive"

Example Property

{
  "name": "resource",
  "state": "Active",
  "attributes": [
    {
      "name": "type",
      "value": "beamline",
      "state": "Active"
    },
    {
      "name": "name",
      "value": "BL01",
      "state": "Active"
    }
  ]
}

Properties in Templates

Templates can include properties to pre-fill structured data. When users create logs from templates, they can modify the property values:

{
  "name": "Beamline Incident Report",
  "title": "Report beamline incident with resource details",
  "properties": [
    {
      "name": "resource",
      "attributes": [
        {"name": "type", "value": null},
        {"name": "name", "value": null}
      ]
    }
  ]
}

Properties in Log Entries

Log entries can include properties with actual values:

{
  "title": "Beamline BL01 vacuum issue",
  "description": "Vacuum pressure increased unexpectedly",
  "properties": [
    {
      "name": "resource",
      "attributes": [
        {"name": "type", "value": "beamline"},
        {"name": "name", "value": "BL01"}
      ]
    }
  ]
}

Note: Properties must be defined at the system level before they can be used in templates or log entries.

Creating Custom Properties

To create custom properties, you need to modify the metadata.json file in your backup directory and restore it. Properties are defined at the system level and can then be referenced by templates and log entries.

Example of adding a custom property:

{
  "name": "experiment",
  "state": "Active", 
  "attributes": [
    {
      "name": "id",
      "value": null,
      "state": "Active"
    },
    {
      "name": "proposal",
      "value": null,
      "state": "Active"
    },
    {
      "name": "scientist",
      "value": null,
      "state": "Active"
    }
  ]
}

After adding custom properties to your metadata.json, use the restore-metadata command to create them on the target server.

Installation

From PyPI (Recommended)

pip install olog-tool

From Source

1. Clone or download this repository
2. Install dependencies:

pip install -r requirements.txt

Or install in development mode:

pip install -e .

Usage

Basic Syntax

olog-tool --url <olog-server-url> [OPTIONS] COMMAND

Authentication

Use the --username and --password options for authenticated access:

olog-tool --url https://olog.example.com --username myuser --password mypass COMMAND

Time Range

Use --start and --end to specify a time range (ISO format):

olog-tool --url https://olog.example.com --start 2025-01-01T00:00:00 --end 2025-12-31T23:59:59 COMMAND

Commands

Backup

Backup all logs and attachments from the Olog server:

olog-tool --url https://olog.example.com --username user --password pass backup --output ./backup_dir

With time range:

olog-tool --url https://olog.example.com \
  --start 2025-01-01T00:00:00 \
  --end 2025-01-31T23:59:59 \
  backup --output ./backup_january

Backup Metadata

Backup all metadata entities (tags, properties, logbooks, levels, and templates) from the Olog server:

olog-tool --url https://olog.example.com --username user --password pass backup-metadata --output ./metadata_backup

Restore Metadata

Restore metadata entities from a backup directory:

olog-tool --url https://olog.example.com --username user --password pass restore-metadata --input ./metadata_backup

Note: Metadata restore operations create new entities, so IDs will be different from the original server.

Create Template

Create a new template with specified metadata and optional source text:

olog-tool --url https://olog.example.com --username user --password pass create-template \
  --name "maintenance-template" \
  --title "Equipment Maintenance" \
  --level "Info" \
  --source "# Maintenance Checklist\n\n- Equipment: \n- Issue: \n- Action: " \
  --logbooks "operations,maintenance" \
  --tags "maintenance,checklist" \
  --properties '[{"name": "resource", "attributes": [{"name": "type", "value": null}, {"name": "name", "value": null}]}]'

Create Property

Create a new property with specified attributes:

olog-tool --url https://olog.example.com --username user --password pass create-property \
  --name "experiment" \
  --attributes '[{"name": "id", "value": null, "state": "Active"}, {"name": "name", "value": null, "state": "Active"}]'

Create Logbook

Create a new logbook:

olog-tool --url https://olog.example.com --username user --password pass create-logbook \
  --name "beamline-a" \
  --owner "operator"

Properties in Templates

When creating templates with properties, you must reference existing properties defined in the Olog system. Each property reference must include ALL attributes defined for that property in the system.

Critical Note: When creating templates with properties, ALL attributes must have "value": null. This allows users to fill in the actual values when creating log entries from the template. Providing non-null values will result in a 400 error.

Example Template with Properties:

{
    "name": "Maintenance Template",
    "title": "Beamline maintenance checklist",
    "level": "Info",
    "logbooks": [{"name": "operations"}],
    "tags": [{"name": "maintenance"}],
    "properties": [
        {
            "name": "resource",
            "attributes": [
                {"name": "type", "value": null},
                {"name": "name", "value": null},
                {"name": "location", "value": null}
            ]
        }
    ]
}

Important:

• Properties must already exist in the Olog system
• All attributes defined for the property must be included
• Each attribute must have name, value, and state fields
• Always use "value": null for template properties to allow user input

Generate Markdown Report

Create a Markdown document with logs and their metadata:

olog-tool --url https://olog.example.com markdown --output report.md

With time range and authentication:

olog-tool --url https://olog.example.com \
  --username user --password pass \
  --start 2025-01-01T00:00:00 \
  --end 2025-01-31T23:59:59 \
  markdown --output january_report.md

Generate PDF Report

Create a PDF document with logs and their metadata. Image attachments are automatically embedded in the PDF:

olog-tool --url https://olog.example.com pdf --output report.pdf

With time range and authentication:

olog-tool --url https://olog.example.com \
  --username user --password pass \
  --start 2025-01-01T00:00:00 \
  --end 2025-01-31T23:59:59 \
  pdf --output january_report.pdf

To disable embedding of image attachments (faster, smaller file):

olog-tool --url https://olog.example.com pdf --output report.pdf --no-embed

Generate HTML Report

Create an HTML document with logs and their metadata. Image attachments are automatically embedded as base64:

olog-tool --url https://olog.example.com html --output report.html

With time range and authentication:

olog-tool --url https://olog.example.com \
  --username user --password pass \
  --start 2025-01-01T00:00:00 \
  --end 2025-01-31T23:59:59 \
  html --output january_report.html

To disable embedding of image attachments:

olog-tool --url https://olog.example.com html --output report.html --no-embed

Examples

Example 1: Monthly Backup

Backup all logs from October 2025:

olog-tool \
  --url https://olog.example.com \
  --username admin \
  --password secret \
  --start 2025-10-01T00:00:00 \
  --end 2025-10-31T23:59:59 \
  backup --output ./backup_2025_10

Example 2: Generate Reports with Custom Titles

Generate Markdown, HTML, and PDF reports for a specific time range with custom titles:

# Markdown report with custom title
olog-tool \
  --url https://olog.example.com \
  --start 2025-11-01T00:00:00 \
  --end 2025-11-07T23:59:59 \
  --title "Weekly Operations Summary" \
  markdown --output weekly_report.md

# HTML report with embedded images and custom title
olog-tool \
  --url https://olog.example.com \
  --start 2025-11-01T00:00:00 \
  --end 2025-11-07T23:59:59 \
  --title "Beamline Status Report" \
  html --output weekly_report.html

# PDF report with embedded images and custom title
olog-tool \
  --url https://olog.example.com \
  --start 2025-11-01T00:00:00 \
  --end 2025-11-07T23:59:59 \
  --title "Weekly Operations Report" \
  pdf --output weekly_report.pdf

Example 3: Complete Backup and Restore

Backup from one server and restore to another:

# Backup from production
olog-tool \
  --url https://olog-prod.example.com \
  --username admin \
  --password prod_pass \
  backup --output ./prod_backup

# Restore to development
olog-tool \
  --url https://olog-dev.example.com \
  --username admin \
  --password dev_pass \
  restore --input ./prod_backup

Example 4: Metadata Backup and Restore with Templates

Backup and restore metadata including custom templates:

# Backup metadata from production
olog-tool \
  --url https://olog-prod.example.com \
  --username admin \
  --password prod_pass \
  backup-metadata --output ./metadata_backup

# Restore metadata to development (includes templates)
olog-tool \
  --url https://olog-dev.example.com \
  --username admin \
  --password dev_pass \
  restore-metadata --input ./metadata_backup

Example 5: Creating Individual Metadata Entities

Create custom properties, logbooks, and templates individually:

# Create a custom property for experiments
olog-tool \
  --url https://olog.example.com \
  --username admin \
  --password pass \
  create-property \
  --name "experiment" \
  --attributes '[{"name": "id", "value": null, "state": "Active"}, {"name": "name", "value": null, "state": "Active"}, {"name": "type", "value": null, "state": "Active"}]'

# Create a new logbook
olog-tool \
  --url https://olog.example.com \
  --username admin \
  --password pass \
  create-logbook \
  --name "beamline-maintenance" \
  --owner "maintenance-team"

# Create a template with the new property and logbook
olog-tool \
  --url https://olog.example.com \
  --username admin \
  --password pass \
  create-template \
  --name "maintenance-report" \
  --title "Beamline Maintenance Report" \
  --level "Info" \
  --source "# Maintenance Report\n\n**Issue:** \n**Action Taken:** \n**Next Steps:** " \
  --logbooks "beamline-maintenance,operations" \
  --tags "maintenance,beamline" \
  --properties '[{"name": "experiment", "attributes": [{"name": "id", "value": null, "state": "Active"}, {"name": "name", "value": null, "state": "Active"}, {"name": "type", "value": null, "state": "Active"}]}]'

Create a custom template with properties by editing the metadata.json file in your backup directory. Ensure properties reference existing system properties with all required attributes.

Backup Directory Structure

When you create a backup, the following structure is created:

backup_dir/
├── logs.json              # JSON file with all log metadata
├── metadata.json          # JSON file with all metadata entities (tags, properties, logbooks, levels, templates)
└── attachments/           # Directory containing attachments
    ├── 123/               # Directory for log ID 123
    │   ├── image1.png
    │   └── data.csv
    └── 124/               # Directory for log ID 124
        └── document.pdf

For metadata-only backups, only the metadata.json file is created.

Requirements

• Python 3.6 or higher
• requests library
• reportlab library (for PDF generation)

API Reference

This tool uses the Olog REST API. The main endpoints used are:

• GET /logs - Search for logs
• GET /logs/{id} - Get a specific log
• GET /logs/attachments/{id}/{filename} - Download attachment
• PUT /logs - Create a new log
• POST /logs/attachments/{id} - Upload attachment

Limitations

• The backup/restore process creates new log entries, so log IDs will be different
• Very large backups may take significant time and disk space
• PDF generation requires the reportlab library
• Image embedding in PDF/HTML works for common formats (PNG, JPG, GIF, BMP, SVG)
• Large images are automatically scaled in PDF to fit the page
• HTML reports with many embedded images may be large files
• Properties in templates must reference existing system properties with all required attributes
• Template properties are validated against system property definitions

License

This tool is provided as-is for use with Olog servers.