FlexMetric

FlexMetric is a lightweight, pluggable, and extensible Prometheus exporter that helps you collect, expose, and visualize custom metrics with minimal setup and maximum flexibility.

Features

• Command Execution: Run system commands and expose their output as Prometheus metrics
• Database Monitoring: Execute SQL queries against SQLite, PostgreSQL, and ClickHouse databases
• Python Functions: Call custom Python functions and export their results as metrics
• REST API: Receive externally submitted metrics via a secure Flask-based API with optional HTTPS (TLS)
• Security First: Built-in command and query sanitization to prevent dangerous operations
• Prometheus Ready: All metrics are exposed in Prometheus-compatible format for Grafana visualization

Safety Features

• Command Safety: Harmful commands (file deletion, system shutdown) are automatically blocked
• Query Protection: Potentially dangerous SQL queries (DROP, DELETE, TRUNCATE) are blocked by default
• Input Validation: Comprehensive sanitization and validation ensure only safe operations are executed

Installation

Install FlexMetric from PyPI:

pip install flexmetric

Quick Start

Run FlexMetric with command execution mode:

flexmetric --commands --commands-config commands.yaml --port 8000

Access your metrics at: http://localhost:8000/metrics

Available Modes

FlexMetric supports multiple modes that can be used individually or combined:

Using Multiple Modes Together

flexmetric \
  --commands --commands-config commands.yaml \
  --database --database-config database.yaml --queries-config queries.yaml \
  --functions --functions-file executable_functions.txt \
  --expose-api --port 8000

Configuration Examples

Commands Mode

Create a commands.yaml file to define system commands to execute:

commands:
  - name: disk_usage
    command: df -h
    main_label: disk_usage_filesystem_mount_point
    labels: ["filesystem", "mounted"]
    label_columns: [0, -1]
    value_column: 4
    timeout_seconds: 60

Example command output:

Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        50G   20G   28G  42% /
/dev/sdb1       100G   10G   85G  10% /data

Configuration Fields

Database Mode

FlexMetric supports multiple databases with simple YAML configuration:

Supported Databases

Database Configuration (database.yaml)

databases:
  - id: "local_sqlite"
    type: "sqlite"
    db_connection: "/path/to/example.db"

  - id: "analytics_pg"
    type: "postgres"
    host: "localhost"
    port: 5432
    database: "metricsdb"
    username: "postgres"
    password: "postgres_password"
    sslmode: "disable"

  - id: "clickhouse_cluster"
    type: "clickhouse"
    host: "localhost"
    port: 8443
    username: "default"
    password: "clickhouse_password"
    secure: true

Query Configuration (queries.yaml)

commands:
  - id: "active_user_count_pg"
    type: "postgres"
    database_id: "analytics_pg"
    query: |
      SELECT
        country AS country_name,
        COUNT(*) AS active_user_count
      FROM users
      WHERE is_active = true
      GROUP BY country;
    main_label: "active_user_count"
    labels: ["country_name"]
    value_column: "active_user_count"

Functions Mode

Create an executable_functions.txt file listing your Python functions:

function_name_1
function_name_2

Python Function Output Format

Each function should return a dictionary:

{
    'result': [
        { 'label': [label_value1, label_value2, ...], 'value': numeric_value }
    ],
    'labels': [label_name1, label_name2, ...],
    'main_label': 'your_main_metric_name'
}

Flask API Mode

Start FlexMetric with the Flask API:

flexmetric --expose-api --port 5000 --host 0.0.0.0

Endpoints:

• Metrics: http://localhost:5000/metrics
• API: http://localhost:5000/update_metric

Submit Metrics via API

curl -X POST http://localhost:5000/update_metric \
-H "Content-Type: application/json" \
-d '{
  "result": [
    { 
      "label": ["cpu", "core_1"], 
      "value": 42.5 
    }
  ],
  "labels": ["metric_type", "core"],
  "main_label": "cpu_usage_metric"
}'

Secure HTTPS Mode

flexmetric --expose-api --enable-https --ssl-cert=cert.pem --ssl-key=key.pem

HTTPS Endpoints:

• Metrics: https://localhost:5000/metrics
• API: https://localhost:5000/update_metric

Submit Metrics via HTTPS API

curl -k -X POST https://localhost:5000/update_metric \
-H "Content-Type: application/json" \
-d '{
  "result": [
    { 
      "label": ["cpu", "core_1"], 
      "value": 42.5 
    }
  ],
  "labels": ["metric_type", "core"],
  "main_label": "cpu_usage_metric"
}'

Python HTTPS Client Example

import requests
import json
import urllib3

# Disable SSL warnings for self-signed certificates
urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)

url = "https://localhost:5000/update_metric"
headers = {
    "Content-Type": "application/json"
}
data = {
    "result": [
        {
            "label": ["cpu", "core_1"],
            "value": 42.5
        }
    ],
    "labels": ["metric_type", "core"],
    "main_label": "cpu_usage_metric"
}

# Use verify=False for self-signed certificates (not recommended for production)
response = requests.post(url, headers=headers, data=json.dumps(data), verify=False)

print(f"Status Code: {response.status_code}")
print("Response:", response.text)

SSL Certificate Generation

For development/testing, you can generate self-signed certificates:

# Generate private key
openssl genrsa -out key.pem 2048

# Generate certificate
openssl req -new -x509 -key key.pem -out cert.pem -days 365 -subj "/C=US/ST=State/L=City/O=Organization/CN=localhost"

Note: For production use, use certificates from a trusted Certificate Authority (CA).

Command-Line Options

Example Output

FlexMetric exposes metrics in Prometheus format:

# HELP disk_usage_gauge Disk usage percentage
# TYPE disk_usage_gauge gauge
disk_usage_gauge{filesystem="/dev/sda1",mounted="/"} 42.0
disk_usage_gauge{filesystem="/dev/sdb1",mounted="/data"} 10.0

Future Enhancements

• Support for additional databases (MySQL, MongoDB)
• Enhanced label extraction and parsing
• Web UI for configuration management
• Advanced security features and authentication