kymoscope-collector

Site collector for Kymoscope. It runs on one host in a plant (normally the DMZ), reads an OPC UA server, a folder of logger files or your own Python source, encrypts the data and publishes it as Kymoscope streams. Outbound connections only; the disk buffer holds ciphertext.

Install

pip install kymoscope-collector              # core: OPC UA, CSV files, Python sources
pip install "kymoscope-collector[files]"     # + Parquet, TDMS and MF4 files
pip install "kymoscope-collector[windows]"   # + Windows service

Requires Python 3.10 to 3.13 (python.org or your distribution; the Microsoft Store Python cannot host the Windows service and service install refuses it).

• Docker: ghcr.io/javijur/kymoscope-collector (mount a folder with collector.yaml at /data). The container runs as uid 10001, so give it the folder: sudo chown -R 10001:10001 /srv/kymoscope (or use --user). It stops with a clear message when /data is not writable.
• Linux (systemd): sudo useradd --system --no-create-home --shell /usr/sbin/nologin kymoscope, then copy the unit file to /etc/systemd/system/. It ships in the package at <site-packages>/kymoscope_collector/packaging/kymoscope-collector.service (and in the repository at packages/collector/packaging/). Uncomment ReadWritePaths= for any folder a files source writes to (ProtectSystem=strict).
• Windows service: runs as LocalSystem by default. For a dedicated account: sc config KymoscopeCollector obj= .\kymoscope password= ..., plus the "Log on as a service" right (sc config doesn't grant it), read access to the Python installation and the venv (so not a Python installed under C:\Users), modify rights on state_dir and on every watched folder (its done and failed folders too), and read access to state_dir\pki\own\key.pem if it already exists (it is private to the account that created it and doesn't inherit: icacls ...\pki\own\key.pem /grant kymoscope:R). The service restarts itself after any failure.

Before installing (for the plant's IT)

• Application control (WDAC, AppLocker, Smart App Control): the service host pythonservice.exe from pywin32 is not code-signed (python.exe is). If unsigned binaries are blocked, allow it by path or hash, use Docker or WSL, or wait for the signed installer.
• Outbound TCP 443 to Kymoscope, directly or through an HTTP proxy (below). Nothing inbound.
• Python 3.10 to 3.13, installed for all users if the service runs under a dedicated account. Not the Microsoft Store Python.

Updating

pip can't replace files while the service runs (WinError 32), so stop it first:

kymoscope-collector service stop --config C:\ProgramData\Kymoscope\collector.yaml
pip install -U "kymoscope-collector[windows]"
kymoscope-collector service start --config C:\ProgramData\Kymoscope\collector.yaml

Docker: docker pull ghcr.io/javijur/kymoscope-collector, then remove and recreate the container with the same options. systemd: systemctl stop kymoscope-collector, pip install -U kymoscope-collector in the unit's environment, systemctl start kymoscope-collector. The buffer and certificates in state_dir survive the update.

Proxy

proxy: env:KYMOSCOPE_PROXY      # http://host:port or http://user:password@host:port (percent-encode @ : \)
no_proxy: [historian.plant.local]

Both the HTTPS calls and the relay WebSocket tunnel through it (HTTP CONNECT, Basic auth); OPC UA never does. A Windows service doesn't see a user's Windows proxy settings, so set it here. The log never shows the credentials.

Code-signed installers are coming.

Quick start (no plant needed)

kymoscope-collector simulate                                   # demo OPC UA server on port 4840
kymoscope-collector browse --url opc.tcp://localhost:4840 --out tags.csv
kymoscope-collector run --config collector.yaml

A minimal collector.yaml:

api_key: env:KYMOSCOPE_API_KEY
sources:
  - name: Demo plant
    type: opcua
    url: opc.tcp://localhost:4840
    security: None
    allow_insecure: true      # demo server only
    tags: tags.csv

Notes:

• Unknown keys in collector.yaml are an error that lists the valid ones; a password with security: None is allowed but logged as a warning.
• Editing tags.csv reconnects the source, so its tags have a short gap.
• One OPC UA source handles about 2,000 tags reporting at distinct millisecond times per second. Split larger tag lists into several type: opcua sources with different tags files.
• The enumeration labels are not sent. Blocks wait in a memory queue bounded by buffer.memory_mb (default 256) while a stream cannot register; the oldest are dropped and counted in <source>.queue_dropped.
• The private key state_dir/pki/own/key.pem is created owner-only (0600; on Windows user, SYSTEM and Administrators).

Other commands: cert (show or renew the collector's certificate), service install|remove|start|stop (Windows).

Documentation

Configuration reference, certificates, security notes, health stream and troubleshooting: https://kymoscope.com/docs/collector