sciveo 0.2.20

No project description provided

SCIVEO - ML/AI and Scientific tools

Sciveo is a practical toolkit for running, observing, and automating scientific, ML, AI, and media systems. The package is intentionally broad, but the main modules are ordered below by day-to-day operational value and how often they are expected to be used.

Module Positioning

1. Monitoring and Alerts - the primary operations module for machine telemetry, GPU/server health, server power usage, alert rules, alert history, and notification delivery.
2. Agent Console and Local LLM Runtime - an interactive and one-shot coding/research assistant with local tools, image inputs, hosted providers, and owned local Hugging Face/GGUF runtimes.
3. Notifications - reusable Slack and Viber notification providers used by monitoring alerts and available to other Sciveo components.
4. Network and Industrial IO - network scanning, Modbus read/write, and Modbus device emulation for lab, edge, and plant-facing integrations.
5. Media Capture and Processing - RTSP/NVR capture, screen/camera capture, and media pipeline workers.
6. Experiments Management Client - experiment tracking, parameter sampling, scoring, plots, and remote synchronization with sciveo.com.
7. Supporting API, DB, Web, Content, ML, and Tools packages - shared client APIs, storage helpers, web/server utilities, content abstractions, ML helpers, and daemon infrastructure used by the modules above.

CLI Help and Manifest

Every Sciveo CLI command has a focused help page:

sciveo --help
sciveo watchdog --help
sciveo help watchdog

The same command metadata is available as a JSON manifest for agents, scripts, and documentation tooling:

sciveo help --json
sciveo help watchdog --json
sciveo agent --help --json

The JSON output is the machine-readable source for command summaries, usage forms, options, notes, and examples.

Monitoring and Alerts

The monitoring module is the main Sciveo operations component. It monitors machines ranging from small edge devices to large GPU servers and sends samples to the configured Sciveo API. The local configuration is created with sciveo init and normally lives in ~/.sciveo/.

The monitor currently collects:

• CPU: total usage and per-core usage.
• RAM: used, total, and free memory.
• DISK: read/write counters and speeds.
• NET: sent/received counters and speeds.
• TEMP: available host temperature sensors.
• GPU: GPU telemetry from the GPU monitor, including GPU power draw when available.
• POWER: whole-machine/server power, currently stored as POWER system in watts.
• INFO: human-readable metadata and collector debug information.

Server power is intentionally separate from GPU power. GPU power remains under GPU, while POWER system means whole-machine power. When the machine exposes a supported server power meter, Sciveo reports it automatically. When power is not available, the metric is simply omitted instead of sending a fake zero value.

Start monitoring the current machine with a 60 second period:

sciveo monitor --period 60

Write samples to a local file as well. When a directory is passed, the default file name is sciveo_monitor.json:

sciveo monitor --period 120 --output-path ./

Start monitoring from Python without blocking the rest of the program:

import sciveo

sciveo.monitor(period=120, block=False)

Plant/EMS monitoring is separate from server power monitoring and uses the plant source:

sciveo monitor --src plant --host 192.168.1.50 --port 502 --period 60 --serial ems300-1

Alert Rules

The Sciveo web monitoring layer evaluates alert definitions against the latest monitoring samples. Alerts are configured per owner, can be made public by staff/admin users, and can target one or more monitor serials. Public alert definitions are global and visible to all users.

Alert definitions have a type:

• metric_threshold: evaluates a numeric metric from monitoring samples.
• alive: evaluates how old the latest sample is and detects stale or missing monitors.

Metric alert fields:

• Metric: a metric key such as CPU usage, RAM usage, POWER system, GPU power.draw, or any metric name that exists in the flattened monitor sample.
• Window/Fresh time: how many recent minutes of samples are considered for evaluation.
• Policy/Aggregation: latest, max, min, or avg.
• Direction: above when higher values are worse, below when lower values are worse.
• Thresholds: any number of ordered severity thresholds. Each threshold has a severity name, a color, and a numeric value.

The severity name is the operational label shown in the UI and notification text. The color is presentation metadata used for badges and Slack color bars. The default color labels are only defaults: if the severity name is changed from caution to down, then down is the severity label for that threshold.

Example CPU alert for a service that should keep the CPU busy:

Type: metric_threshold
Metric: CPU usage
Window/Fresh time: 5 min
Policy: max
Direction: below
Thresholds:
  OK       green   50
  idle     yellow  30
  stopped  red     10

This evaluates the maximum CPU usage in the last five minutes. A short restart dip below 50% will not trigger if at least one sample in the window is still above 50%. If no samples arrive at all, use a separate alive alert to detect that condition.

Example server power alert:

Type: metric_threshold
Metric: POWER system
Window/Fresh time: 5 min
Policy: latest
Direction: above
Thresholds:
  normal    green   1200
  warning   yellow  1800
  critical  red     2200

Alive alerts use the synthetic metric Sample age minutes. They can also have any number of thresholds:

Type: alive
Metric: Sample age minutes
Policy: latest
Thresholds:
  OK       green   0
  down     yellow  5
  offline  red     15

In a Sciveo web deployment, alert evaluation and notification delivery run as background services. Users normally configure alert rules, notification targets, and alert history from the monitoring/admin UI.

Alert Events, History, and Time

Alert events are created on severity transitions. The current state table shows the latest state per alert and monitor, while the event history keeps transition records. The UI shows today's events first and loads older history by date with normal paginated API responses. Owners are shown where permissions allow, because staff/admin users can see a wider alert set than regular users.

Alert and notification timestamps are stored as UTC and serialized by the API with an explicit UTC timezone. The web UI renders them in the browser/client local time.

Notifications

Notification targets are standalone objects. Alerts select one or more notification targets, and when an alert event is created the selected targets receive the message. This keeps Slack/Viber configuration reusable by other components instead of hard-coding notification behavior into alert definitions.

The Sciveo notification library currently includes Slack and Viber providers. Slack messages use a short title plus a colored side bar derived from the alert color; the body includes the alert name, severity label, monitor, metric, value, threshold, owner, and evaluation details.

Watchdog Monitoring With Action

Watchdogs are local process-style guards that execute a command when a simple local condition is exceeded. They are useful for emergency cleanup or local remediation, and are separate from the web alert system.

Memory usage watchdog for 90% max memory, measured every 10 seconds:

sciveo watchdog --src memory --threshold 90 --period 10 --execute "<some command to handle the high memory event>"

Disk usage watchdog for 80% max disk usage on a path, measured every 600 seconds:

sciveo watchdog --src disk --input-path "/some/path/to/watch" --threshold 80 --period 600 --execute "find /some/path/to/watch -type f -mtime +1"

Network/internet connectivity watchdog checks a list of host:port targets and executes an action when the selected policy is unhealthy:

sciveo watchdog \
  --src network \
  --targets "1.1.1.1:443,8.8.8.8:53,google.com:443" \
  --policy at-least-one \
  --threshold 3 \
  --period 30 \
  --timeout 2 \
  --execute "echo internet outage"

--targets accepts either a comma-separated list or a JSON list:

sciveo watchdog --src network --targets '["1.1.1.1:443", "8.8.8.8:53"]' --execute "echo outage"

Network watchdog policies:

• at-least-one or any: healthy when at least one target is reachable. This is the usual internet outage policy.
• all: healthy only when every target is reachable.
• majority: healthy when more than half of the targets are reachable.
• count:N: healthy when at least N targets are reachable, for example count:2.

For the network watchdog, --threshold means consecutive failed checks before the action runs. For example, --threshold 3 --period 30 waits for about 90 seconds of failed checks before triggering.

Agent Console and Local LLM Runtime

The agent module is the interactive AI workbench in Sciveo. It can run as a REPL or one-shot command, use local workspace tools, attach images, talk to hosted providers, or connect to a Sciveo-owned local Hugging Face/GGUF runtime.

The console includes basic local tools such as bash, read_file, write_file, list_dir, and pwd. Approval controls allow you to choose how much the agent can do automatically.

Start a local coding-agent console:

sciveo agent --provider auto

OPENAI_API_KEY=<your-api-key> sciveo agent

Run a one-shot prompt instead of opening the REPL:

sciveo agent --provider auto --prompt "List the repo root and summarize the important files"

OPENAI_API_KEY=<your-api-key> sciveo agent --model gpt-5.4-mini --prompt "Summarize the latest changes in this repo"

openai is the default agent provider when OPENAI_API_KEY is configured, with gpt-5.4 as the default model. --provider auto selects from configured API keys. A remote URL can also be enough to select the runtime:

sciveo agent --provider http://127.0.0.1:8910 --mode stream
sciveo agent --url http://127.0.0.1:8910 --mode stream

When --provider itself is a URL, or when --url/--host/--port is provided with auto, Sciveo treats it as a remote Hugging Face-compatible runtime endpoint. --provider local is an alias for hf. --provider ollama and --provider olama are supported as compatibility providers.

Attach a local image path or base64 image to the next agent prompt. In the REPL, attachments are one-turn only:

sciveo agent --provider openai --prompt "Describe this image" --input-path ./screenshot.png
sciveo agent --provider hf --prompt "What is visible here?" --data-json '{"images":[{"path":"./screenshot.png"}]}'
sciveo agent --provider gemini --prompt "Read this chart" --data-json '{"images":[{"data":"data:image/png;base64,...","name":"chart.png"}]}'

Inside the REPL:

/image ./screenshot.png
/image-data image/png iVBORw0KGgo...
/images
/image-clear

Install hosted agent providers with the agent extra:

pip install "sciveo[agent]"

Install the owned local Hugging Face runtime with:

pip install "sciveo[agent-local]"

Pull a model snapshot from Hugging Face, run the Sciveo local runtime server, then connect the agent console to it:

sciveo agent --action pull --model Qwen/Qwen2.5-Coder-3B-Instruct --alias qwen-coder-3b
sciveo agent --action run --model qwen-coder-3b --host 127.0.0.1 --port 8910 --device mps --context 8192
sciveo agent --provider hf --prompt "Say hello in one sentence"
sciveo agent --provider hf --mode batch --prompt "Say hello in one sentence"

For GGUF repositories with many quantized files, use --file to pull only one model file:

sciveo agent --action pull --model unsloth/Qwen3.5-4B-GGUF --file Qwen3.5-4B-Q4_K_M.gguf --alias qwen3.5-4b-q4

For GGUF vision models, also pull the matching projector file:

sciveo agent --action pull \
  --model some/vlm-gguf \
  --file model-Q4_K_M.gguf \
  --data-json '{"mmproj_file":"mmproj-model.gguf"}' \
  --alias vlm-q4

The HF runtime streams live output by default; use --mode batch for request/response style. Use --device cpu, --device mps, --device cuda, or --device auto on agent --action run to choose where the local model runs. Use --context to choose the llama.cpp context size for GGUF models; larger values use more memory. GGUF model snapshots are supported through llama-cpp-python; on Apple Silicon, --device mps requests llama.cpp Metal acceleration when the installed backend supports it.

Network and Industrial IO

• Network Scan Networks of hosts to scan for a single port

sciveo scan --port 22

sciveo scan --net 192.168.0.0/24 --port 22 --timeout 0.5

• Network Scan Single Host list of ports to scan

sciveo scan --host 192.168.0.10

sciveo scan --host 192.168.0.10 --ports [443,80]

• SSH Scan And Execute Scan a network for SSH, try one or more users with an identity file, and run remote commands

sciveo ssh \
  --net 10.37.0.0/24 \
  --port 22 \
  -i ~/develop/aws/smiveo/sciveo_stan_ed25519 \
  --users "stan,sgeorgiev,ubuntu,ec2-user" \
  --execute hostname \
  --render online

sciveo ssh \
  --host "ubuntu@10.37.0.18,ec2-user@10.37.0.22" \
  -i ~/.ssh/id_ed25519 \
  --execute '["hostname", "uptime"]' \
  --render text

--host accepts host, user@host, comma-separated hosts, or a JSON list. --users accepts a comma-separated list or a JSON list. --execute accepts a single command, newline-separated commands, or a JSON list of commands. --render online prints a compact connected/failed table; --render json returns structured results.

• Modbus Read Holding or input registers over TCP

sciveo read --proto modbus --transport tcp --host 192.168.0.10 --port 502 --id 1 --address 30001 --kind input --type RAW --count 2

sciveo read --proto modbus --transport serial --serial-port /dev/ttyUSB0 --baudrate 9600 --bytesize 8 --parity N --stopbits 1 --id 1 --reg '[5039, "U16", 0.1, 1]'

• Modbus Write Holding registers over TCP or serial

sciveo write --proto modbus --transport tcp --host 192.168.0.10 --id 1 --reg '[40010, "U16", 1, 1]' --value 123

sciveo write --proto modbus --transport serial --serial-port /dev/ttyUSB0 --baudrate 9600 --bytesize 8 --parity N --stopbits 1 --id 1 --address 10 --type RAW --value "[1,2]"

--reg accepts a compact value spec like [address, type, factor, count]. --proto currently defaults to modbus.

• Modbus Emulate Run a Modbus TCP emulator with a bundled profile or a custom profile override

sciveo emulate --server modbus --profile ems300 --host 0.0.0.0 --port 1502

sciveo emulate --server modbus --profile custom --host 0.0.0.0 --port 1502 --data-json '{"device_id": 7, "input": {"30001": [10, 20]}, "holding": {"40010": 123}}'

--profile ems300 starts the bundled EMS300 preset. Other bundled profiles include logger1000-sg for Sungrow Logger1000 and logger1000-hw for Huawei SmartLogger1000. --host 0.0.0.0 binds the emulator on all interfaces; use 127.0.0.1 for local-only testing. --data-json can supply or override emulator register data using external register numbers.

Media Capture and Processing

Remote Network Video Recorder

• NVR RTSP cams video recorder

sciveo nvr --input-path cams.json

where cams.json has the cam and nvr configuration like

{
  "video_retention_period": 5,
  "max_video_len": 60,
  "transport": "tcp",
  "path": {
    "tmp": "tmp/nvr/tmp",
    "video": "tmp/nvr/video"
  },
  "cam": {
    "cam_livingroom": {
      "url": "rtsp://192.168.1.11/stream1"
    },
    "cam_kitchen": {
      "url": "rtsp://rtsp://192.168.1.12"
    }
  }
}

Media Pipelines Processing Worker

Media pipelines turn uploaded or captured content into derived content: resized images, video frames, albums, extracted audio, waveforms, metadata, embeddings, generated captions, object detections, time-series artifacts, and similar outputs. A pipeline job has input content, a processor graph, and optional postprocessors that store or attach generated outputs.

Start the media pipeline worker:

sciveo media-server

The worker consumes configured media jobs and runs the requested processors. Common processor names include:

• media-info: read image/video metadata.
• image-resize: create resized image variants.
• image-histogram, image-filters, image-watermark: image analysis and transformation.
• image-to-text, image-object-detection, image-segmentation, image-depth-estimation, image-embedding: ML-oriented image processors.
• video-resize, video-downsample, video-frames-extract, video-motion-detector, video-album, video-to-text: video processors.
• audio-extract: extract audio and create waveform images.
• archive-zip: archive generated or selected files.
• sci-timeseries-trainer, sci-timeseries-predictor, project-datasets-plots: scientific/time-series processors.

Run the currently exposed direct helper for audio waveform plotting:

sciveo media-run --processor audio-plot --width 1280 --height 320 --rate 16000 --input-path ./audio.aac --output-path ./waveform.png

Run a small local pipeline from Python:

from sciveo.media.pipelines.pipeline import MediaPipeline

job = {
    "id": "local-image-job",
    "input": [
        {
            "guid": "image-1",
            "content_type": "image",
            "local_path": "./input.jpg",
            "key": "input.jpg",
            "processors": "",
        }
    ],
    "configuration": {
        "force_processing": True,
        "processors": {
            "image-resize": {
                "configuration": {"heights": [720, 480]}
            }
        },
        "postprocessors": {},
    },
}

result = MediaPipeline(job).run()
print(result["output"])

Processor graphs can be chained with next blocks. For example, extract representative video frames and resize each generated frame:

job["configuration"]["processors"] = {
    "video-frames-extract": {
        "configuration": {"frames-count": 8},
        "next": {
            "image-resize": {
                "configuration": {"heights": [720, 360]}
            }
        },
    }
}

Use pip install "sciveo[media]" for core media processing. Use pip install "sciveo[media-ml]" when the pipeline needs ML processors such as image captioning, segmentation, embeddings, diffusion, or time-series training.

Experiments Management Client

sciveo is a Python library that serves as a client for managing machine learning and scientific experiments on the sciveo.com platform. This library provides a convenient interface to interact with the sciveo.com API, enabling users to organize, track, and analyze their experiments efficiently. There are few configuration params samplers, which allows easy parameter tuning. The "auto" sampler perhaps is the easiest to use, but also "random" and "grid" ones are available.

Features

• Experiment Tracking: Easily log and track your machine learning experiments.
• Experiment Comparison: Compare different experiments and their results.
• Data Visualization: Visualize experiment metrics and results.
• Integration with sciveo.com: Seamlessly connect and synchronize with the sciveo.com platform.
• Monitoring machines (from HPC to jetson nano): Visualisation and metrics collection in sciveo platform.

Installation

Main Sciveo package:

pip install sciveo

Monitoring:

pip install "sciveo[mon]"

Agent hosted providers:

pip install "sciveo[agent]"

Agent owned local Hugging Face/GGUF runtime:

pip install "sciveo[agent-local]"

Network and Modbus tools:

pip install "sciveo[net]"

The net extra includes netifaces, scapy, and pymodbus.

Server/web helpers:

pip install "sciveo[server]"
pip install "sciveo[web]"

Full operations install without media/ML support:

pip install "sciveo[all]"

Media and ML processors:

pip install "sciveo[media]"
pip install "sciveo[media-ml]"

Example usage

There are few public examples in sciveo.com.

The library has local and remote mode. The local one is ready to use, but for the remote one you will need a sciveo.com account.

When have sciveo account:

sciveo init

Where ~/.sciveo/ path and ~/.sciveo/default file will be created. Just need to change the secret_access_key value.

or

export SCIVEO_SECRET_ACCESS_KEY='my_sciveo_user_auth_token'

or create a file like ~/.sciveo/some_file_name where put: secret_access_key=my_sciveo_user_auth_token

Sciveo Monitoring cli

sciveo monitor --period 120

Monitoring started along with other python code.

import sciveo

# Non blocking monitoring, so continue the code execution after it.
sciveo.monitor(period=120, block=False)

#rest of your python code here

Experimental Projects management

# These are experiment specific imports for the demo purposes only.
from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score, mean_absolute_percentage_error
from ml.time_series import TimeSeriesTrainer, TimeSeriesPredictor

# This is the only needed import when using sciveo along with the experiment-related imports
import sciveo

def train():
    # sciveo.open() method returns current Experiment object, with its configuration sample
    with sciveo.open() as E:
        # Just an example time series trainer (using TF/Keras simple 1D conv model).
        trainer = TimeSeriesTrainer(
            ds,
            E.config.input_window, # Experiment.config is the configuration, so input_window as hyper parameter.
            E.config.input_window,
            E.config.steps # steps parameter.
        )
        trainer.create()

        history = trainer.train(E.config.max_epochs, E.config.patience)
        trainer_eval = trainer.evaluate()

        model_name, model_path = trainer.save("model-name-path.timeseries")

        # Experiment logging for everything which seems interesting for the experiment.
        E.log({"model_path": model_path})

        E.log({"train history": history.history})
        E.log({"trainer_eval": trainer_eval})

        # Plot data, various input types (dict, list etc.).
        # Showing data as charts (single and combined) and tables.
        # There is a more advanced render option which could be used for tables, charts definition.
        E.plot("train history", history.history)

        predictor = TimeSeriesPredictor(model_path)
        Y_predicted, Y_valid, x = predict_chunk(ds.dataset["test"], predictor)

        # Plot predicted and labeled
        for i, col_name in enumerate(ds.columns):
            y_predicted = Y_predicted[0,:,i].numpy().tolist()
            y_valid     = Y_valid[0,:,i].numpy().tolist()

            # Could provide x column for the plot, there are multiple options like timestamps etc.
            # The "X" is reserved for x column name, if not present default range [1, N]
            E.plot(col_name, { "predicted": y_predicted, "label": y_valid, "X": x })

        mse = mean_squared_error(y_valid, predictions)
        mae = mean_absolute_error(y_valid, predictions)
        rmse = np.sqrt(mean_squared_error(y_valid, predictions))
        r2 = r2_score(y_valid, predictions)
        mape = mean_absolute_percentage_error(y_valid, predictions)

        E.log({"R2": r2})
        E.log(f"RMSE: {rmse}")
        E.log("MAPE", mape)
        E.log("R2", r2, "RMSE", rmse, "MAPE", mape)

        # There is a sorting option for the Project's experiments
        # By default it is "score", so there is a method Experiment.score() which could be used for experiments evaluation.
        E.score(100 - mape)
        # There is explicit Experiment "eval" section where all available evaluation metrics could be logged.
        E.eval("R2", r2)
        E.eval("RMSE", rmse)
        E.eval("MAPE", mape)


# Configuration of the Project's experiments run.
configuration = {
    "input_window": {
        "values": [10, 20, 30, 40, 50, 100, 200] # "values" option provides selection from a list of values.
    },
    "steps": {
        # "min"/"max" is a range of values where sampling will get next value.
        # It is int/float sensitive, so if range is [1, 100], the sampled value will be integer.
        # If range is [1.0, 100.0], sampling float values.
        "min": 1, "max": 100
    },
    "max_epochs": (10, 50), # Same range of values but using a tuple (min, max).
    "patience": (1, 3),
    "idx": {
        "seq": 1 # Sequence sampling, so just increase it on every run, could be used as experiment index.
    }
}

# Dataset info
sciveo.dataset({"name": "EURUSD60.csv", "split": ds.ratios}) # any dict with params.

# sciveo.start() method starts the Project's experiments run.
sciveo.start(
    project="TimeSeriesTrainer param tune", # Project name, could be existing or a new one.
    configuration=configuration, # The hyper param configuration
    function=train, # Function which will be executed on every loop.
    remote=True, # There are 2 modes: local and remote. For remote option there is a need of sciveo.com authentication.
    count=20, # Number of experiments which will be run.
    sampler="random" # Configuration sampling method, options currently are "random" (by default), "grid" and "auto".
)

Who do I talk to?

• Pavlin Georgiev
• [e-mail]