sciveo 0.2.64

No project description provided

SCIVEO

Sciveo is a Python toolkit and CLI for ML/AI workflows, engineering automation, experiment tracking, machine monitoring, local operations, network/industrial IO, media processing, AI agents, local storage, and production ML inference.

It is meant for practical systems where models, data, devices, services, and operators meet: train or evaluate ML models, collect measurements, process media, monitor machines, communicate with edge or plant equipment, run agents, and keep the operational tooling close to the Python code that uses it.

The package is intentionally broad. The CLI is the easiest entry point for operations and lab work, while the Python modules expose the same building blocks for notebooks, services, experiments, and pipelines.

ML/AI and Engineering Focus

Sciveo is built around the common shape of applied ML, AI, and engineering work:

• define and run parameterized experiments;
• sample parameter spaces and record configurations;
• attach datasets, metrics, scores, plots, media, and generated artifacts;
• monitor machines that produce data or run services;
• communicate with lab, plant, edge, and networked devices;
• capture and process image/video/audio data;
• run local or remote ML inference close to the data;
• coordinate AI agents and local tools;
• synchronize results to an API-backed project when remote tracking is needed.

The operations modules are part of the same picture. Monitoring, network IO, local storage, media workers, and admin tooling are there because real ML and engineering deployments often include cameras, sensors, edge boxes, industrial controllers, GPU machines, local disks, APIs, and long-running background jobs.

Main Capabilities

• Monitoring and watchdogs: host metrics, GPU/server telemetry where available, plant/Modbus monitoring, local watchdog actions, and service installation helpers.
• Experiments and project helpers: project runs, parameter sampling, datasets, scores, plots, metadata, and optional remote synchronization.
• Network and industrial IO: network discovery, SSH inventory/execution, Modbus TCP/RTU reads and writes, SNMP, MQTT, HTTP helpers, and protocol emulators.
• Admin and diagnostics: first-boot edge administration, Wi-Fi access point provisioning, service checks, doctor reports, and fleet inventory.
• Media capture and processing: RTSP/NVR capture, screen/camera capture, local media processing, ML media processors, and queue/API-backed media workers.
• AI agent console: interactive and one-shot agent workflows with hosted providers, local tools, image inputs, and optional local Hugging Face/GGUF runtimes.
• Encrypted chat rooms: shared-token chat rooms for lightweight operator and agent coordination.
• Local S3-compatible storage: a boto3-compatible object store backed by mounted local paths.
• Certification readiness tests: local, framework-neutral self-assessment, evidence-window scoring, and remediation suggestions, starting with an unofficial SOC 2-style security profile.
• SCIVEYOLO inference: Sciveo-owned SCIVEYOLO inference artifacts and runtime for production object detection without requiring Ultralytics at runtime.
• Supporting API, DB, web, content, and tools modules: shared building blocks used by experiments, services, and pipeline jobs.

Typical Workflows

Sciveo is useful when a project needs more than one isolated script. The common pattern is a Python workflow that touches models, measurements, local services, devices, and result tracking.

ML/AI Workflow

Use Sciveo to keep model work reproducible and close to the runtime environment:

import sciveo

def train_one_run():
    with sciveo.open() as experiment:
        learning_rate = experiment.config.learning_rate
        batch_size = experiment.config.batch_size

        metrics = {
            "loss": 0.18,
            "accuracy": 0.94,
        }

        experiment.log({"learning_rate": learning_rate, "batch_size": batch_size})
        experiment.eval("loss", metrics["loss"])
        experiment.eval("accuracy", metrics["accuracy"])
        experiment.score(metrics["accuracy"])

sciveo.start(
    project="vision-model-eval",
    function=train_one_run,
    configuration={
        "learning_rate": {"values": [0.001, 0.0005]},
        "batch_size": {"values": [8, 16]},
    },
    remote=False,
    count=4,
)

Engineering and Edge Workflow

Use the CLI to inspect and operate machines that produce data or run inference:

sciveo doctor --render json
sciveo scan --net 192.168.10.0/24 --health
sciveo fleet --net 192.168.10.0/24 --users operator,admin -i ~/.ssh/id_ed25519 --render json
sciveo monitor --period 60
sciveo watchdog --src memory --threshold 90 --period 10 --execute "systemctl restart app"

Media and Inference Workflow

Use media commands for local processing, and .sciveyolo artifacts for production object detection without requiring Ultralytics at runtime:

sciveo rtsp --url rtsp://camera/stream --output-path ./clip.mp4
sciveo media-run --input-path ./clip.mp4 --output-path ./media-out --render text
sciveo media-run --mode ml --processor image-object-detection --input-path ./frame.jpg --output-path ./detections

from sciveo.ml.images.sciveyolo import SCIVEYOLO

model = SCIVEYOLO("detector.sciveyolo")
results = model.predict("frame.jpg")

Device and Protocol Workflow

Use network and industrial IO helpers directly from the CLI during bring-up, testing, and diagnostics:

sciveo read --proto modbus --transport tcp --host 192.168.0.10 --port 502 --id 1 --address 30001 --kind input --type RAW --count 2
sciveo write --proto mqtt --host broker.local --topic plant/cmd --payload '{"limit":80}' --qos 1 --retain
sciveo emulate --server http --host 0.0.0.0 --port 8080 --data-json '{"status":{"ok":true}}'

Installation

Base install:

pip install sciveo

Useful extras:

pip install "sciveo[mon]"          # host monitoring helpers
pip install "sciveo[net]"          # network, Modbus, SNMP, MQTT helpers
pip install "sciveo[agent]"        # hosted AI agent providers
pip install "sciveo[agent-local]"  # local HF/GGUF runtime support
pip install "sciveo[media]"        # media capture/processing
pip install "sciveo[media-ml]"     # ML model/runtime dependencies
pip install "sciveo[server]"       # API/server helpers
pip install "sciveo[web]"          # Django/web helpers
pip install "sciveo[all]"          # core operations extras

media-ml intentionally does not install Ultralytics. For image/video ML workflows, install both media extras so OpenCV and model runtimes are present:

pip install "sciveo[media,media-ml]"

Sciveo SCIVEYOLO runtime loads .sciveyolo artifacts with PyTorch or ONNX Runtime depending on the artifact engine.

CLI Overview

Show CLI help and version:

sciveo --help
sciveo --version
sciveo help --json
sciveo help monitor
sciveo help monitor --json

Current top-level CLI commands:

init
monitor
watchdog
scan
ssh
read
write
emulate
admin
doctor
fleet
extensions
certification
nvr
rtsp
capture
media-server
media-run
agent
chat
storage
predictors-server

The sciveo help --json manifest is the machine-readable source for command summaries, usage forms, options, notes, and examples.

Certification Readiness Tests

See CERTIFICATION.md for the platform architecture, verification strategy, implemented scope, and phased roadmap.

Generate an editable SOC 2-style test input, complete its answers and evidence metadata, then evaluate it locally:

sciveo certification --action template --framework soc2 --output-path soc2-test.json
sciveo certification --action example --scenario software-company --output-path software-company.json
sciveo certification --action evaluate --framework soc2 --input-path soc2-test.json
sciveo certification --action evaluate --framework soc2 --input-path soc2-test.json --render json --output-path soc2-report.json

The initial profile is a Sciveo-authored general security checklist. It scores the supplied status and evidence metadata, highlights critical gaps, checks a configurable observation target (90 days by default), and returns concrete suggestions. Reports also contain structured control-test results, a ranked remediation plan, evidence-reference classifications, domain scores, and per-system gap summaries. The summary is an explanatory roll-up; per-check system reference coverage can cap scores. A reference does not mean the control is supported, and Sciveo does not open or verify referenced evidence.

This feature is an unofficial readiness self-assessment. It is not a SOC 2 examination, audit, certification, CPA opinion, or guarantee of an audit result; it does not reproduce the AICPA Trust Services Criteria. An independent licensed CPA firm determines the actual examination scope, observation period, testing, exceptions, and report. Use sciveo certification --help for the CLI contract.

The generated template is the versioned machine-readable input contract. Answers use implemented, partial, not_implemented, not_assessed, or not_applicable; the last status requires a text rationale, and every proposed exclusion remains scored until an independent review process exists. Evidence is a list of non-empty text references or objects containing name, reference, path, url, or id, plus optional source, system_ids, and an ISO observed_at date. source alone is not a concrete reference. Dated evidence outside the configured period is classified and ignored for that period. Answers may also set applicable_system_ids, gap_system_ids, applicability_rationale, remediation_action, owner, and target_date. Omitting a declared system requires a rationale and remains an explicit scope review item; critical scope exclusions prevent the strong readiness band. Targets must be integer values from 1–366 days. An explicitly supplied --observation-days overrides the input target; otherwise the input value wins, with 90 days as the fallback.

Complete Hybrid Software Company Example

Generate the bundled fictional assessment and evaluate it without contacting any external service:

sciveo certification --action example \
  --scenario software-company \
  --output-path software-company.json

sciveo certification --action evaluate \
  --input-path software-company.json

sciveo certification --action evaluate \
  --input-path software-company.json \
  --render json \
  --output-path software-company-report.json

The example fixes a 90-day window from 2026-01-01 through 2026-03-31 so its inputs and score remain reproducible. It models the following system boundary:

System	Synthetic population/sample	Readiness evidence and deliberate gaps
Git	18 repositories / 4 sampled	Review rules, approvals, CI results, access reviews
AWS EC2	24 instances / 6 sampled	Infrastructure changes, vulnerability scans, logging; no account IDs or live calls
AWS S3	8 logical locations / 4 sampled	Storage configuration and restoration references; no bucket names or credentials
Self-hosted servers	12 servers / 4 sampled	Patch/change/access records with deliberate local-account gaps
Edge IoT	240 gateways / 12 sampled	Inventory and monitoring references with firmware/access lifecycle gaps
Edge NVR	120 machines / 10 sampled	Backup records with deliberate logging and restoration-test gaps

The synthetic answers produce a stable 60.0% developing readiness result, seven critical gaps, and ten ranked remediation tasks: Git and AWS practices are comparatively mature, while shared local accounts, edge firmware governance, centralized edge/NVR logging, NVR restoration tests, and device-vendor lifecycle reviews create remediation work. This is an example of how the scoring behaves, not a conclusion about any real company.

The example builder is frozen local data. Shared Sciveo imports may load global configuration, but certification does not authenticate to services or use, transmit, or copy credential values. It never invokes Git, connects to AWS, opens SSH sessions, contacts self-hosted servers, or talks to edge devices, cameras, or NVRs. Replace the synthetic inventory, answers, evidence IDs, owners, dates, and gaps with your own metadata before using the evaluator for internal readiness planning.

Configuration

Initialize local configuration:

sciveo init

The default local configuration lives under ~/.sciveo/. API credentials can also be supplied through environment variables where supported, for example:

export SCIVEO_SECRET_ACCESS_KEY="..."

Monitoring

Start host monitoring:

sciveo monitor --period 60

Monitor the root filesystem plus additional mounted paths or block devices. --paths accepts mount directories and always includes /. --disks accepts filesystem-bearing block devices or partitions and resolves each one to its current mountpoint:

sciveo monitor --period 60 \
  --paths '["/mnt/data-1"]' \
  --disks '["/dev/sdb1","/dev/sdc1"]'

Root capacity remains under DISK; requested mount paths are reported under PATHS, and requested block devices are reported under DISKS with their resolved device, mountpoint, filesystem, capacity, and mount state. Sciveo does not implicitly expand a whole disk such as /dev/sdb to /dev/sdb1; name the filesystem-bearing partition explicitly. Stable /dev/disk/by-uuid/... aliases are preferred when device names may change across boots.

Install monitoring as a service:

sciveo monitor --period 60 \
  --paths '["/mnt/data-1"]' \
  --disks '["/dev/sdb1"]' \
  --install

Run --install as the account that should run the monitor. Sciveo asks for sudo only while copying the systemd units and running systemctl; the installed service keeps the invoking account as its runtime user.

Every --install rewrites both sciveo-monitor.service and sciveo-monitor.timer, reloads systemd, and restarts the service so updated monitor arguments take effect immediately. The timer starts the continuous service once after each boot; the service itself continues sampling at --period intervals.

Inspect the installed units and follow monitor logs:

systemctl status sciveo-monitor.timer sciveo-monitor.service
systemctl list-timers --all sciveo-monitor.timer
journalctl -u sciveo-monitor.service -f

Write samples to a local path:

sciveo monitor --period 120 --output-path ./monitor.json

Start non-blocking monitoring from Python:

import sciveo

sciveo.monitor(period=120, block=False)

Plant/industrial monitoring uses the plant source:

sciveo monitor --src plant --host 192.168.1.50 --port 502 --period 60 --serial plant-1

Watchdogs

Watchdogs run local checks and either execute a configured command or use the built-in internet repair path when a condition remains unhealthy.

sciveo watchdog --src memory --threshold 90 --period 10 --execute "systemctl restart app"
sciveo watchdog --src disk --input-path /data --threshold 80 --period 600 --execute "find /data/tmp -type f -mtime +1 -delete"
sciveo watchdog --src network --targets '["1.1.1.1:443","8.8.8.8:53"]' --threshold 3 --period 30 --execute "echo network outage"

The network/internet watchdog performs concurrent deadline-bounded TCP checks, records resolution/bind/connect results, and applies at-least-one, all, majority, or count:N quorum. With --execute, it runs the configured shell command after --threshold consecutive failures and repeats the command on every failed period until a healthy check resets the counter. scan --health adds read-only gateway/DNS diagnostics, while monitor HTTP probes record endpoint health without repairs.

Use built-in internet recovery without running the whole watchdog as root:

sciveo watchdog --net --repair --period 30 --threshold 3

On Linux systemd machines, install the same configuration as a supervised boot service. Run this command from the non-root account that should run the watchdog; Sciveo asks for sudo only while installing root-owned files and using systemctl:

sciveo watchdog --net --repair --period 30 --threshold 3 --install
sciveo watchdog --net --repair --preflight
systemctl status sciveo-network-watchdog.service
journalctl -u sciveo-network-watchdog.service -f

--install and --reinstall preserve the network repair targets, quorum, timing, platform, interface, and log-level options in the unit's ExecStart. The service runs as the invoking non-root user with Restart=always; it does not run the watchdog process as root. Every service start first runs a read-only preflight that validates the unattended helper, exact service-unit allowlist, detected platform, selected interface, network manager, and that manager's required core helper actions. Remove all managed watchdog files and persistent incident state with:

sciveo watchdog --uninstall

Installation places a self-contained root-owned helper at /usr/local/libexec/sciveo-network-repair and grants the service user passwordless sudo access only to that exact executable. The helper accepts a fixed action enum, validates physical interface names and canonical NetworkManager UUIDs, uses fixed absolute system-program paths, and rejects arbitrary commands. Privileged actions have a fixed 30-second execution deadline. The optional service allowlist is accepted only as a regular, root-owned file that is neither group- nor world-writable, and its exact units are checked by preflight. The helper's DNS configuration repair refuses to overwrite a regular /etc/resolv.conf or a working symlink. Optional VPN/API service restarts are limited to canonical .service units in that allowlist.

Bare --repair also implies the network source, but spelling the mode as --net --repair is preferred because it makes the watchdog purpose explicit.

Repair mode defaults to 1.1.1.1:443, softel.bg:443, and sciveo.com:443 with majority policy. Additional VPN or API endpoints supplied through --targets are added to those defaults and deduplicated by host and port. Use --policy all when every configured endpoint must participate in the health decision.

Before changing an interface, repair mode snapshots the owning manager, link/carrier, usable addresses, default route, gateway-neighbor state, and DNS, then separately probes numeric underlay anchors (1.0.0.1:443, 8.8.8.8:53, and [2606:4700:4700::1111]:443) through the selected physical interface/source. The at-least-one default keeps IPv4-only and IPv6-only links valid while providing dual-stack evidence when both are available. It repairs the lowest failed layer. If link, address, route, and gateway are healthy while the public underlay is unavailable, the incident is classified as upstream and local networking is left intact.

An isolated VPN/API endpoint is never mapped to an arbitrary process. To restart a known local service after proving that the Internet underlay is healthy, provide an exact target mapping at installation:

sciveo watchdog --net --repair \
  --targets "vpn.example.com:443" --policy all \
  --repair-services '{"vpn.example.com:443":["[e-mail]"]}' \
  --install

Only those installed units can pass the privilege helper. Sciveo watchdog, Sciveo Admin, SSH, hostapd/dnsmasq, and the network-manager/resolver control units are protected and cannot be mapped. A remote API outage with no explicit local mapping is reported and safely skipped.

--platform auto detects Linux or macOS; on Linux it separately detects per-interface ownership by NetworkManager, systemd-networkd, or dhcpcd. Explicit overrides include linux, ubuntu, debian, raspbian, raspberry-pi, rpi, macos, mac, and darwin, so Intel and Raspberry Pi Ubuntu machines use the same manager-aware Linux repair backend. At startup, repair mode logs the requested and detected platform at INFO level. On a multi-homed edge device, --interface eth0 is the explicit safe physical-uplink override; otherwise Sciveo selects the default-route physical interface and avoids tunnel devices.

The network --threshold, --recovery-checks, and --max-disruptive-repairs must be positive integers. --period, --timeout, and --repair-cooldown must be finite positive values, and the initial repair cooldown cannot exceed the 900-second backoff cap. --underlay-targets accepts numeric IP targets only; --underlay-policy controls their quorum.

Stage or condition	Check	Repair action	Verification
Outage confirmation	Concurrent target quorum fails for the configured consecutive threshold	No mutation before the threshold	Re-evaluate all targets
Manager/link	Manager inactive, device down, no carrier, rfkill, or disconnected exact saved profile	Restart a proven inactive manager, bring up the selected interface, unblock/re-enable Wi-Fi, or activate only one unambiguous bound profile	Recheck the full target quorum
Address	No usable global address or a lost DHCP lease	Rebind/renew through the detected owning NetworkManager, networkd, or dhcpcd backend	Compare address state and recheck targets
Route/gateway	IPv4 and IPv6 default routes are missing, or every known gateway neighbor explicitly fails	Reapply the exact profile, reconfigure the owning backend, then renew its lease	Re-snapshot dual-stack routes/gateways and recheck targets
DNS cache/server	Named resolution fails while numeric underlay works	Flush cache and reset learned resolver-server features	Recheck named targets
DNS resolver/config	DNS remains broken or the resolved symlink is missing/broken	Restart only the detected resolver; safely restore only a missing/broken resolved symlink	Recheck resolution and configured servers
DHCP-provided DNS	Resolver repair did not recover	Renew DNS/lease through the proven interface owner	Recheck DNS and target quorum
Upstream provider	Local link, address, route, and gateway are healthy but numeric Internet probes fail	No local mutation; wait with bounded backoff	Re-probe underlay
VPN/API endpoint	Underlay is healthy and one configured endpoint fails	Restart only an exact target-mapped, root-allowlisted .service; otherwise safely skip	Recheck all configured targets
Disruptive guard	Manager/service restart is considered	Block for Admin AP/hostapd, active VPN, or SSH; enforce per-incident budget	Open durable circuit after the budget
Recovery	Repairs or external recovery restore quorum	Require stable healthy checks before clearing state	Reset cooldown/circuit after --recovery-checks successes

Built-in repair uses structured commands, never flushes routes, never reboots, and never guesses which arbitrary VPN/API services to restart. The managed service sends privileged actions only through its installed strict helper; operators do not need to grant direct sudo rights to nmcli, networkctl, dhcpcd, systemctl, or a shell. Its incident layer, cooldown, and disruptive circuit survive process restarts; stable healthy checks clear them. Watchdog repairs and Sciveo Admin network changes share one mutation lock. Admin opens a bounded maintenance lease before its first network mutation and keeps repairs inhibited until confirmation or lease expiry, including after a partially failed Admin apply.

This recovery path deliberately does not guess new static addressing, unknown Wi-Fi/VPN credentials, router configuration, or arbitrary services. Physical NIC/cable/router failures and provider outages require a redundant uplink or out-of-band management; a local watchdog also cannot prove inbound Internet reachability without an external heartbeat.

Diagnostics and Fleet

Local diagnostic report:

sciveo doctor
sciveo doctor --render json
sciveo doctor --logs --output-path /tmp/sciveo-doctor.json

Fleet inventory over SSH:

sciveo fleet --host [e-mail] -i ~/.ssh/id_ed25519
sciveo fleet --net 192.168.10.0/24 --users operator,admin -i ~/.ssh/id_ed25519 --render json

Network and Industrial IO

Network scans:

sciveo scan
sciveo scan --net 192.168.0.0/24 --port 22 --timeout 0.5
sciveo scan --health
sciveo scan --health --l2
sciveo scan --host 192.168.0.10 --health --ports '[22,80,443,502,554,161]'

SSH scan and command execution:

sciveo ssh --net 192.168.0.0/24 --users operator,admin -i ~/.ssh/id_ed25519 --list-shell '["hostname","uptime"]'

Modbus reads and writes:

sciveo read --proto modbus --transport tcp --host 192.168.0.10 --port 502 --id 1 --address 30001 --kind input --type RAW --count 2
sciveo write --proto modbus --transport tcp --host 192.168.0.10 --id 1 --reg '[40010,"U16",1,1]' --value 123
sciveo read --proto modbus --action scan --net 192.168.0.0/24 --render text

Serial Modbus:

sciveo read --proto modbus --transport serial --serial-port /dev/ttyUSB0 --baudrate 9600 --bytesize 8 --parity N --stopbits 1 --id 1 --reg '[5039,"U16",0.1,1]'

SNMP, MQTT, and HTTP helpers:

sciveo read --proto snmp --host 192.168.0.1 --oid 1.3.6.1.2.1.1.1.0
sciveo read --proto snmp --host 192.168.0.1 --action walk --oid 1.3.6.1.2.1.1
sciveo write --proto mqtt --host broker.local --topic plant/cmd --payload '{"limit":80}' --qos 1 --retain
sciveo read --proto http --url http://192.168.0.10/status
sciveo write --proto http --url http://192.168.0.10/api/control --value '{"enabled":true}'

Protocol emulators:

sciveo emulate --server modbus --profile custom --host 0.0.0.0 --port 1502 --data-json '{"device_id":7,"holding":{"40010":123}}'
sciveo emulate --server snmp --host 0.0.0.0 --port 1161 --data-json '{"oids":{"1.3.6.1.2.1.1.1.0":"lab-agent"}}'
sciveo emulate --server mqtt --host 0.0.0.0 --port 1883 --data-json '{"retained":{"plant/power":1234}}'
sciveo emulate --server http --host 0.0.0.0 --port 8080 --data-json '{"status":{"ok":true}}'

Admin UI

The admin command is for first-boot and field administration of an edge machine:

sciveo admin
sciveo admin --web --wifi-ap --net 10.137.19.0/24 --ap-ssid sciveo-setup --ap-password CHANGE_ME --admin-auth none
sudo sciveo admin --install --web --wifi-ap --net 10.137.19.0/24 --ap-ssid sciveo-setup --ap-password CHANGE_ME --admin-auth none

The admin UI covers dashboard diagnostics, Ethernet and Wi-Fi setup, service state, pending configuration, and installed service management. Pending configuration is stored under ~/.sciveo/admin/.

VS Code Extension

The Python package can install the bundled VS Code extension asset:

sciveo extensions --install --vscode
sciveo extensions --reinstall --vscode
sciveo extensions --uninstall --vscode

If the editor CLI is not on PATH, set:

export SCIVEO_VSCODE_CLI="/path/to/code"

Extension config is stored under ~/.sciveo/extensions/vscode/.

Encrypted Chat Rooms

Start a room:

sciveo chat --serve ops-room --host 0.0.0.0 --port 8090 --max-clients 5

Join from another terminal:

sciveo chat --client 'sciveo-chat-v1....' --url ws://HOST:8090/ws/chat --name operator-a

Messages use encrypted AES-GCM envelopes, while connection setup uses a shared-token HMAC proof. Display names are decorative; anyone with the shared token is trusted as a room participant.

Persist and reload encrypted room history:

sciveo chat --serve ops-room --output-path ./chat-history.jsonl
sciveo chat --serve ops-room --input-path ./chat-history.jsonl --output-path ./chat-history.jsonl

Suppress decrypted message bodies in server logs:

sciveo chat --serve ops-room --silent

TLS:

sciveo chat --serve ops-room --tls-cert ./chat.crt --tls-key ./chat.key
sciveo chat --client 'sciveo-chat-v1....' --url wss://HOST:8090/ws/chat --name operator-a --tls-no-verify

Agent Console

Interactive agent console:

sciveo agent --provider auto
sciveo agent --profile coder --provider auto

One-shot prompt:

sciveo agent --provider auto --prompt "List the repo root and summarize the important files"

Profiles can be loaded with --profile NAME or --config PATH. Predefined profiles include coding, review, testing, research, and operations-oriented specializations.

Local runtime examples:

sciveo agent --action pull --model org/model-name --alias local-agent-model
sciveo agent --action run --model local-agent-model --host 127.0.0.1 --port 8910 --device mps --context 8192
sciveo agent --provider hf --prompt "Say hello in one sentence"

Agent orchestration uses the chat transport:

sciveo agent --action orchestrate --serve design-room --prompt "Review this module" --agents researcher,coder,reviewer,tester
sciveo agent --action orchestrate --serve design-room --prompt "Plan the next pass" --agent-write-policy discussion-only

Media Capture and Processing

Capture and stream helpers:

sciveo nvr --input-path cams.json
sciveo rtsp --url rtsp://camera/stream --output-path ./clip.mp4
sciveo capture --output-path ./screen.mp4 --fps 10

Run the queue/API-backed media worker:

sciveo media-server
sciveo media-run --mode worker

Run local/offline media processing:

sciveo media-run --input-path ./media --output-path ./media-out --render text
sciveo media-run --processor image-resize --input-path ./input.jpg --output-path ./image-out
sciveo media-run --config pipeline.json --input-path ./input.mp4 --output-path ./video-out

Run local ML media processing:

sciveo media-run --mode ml --processor image-to-text --input-path ./input.jpg --output-path ./ml-out

Local mode does not require API, queue, S3, or cloud credentials. Worker mode is for API/queue/storage-backed jobs.

SCIVEYOLO Object Detection

Sciveo provides a SCIVEYOLO inference runtime in sciveo.ml.images.sciveyolo.

Goals:

• load .sciveyolo artifacts without Ultralytics installed;
• use a Sciveo-owned PyTorch SCIVEYOLO graph for engine="pt";
• preserve a familiar SCIVEYOLO(...).predict(...) style API;
• support SCIVEYOLO model sizes n, s, m, l, and x through one generic scale-parameterized network;
• keep build-time conversion separate from production runtime.

Basic use:

from sciveo.ml.images.sciveyolo import SCIVEYOLO

model = SCIVEYOLO("model.sciveyolo")
results = model.predict("image.jpg", conf=0.25, iou=0.7)

Build a Sciveo artifact from a PyTorch checkpoint:

from sciveo.ml.images.sciveyolo import SCIVEYOLO

SCIVEYOLO.build("detector.pt", "detector.sciveyolo")

The default build engine is pt, which stores a state dict and metadata inside the .sciveyolo file. The artifact metadata includes engine, runtime, architecture, scale, model_size, variant, class names, image size, and source hash.

Auto-build sidecar behavior:

model = SCIVEYOLO("detector.pt")

When given a .pt path, Sciveo first looks for a same-name .sciveyolo sidecar next to it. If present, that artifact is loaded. If absent, Sciveo attempts to build it with engine="pt" and then loads the generated artifact.

Build an ONNX artifact only when explicitly wanted:

SCIVEYOLO.build("detector.pt", "detector-onnx.sciveyolo", engine="onnx")

Build a TensorRT-backed artifact when running on NVIDIA CUDA hosts:

SCIVEYOLO.build("detector.pt", "detector-trt.sciveyolo", engine="trt")

Build-time conversion may use a separate environment with export tooling. The production runtime only needs the dependencies required by the artifact engine, for example PyTorch for engine="pt", ONNX Runtime for engine="onnx", or ONNX Runtime GPU plus TensorRT for engine="trt".

Install the runtime extra that matches the deployment target:

pip install "sciveo[sciveyolo]"
pip install "sciveo[sciveyolo-gpu]"
pip install "sciveo[sciveyolo-trt]"

Use a fresh pyenv for the GPU/TRT extras when possible, because onnxruntime and onnxruntime-gpu provide the same onnxruntime Python module. The TRT extra is intended for CUDA/NVIDIA machines with compatible drivers and TensorRT libraries.

Runtime options:

model = SCIVEYOLO("model.sciveyolo", device="cpu", nms_method="numpy")
model = SCIVEYOLO("model.sciveyolo", fuse=True)
model = SCIVEYOLO("model.sciveyolo", channels_last=True)
model = SCIVEYOLO("model.sciveyolo", device="cuda:0", engine="auto", auto_engine_source="sample-video.mp4")

Native fine-tuning uses UT-style object-detection dataset YAML files:

model = SCIVEYOLO("coco-detector.sciveyolo")
result = model.train(
    data="/datasets/coco128/data.yaml",
    project="/models/object-detection",
    name="coco128-sciveyolo-m",
    epochs=8,
    batch=24,
    augment=False,
    optimizer="SGD",
    lr0=0.005,
    lrf=0.1,
    amp=False,
)

The trainer writes weights/last.pt, weights/best.pt, weights/last.sciveyolo, weights/best.sciveyolo, args.yaml, and results.csv under project/name.

Evaluate an artifact on a dataset split:

model = SCIVEYOLO("coco-detector.sciveyolo")
val_metrics = model.evaluate(data="/datasets/coco128/data.yaml", split="val")
test_metrics = model.test(data="/datasets/coco128/data.yaml")

evaluate() reports both native validation loss for PyTorch artifacts and Sciveo object-detection AP/FP metrics. test() is the same evaluator with split="test" by default.

engine="auto" looks for compatible .sciveyolo artifacts near the requested model, checks what the selected device/runtime can actually load, runs a small probe when multiple candidates are available, and then sets model.engine to the selected engine.

Pass auto_engine_source as an image, video, frame array, or list of sources to make the probe representative of the deployment workload. Without a source, Sciveo logs that it is falling back to a synthetic frame.

For CPU deployment, ONNX Runtime can be faster than the PyTorch engine on some machines. For GPU deployment, engine="pt" is the most portable path, while ONNX/TensorRT performance depends on the installed runtime and GPU generation.

Predictors API Server

Start the predictors API service:

sciveo predictors-server --port 8080

This command starts the configured Sciveo API predictor server and keeps it running as a long-lived process.

Local S3-Compatible Storage

Start a local S3-compatible service:

sciveo storage --s3 --paths '["/mnt/d1","/mnt/d2"]' --port 9000 --threads 32 --health-check-interval 10800

A JSON config can also be supplied:

{
  "paths": [
    {"path": "/mnt/d1", "require_mount": true},
    {"path": "/mnt/d2", "require_mount": true}
  ],
  "credentials": {
    "access_key": "sciveo",
    "secret_key_file": "/etc/sciveo/storage-s3.secret"
  },
  "region": "us-east-1",
  "storage_name": "storage",
  "db_backend": "sqlite",
  "db_path": "~/.sciveo/storage/storage-s3.sqlite3",
  "lock_path": "~/.sciveo/storage/locks",
  "threads": 32,
  "workers": 4,
  "http": {
    "server": "gunicorn",
    "workers": 1,
    "timeout": 3600,
    "keepalive": 65
  },
  "path_health_check_interval": 10800,
  "capacity_check_interval": 5,
  "min_free_ratio": 0.05,
  "min_free_bytes": 1073741824,
  "resume_free_ratio": 0.08,
  "target_free_ratio": 0.15,
  "limits": {
    "max_keys": 1000,
    "max_request_bytes": 10737418240,
    "buckets": {"*": {"max_object_size": 10737418240}}
  },
  "retention": {
    "buckets": {"archive": {"minimum_age": "24h"}}
  }
}

threads is the request thread count in each storage worker. workers is the number of supervised storage workers: worker 0 listens on port, worker 1 on port + 1, and so on. parallel remains a backward-compatible alias for threads. When top-level workers is greater than one, keep http.workers at one; http.workers is the older Gunicorn mode where several processes share a single port.

The following starts one master supervising ports 3900 through 3903, with 32 request threads per worker:

sciveo storage --s3 --config /etc/sciveo/storage-s3.json \
  --port 3900 --workers 4 --threads 32 --master

The master role is the default when neither --master nor --slave is given, including when workers is the default value of one. Keeping --master in new service scripts makes the intended process role explicit. The master does not serve S3 requests. It starts the children, restarts an unexpectedly failed child with bounded crash loop protection, and forwards SIGINT/SIGTERM shutdown. Each child is an independent server using the same paths, metadata DB, and root-derived lock directory.

Existing commands without a role option continue to expose the same configured port, now through a master supervising one child by default. Manual direct process management remains available through an explicit slave, which serves only the exact requested port:

sciveo storage --s3 --config /etc/sciveo/storage-s3.json \
  --slave --port 3902 --threads 32

Multi-Node S3 Cluster

Cluster mode is a separate backend under sciveo.storage.cluster; standalone mode remains the default and keeps its existing file layout and behavior. The S3 protocol flag does not change:

sciveo storage --s3 --mode cluster --role controller --config controller.json
sciveo storage --s3 --mode cluster --role node --config node.json
sciveo storage --s3 --mode cluster --role gateway --config gateway.json

--master, --slave, --workers, and --threads still describe local process supervision. --role describes the network responsibility. A gateway exposes the boto3-compatible endpoint, a controller owns cluster metadata and placement, and a node stores immutable replicas on its configured local paths. Object data flows directly between gateways and nodes or between nodes during repair; it never flows through the controller.

Controller configuration uses authoritative PostgreSQL metadata in production:

{
  "s3": {
    "mode": "cluster",
    "cluster": {
      "role": "controller",
      "name": "media-cluster",
      "node_id": "controller-1",
      "token": "CHANGE_TO_A_LONG_RANDOM_INTERNAL_TOKEN",
      "replication": {"factor": 3, "write_quorum": 2, "read_quorum": 1},
      "heartbeat_interval": 10,
      "heartbeat_timeout": 30,
      "repair_interval": 30,
      "tls": {
        "ca_file": "/etc/sciveo/cluster-ca.pem",
        "cert_file": "/etc/sciveo/controller.pem",
        "key_file": "/etc/sciveo/controller.key"
      }
    },
    "host": "10.0.0.10",
    "port": 9443,
    "db": {
      "backend": "postgres",
      "url": "postgresql://sciveo:[e-mail]/sciveo_cluster"
    },
    "http": {"server": "gunicorn", "workers": 1}
  }
}

Every gateway and node receives a redundant controller URL list. A node also needs a stable identity, an internally reachable advertised URL, mounted paths, and a failure-domain label:

{
  "s3": {
    "mode": "cluster",
    "cluster": {
      "role": "node",
      "name": "media-cluster",
      "node_id": "storage-1",
      "controllers": [
        "https://controller-1.internal:9443",
        "https://controller-2.internal:9443",
        "https://controller-3.internal:9443"
      ],
      "advertise_url": "https://storage-1.internal:9444",
      "token": "CHANGE_TO_A_LONG_RANDOM_INTERNAL_TOKEN",
      "failure_domain": {"site": "primary", "rack": "rack-1"},
      "replication": {"factor": 3, "write_quorum": 2},
      "tls": {
        "ca_file": "/etc/sciveo/cluster-ca.pem",
        "cert_file": "/etc/sciveo/storage-1.pem",
        "key_file": "/etc/sciveo/storage-1.key"
      }
    },
    "host": "10.0.0.21",
    "port": 9444,
    "paths": [
      {"path": "/srv/storage/d1", "require_mount": true},
      {"path": "/srv/storage/d2", "require_mount": true}
    ],
    "db_path": "~/.sciveo/storage/storage-1-node.sqlite3",
    "workers": 1,
    "http": {"server": "gunicorn", "workers": 1}
  }
}

Cluster PUT stages the incoming stream once, calculates CRC32 during that same pass, writes immutable replicas concurrently, and commits the object generation only after write quorum. ETags are opaque UUID tokens, never MD5 file hashes. For replication factor 3, every object is stored on three distinct active nodes: one primary copy plus two redundant copies. Nodes are selected with capacity-weighted rendezvous hashing over the bucket, key, and stable node ID, while preferring distinct failure domains. Placement is deterministic rather than random, distributes different keys across the full healthy node pool, and does not reshuffle unrelated objects when membership changes.

Reads try live replicas in turn. Each gateway rotates equivalent read candidates so concurrent clients do not always select the same physical node; retries still fall through to another replica. This balancing is local to each gateway process, not a cluster-global least-in-flight scheduler. Deletes commit a tombstone before best-effort physical cleanup. Controllers detect live under-replicated generations and ask a new target node to pull directly from a healthy source. They also remove live replicas above the configured factor after failed nodes recover. Active controllers coordinate repair through an expiring lease in the shared metadata database; each object refreshes replica liveness and retries another source if a peer becomes unavailable during a long repair pass.

Adding a node does not require gateway changes. Give the machine a new unique cluster.node_id, configure its controller URLs, advertised peer URL, TLS/token, and mounted paths, then start the node role. It self-registers, heartbeats, and immediately participates in future placement and repairs. Healthy existing factor-three objects are not automatically rebalanced merely because capacity joined the cluster.

Replacing a dead machine may reuse its stable node_id after the controller has classified the old incarnation offline. Every node derives a persistent incarnation ID from its local metadata database. A replacement with a new incarnation invalidates the dead machine's controller replica rows, then normal repair restores factor three from surviving copies. A second incarnation trying to claim an identity that is still healthy is rejected, preventing accidental split brain.

Peer URLs require HTTPS unless cluster.allow_insecure=true is explicitly set for isolated loopback development. Protect the shared internal token and use mTLS certificates on every peer endpoint. Non-loopback controllers should use PostgreSQL HA; SQLite cluster metadata is intended only for local development. When Gunicorn terminates peer TLS directly, controller and node roles require a client certificate whenever tls.cert_file, tls.key_file, and tls.ca_file are configured. The same requirement can instead be enforced by an internal reverse proxy.

The initial cluster backend supports bucket lifecycle, PUT/GET/HEAD/DELETE, range reads, prefix and delimiter pagination, bulk delete, tags, server-side copy, SigV4 presigned GET/PUT, quorum replication, node failover, and repair. Legacy Signature V2 remains disabled; set boto3 Config(signature_version="s3v4") when generating presigned URLs for a custom endpoint. Distributed multipart upload, presigned POST, ACLs, versioning APIs, erasure coding, multi-site replication, and automatic orphan inventory cleanup remain future cluster work. Standalone support for those already implemented S3 operations is unchanged.

sciveo storage --s3 --config ./storage.json

--paths are mounted directories, not raw block devices. A path entry may be a string or an object with path, require_mount, optional root_id, and optional expected device. Sciveo persists an identity marker on every root so a detached/replaced mount is not silently treated as the original storage. Object metadata is stored in ~/.sciveo/storage/<storage-name>-s3.sqlite3 by default with the SQLite backend. Use db_path/--db-path for another SQLite file, or db_backend=postgres with db_url/--db-url for PostgreSQL. storage_name/--storage-name selects the default DB filename and observability label; the persistent root identities and metadata DB, not the display name, define the actual storage set.

For heavier production metadata concurrency, use PostgreSQL:

{
  "s3": {
    "paths": ["/mnt/d1", "/mnt/d2", "/mnt/d3"],
    "access_key": "sciveo",
    "secret_key": "CHANGE_ME",
    "storage_name": "video-cache",
    "db": {
      "backend": "postgres",
      "url": "postgresql://sciveo:CHANGE_ME@127.0.0.1:5432/sciveo_storage"
    }
  }
}

The same settings can also be supplied as flat keys (db_backend, db_url) or CLI options (--db-backend postgres --db-url ...). Install sciveo[storage-postgres] and verify that psycopg2 can load the host libpq before selecting PostgreSQL.

Each metadata database has a persistent store UUID and owns one logical set of storage roots. Starting a service with a completely disjoint path/identity set against an occupied DB is rejected before metadata can be changed. Temporarily omitted roots are retained with an inactive index and become visible again when the same root identity is re-added.

The service does not run a metadata sync pass on normal startup. Run explicit reconciliation with --sync when you want files that exist on disk but are missing from the SQL metadata DB to be added, metadata rows for files missing from currently healthy paths to be removed, and pending deletion tombstones to be completed. Metadata for an unavailable root is retained and hidden from list/search/GET until that root recovers; sync never converts a detached disk into permanent object deletion.

sciveo storage --s3 --sync --config ./storage.json
sciveo storage --s3 --sync --dry-run --config ./storage.json
sciveo storage --s3 --sync --since 10m --config ./storage.json

--since accepts a Unix timestamp, an ISO timestamp, or a relative duration such as 10m, 2h, or 1d. It is intended for fast catch-up after a full index build, for example when preparing a PostgreSQL metadata DB while the SQLite-backed service is still live. Keep the target PostgreSQL service in sync mode only during this phase; do not run two metadata backends as active writers. Both configurations must use the same physical paths and lock_path. Stop the SQLite writers, run one final incremental/full sync with zero inconsistencies, then cut traffic over to PostgreSQL. Incremental sync only reconciles files with filesystem mtimes newer than the timestamp and only prunes recently updated metadata rows; run a full --sync when you need a complete consistency audit. Foreign in-flight mutation journals are skipped and the affected keys return a retryable unavailable response until their owning metadata DB recovers them.

New objects use capacity-weighted rendezvous hashing by persistent root UUID across writable paths. Reordering paths or changing a mount point therefore does not perturb equal-health placement. Paths above target_free_ratio have equal placement weight; paths below that target receive progressively less new data. A path is excluded from new allocations when writing an object would violate either the configured min_free_ratio or min_free_bytes reserve. It returns to placement only after crossing resume_free_ratio, avoiding rapid full/not-full state changes.

A full path remains healthy and readable: indexed objects on it continue to be listed and GET/HEAD continue to work. Only new allocations are redirected. A path is unhealthy only when it is inaccessible, unreadable, detached/unmounted, has a changed device while active, or reports a real filesystem I/O failure. Metadata for an inaccessible root remains indexed but is filtered from client results; when the same root identity returns, its objects become visible again. Configure health checks with --health-check-interval; configure the capacity policy through the generic --config JSON file or SCIVEO_STORAGE_* environment variables.

Known object sizes are reserved with filesystem preallocation where supported. This makes concurrent processes compete for real disk capacity before consuming the HTTP body and allows a failed candidate to fall through to the next ranked path safely. Unknown-length streams are bounded while they are consumed: Sciveo aborts the temporary write before commit if a configured request, object, bucket, or remaining-capacity limit is crossed.

Object listing uses the bucket/key index with a prefix key range rather than a table scan. Empty bucket deletion removes the whole bucket tree below each healthy storage path in one filesystem operation. Unavailable paths are left untouched, preserving their data for recovery.

Object mutations are coordinated by bucket/key across threads, processes, and metadata backends on one host. Every backend uses the same root-set-derived striped POSIX lock directory; lock_path or SCIVEO_STORAGE_LOCK_PATH can pin that directory for hardened services. SQLite adds WAL/retry handling, while PostgreSQL uses transactional/advisory schema locks and a fork-aware connection pool. Multi-host writers sharing the same files are not supported; use one storage host behind local Gunicorn workers and an nginx endpoint. Writes, copies, and local moves keep a temporary destination backup until the SQL transaction commits. Failed metadata commits restore the previous bytes. Deletes first create a transactional tombstone, hiding the object immediately; sync or maintenance can finish physical cleanup after a transient filesystem failure. Maintenance reconciles only indexed tombstones and does not walk object files. Schema v5 records a persistent metadata-store UUID in mutation journals so a migration target cannot roll back a transaction owned by the source DB.

Standalone object ETags are lightweight stat-based metadata tokens; cluster ETags are opaque UUID tokens. Neither is a content MD5 hash. This avoids hashing very large video/object streams during upload. Use an explicit application checksum in object metadata when content verification is required.

Authentication uses AWS Signature V4 for boto3 requests and presigned GET/PUT/POST. Header timestamps, credential scope, POST policy conditions, body SHA-256, and supported streaming checksum trailers are validated. Legacy Signature V2 is disabled unless allow_legacy_auth is explicitly enabled. The demo sciveo/secret credentials are rejected by server startup unless allow_demo_credentials is explicitly enabled for a disposable test. Use a protected secret file and terminate public HTTPS at a hardened reverse proxy.

Gunicorn with gthread workers is the Linux production HTTP runtime. Waitress is an explicitly configured compatibility runtime that buffers request bodies and is intended for local tests. Sciveo fails fast instead of attempting Gunicorn prefork on macOS.

Python client example:

import boto3

s3 = boto3.client(
    "s3",
    endpoint_url="http://127.0.0.1:9000",
    aws_access_key_id="sciveo",
    aws_secret_access_key="CHANGE_ME",
    region_name="us-east-1",
)

s3.upload_file("clip.mp4", "media", "clips/clip.mp4")
s3.download_file("media", "clips/clip.mp4", "clip-copy.mp4")

Object tags use the S3 object-tagging model. Add tags during upload with the normal Tagging/x-amz-tagging mechanism, update them with put_object_tagging, and search them through the Sciveo metadata search API:

s3.put_object(
    Bucket="media",
    Key="clips/clip.mp4",
    Body=open("clip.mp4", "rb"),
    Tagging="kind=raw&project=demo",
)

s3.put_object_tagging(
    Bucket="media",
    Key="clips/clip.mp4",
    Tagging={"TagSet": [{"Key": "stage", "Value": "ready"}]},
)

Sciveo also provides a boto3-like remote client that keeps the normal S3 methods and adds Sciveo-native metadata search endpoints:

import sciveo.storage.s3

s3 = sciveo.storage.s3.client(
    endpoint_url="http://127.0.0.1:9000",
    aws_access_key_id="sciveo",
    aws_secret_access_key="CHANGE_ME",
    region_name="us-east-1",
)

s3.upload_file("clip.mp4", "media", "clips/clip.mp4")
objects = s3.search_objects(Bucket="media", Prefix="clips/", Period=7200)
tagged = s3.search_objects(Bucket="media", Prefix="clips/", Tags={"stage": "ready"})
stats = s3.search_stats(Bucket="media", Prefix="clips/")
metrics = s3.metrics(Bucket="media")

The Sciveo-native HTTP surface is intentionally small and separate from AWS S3 compatibility: /sciveo/storage/search, /sciveo/storage/stats, /sciveo/storage/disk-usage, /sciveo/storage/health, /sciveo/storage/roots, /sciveo/storage/buckets, and /sciveo/storage/metrics. Search and stats are DB-indexed and support exact generic tag filters with tag.NAME=value or Tags={"name": "value"} through the Python client. /sciveo/storage/metrics returns OpenMetrics text for Prometheus-style scraping.

Storage stats also include rolling file-operation counts for the latest 24 UTC hours. The profile is calculated with a SQL aggregation over the existing hourly operation counters, so it adds no counters or writes to the request path. JSON clients read it from operations.hourly_files_utc; terminal users get the same 24-row table with:

sciveo storage --s3 --config /etc/sciveo/storage-s3.json --stats --render text

Each row includes its UTC hour, total file operations, files / 60 average per minute, and write/read/delete counts.

Disk-usage totals count each physical filesystem once even when multiple storage roots share it. Per-root object counts and bytes remain separate so operators can still inspect placement. S3 object tags follow AWS request limits: at most 10 tags, with keys up to 128 characters and values up to 256 characters.

Local workers running on the same storage machine can resolve an object to its committed file path without downloading it through S3:

from sciveo.storage.local import StorageLocalClient

storage = StorageLocalClient.from_config("/etc/sciveo/storage-s3.json")
storage.create_bucket("media")
path = storage.path("media", "clips/clip.mp4")

# Use path directly with local video/image processing code.
storage.copy("media", "clips/clip.mp4", dst_key="processed/clip.mp4")
storage.move("media", "tmp/clip.mp4", dst_key="queue/0/clip.mp4")
storage.delete_objects("media", ["processed/clip.mp4", "queue/0/clip.mp4"])
storage.delete_bucket("media")

StorageLocalClient reads the same storage config and SQL metadata DB as the server. It only returns paths for committed objects whose files still exist, rejects metadata paths outside the configured storage roots, and updates both object files and metadata for local bucket, copy, move, and delete operations. move() defaults to no_copy=True: the file is renamed on its current storage path and only the bucket/key metadata changes, so changing a key does not trigger an inter-disk copy when the destination key hashes to another path. Use move(..., no_copy=False) only when the old copy/delete behavior is desired.

Run bounded metadata, staging, and deferred-delete cleanup explicitly. The deletions section reports tombstones seen, reconciled, and deferred:

sciveo storage --s3 --action maintenance --dry-run --config ./storage.json
sciveo storage --s3 --action maintenance --config ./storage.json

Ordinary boto3 clients do not need an update for these server-side changes. Processes importing StorageLocalClient should run the same current Sciveo release as the servers so they share locking, tombstone, root-identity, and rollback behavior. Update the Sciveo remote client when using newly added native search/health/stats methods; normal delegated S3 method signatures remain boto3-compatible.

When upgrading from a release before metadata schema v5, stop every S3 server and local writer using these roots, update them together, then restart. Do not mix old and new lock/journal implementations during a rolling restart. Schema migration is automatic and does not run --sync; run an explicit dry-run/full sync later as an operational consistency audit.

The repository includes a destructive-only-to-its-marked-directory stress runner. It prints a parameter table before opening sockets and a result table at completion:

PYENV_VERSION=sciveo python test/storage_runtime.py \
  --path /tmp/sciveo-storage-runtime \
  --servers 4 --threads 16 --clients 16 --roots 5 --files 240

The runtime mixes boto3, Sciveo remote, and local clients; exercises presigned requests, transfers, tags, conditions, ranges, copy directives, multipart, pagination, root loss/recovery, process-crash recovery, root omission/re-add, disjoint-DB rejection, live sync, and parallel apply-sync while a local cleanup client deletes the oldest indexed cohort; then verifies reclaimed free space and exact bytes against SQL metadata. Every parallel sync pass, the final apply sync, and the final dry-run must report zero inconsistencies. Every run writes a staged JSON report under /private/tmp unless --report overrides it. Use --db-backend postgres --db-url ... with a fresh disposable database to repeat the same gate with PostgreSQL.

Experiments Client

The experiment helpers expose project runs, parameter sampling, datasets, scoring, plots, metadata, local execution, and optional remote synchronization. They are useful for ML, AI, research, and engineering scripts where each run should have a known configuration and a recorded result.

Minimal pattern:

import sciveo

def run_once():
    with sciveo.open() as experiment:
        experiment.log({"message": "hello experiment"})
        experiment.score(0.0)

sciveo.start(
    project="example-project",
    function=run_once,
    configuration={},
    remote=False,
    count=1,
)

Run a project:

import sciveo

def evaluate_model():
    with sciveo.open() as experiment:
        learning_rate = experiment.config.learning_rate
        experiment.log("learning_rate", learning_rate)
        experiment.score(0.94)

sciveo.start(
    project="example-project",
    function=evaluate_model,
    configuration={"learning_rate": {"values": [0.001, 0.01]}},
    remote=False,
    count=2,
)

Remote synchronization requires a configured Sciveo API account.

Experiment Concepts

• Project: a named ML, research, or engineering workspace containing related runs.
• Experiment: one run with a sampled configuration, measurements, outputs, and score.
• Configuration: parameter values used by the run.
• Dataset records: structured references to input data, split definitions, or generated data artifacts.
• Score: numeric or structured result used for comparison and optimization.
• Local mode: runs experiments on the current machine without API synchronization.
• Remote mode: synchronizes project and experiment data through the Sciveo API when configured.

Example with dataset and score metadata:

import sciveo

def evaluate_dataset():
    with sciveo.open() as experiment:
        experiment.dataset({
            "name": "sensor-window-001.csv",
            "split": {"train": 0.8, "test": 0.2},
        })
        experiment.eval("rmse", 0.034)
        experiment.eval("mae", 0.021)
        experiment.score(0.93)

sciveo.start(
    project="sensor-eval",
    function=evaluate_dataset,
    configuration={},
    remote=False,
    count=1,
)

Example local parameter sweep:

import sciveo

def calibrate():
    with sciveo.open() as experiment:
        window = experiment.config.window
        threshold = experiment.config.threshold
        experiment.log({"window": window, "threshold": threshold})
        experiment.score(0.8)

sciveo.start(
    project="lab-calibration",
    function=calibrate,
    configuration={
        "window": {"values": [32, 64, 128]},
        "threshold": {"values": [0.1, 0.2, 0.3]},
    },
    remote=False,
    sampler="grid",
)

Sciveo does not force a specific analysis or ML stack. Experiments can call NumPy, SciPy, scikit-learn, PyTorch, TensorFlow, OpenCV, custom sensor clients, or any other Python code that fits the workflow.

Python Module Map

Common package areas:

• sciveo.monitoring: metrics, watchdogs, monitoring CLI.
• sciveo.network: scan, SSH, Modbus, SNMP, MQTT, HTTP, emulators.
• sciveo.admin: edge admin web UI and service helpers.
• sciveo.ops: doctor and fleet diagnostics.
• sciveo.agents: agent console, providers, local runtimes.
• sciveo.chat: encrypted chat server/client transport.
• sciveo.media: capture, media CLI, pipeline workers.
• sciveo.ml.images: image ML helpers, SCIVEYOLO inference, embeddings, descriptions, image transforms.
• sciveo.storage: local S3-compatible object store.
• sciveo.certification: framework profiles, readiness scoring, suggestions, rendering, and certification CLI delegation.
• sciveo.api: API clients and predictor server.
• sciveo.db, sciveo.web, sciveo.content, sciveo.tools: shared support modules.

Development Checks

Useful local checks:

python -m py_compile sciveo/cli.py sciveo/ml/images/sciveyolo/model.py sciveo/ml/images/sciveyolo/torch_net.py
python -m unittest discover -s test -p "test_*.py" -v

For SCIVEYOLO runtime validation, use a UT-capable build environment to create .sciveyolo artifacts, then test production loading in an environment that has PyTorch/OpenCV/NumPy but no Ultralytics installed.

Contact

Pavlin Georgiev
[e-mail]