pywats-api 0.4.0b1

Python API library for WATS - manufacturing test data management

PyWATS

🤖 AI-Powered Development: This entire project was designed and implemented using AI-assisted coding by Ola Lund Reppe, Integration Architect at The WATS Company AS.

A Python library for interacting with the WATS test data management platform API.

⚠️ Beta Release: This is a beta version. The API is stabilizing but may have changes before 1.0.

Please read the Beta Disclaimer to understand the current stability levels of different components.

Requirements

• Python 3.10 or later
• WATS Server 2025.3.9.824 or later

Platform Support

Platform	Library	GUI Client	Headless Client	Service/Daemon
Windows 10/11, Server 2019+	✅	✅	✅	✅ Native Service
Windows IoT Enterprise LTSC	✅	⚠️	✅	✅ Native Service
Ubuntu 22.04/24.04 LTS	✅	✅	✅	✅ systemd
Debian 11/12	✅	✅	✅	✅ systemd
RHEL/Rocky/Alma 8/9	✅	✅	✅	✅ systemd + SELinux
macOS 12+ (Intel & Apple Silicon)	✅	✅	✅	✅ launchd
Raspberry Pi (64-bit)	✅	⚠️	✅	✅ systemd
Docker (amd64/arm64)	✅	❌	✅	✅ Container

✅ Full support ⚠️ Limited ❌ Not available

See Platform Compatibility Guide for detailed requirements.

Features

• PyWATS Library - Core API library for WATS integration

  • Async-First Architecture - Built on httpx with native async support
  • Sync Compatibility - Full sync API via thin wrappers (no code changes needed)
  • 9 Domain Services: Product, Asset, Report, Production, Analytics, Software, RootCause, Process, SCIM
  • 170+ API Endpoints - Centralized route management
  • Report creation and submission with comprehensive step types
  • OData filtering and pagination support
  • Structured logging with configurable verbosity
  • Performance Optimizations ⚡
    • Enhanced TTL Caching - Automatic expiration with background cleanup (100x faster cache hits)
    • Connection Pooling - HTTP/2 multiplexing with 100 max connections (3-5x faster bulk operations)
    • Request Batching - Time-window and size-based batching utilities (95% reduction in server calls)
    • MessagePack Serialization - 50% smaller payloads, 3x faster serialization (optional)
  • Offline Queue - File-based queue for reliable report submission when offline
    • WSJF format as standard
    • Format conversion from WSXF, WSTF, ATML
    • Automatic retry with configurable max attempts
    • Queue capacity management (configurable max size)
    • Concurrent upload control (configurable concurrency)
    • Metadata tracking and statistics

• PyWATS Client - Desktop and headless client application

  • Priority-Based Converter Processing ✅ NEW
    • Configure converter priority (1=highest, 10=lowest)
    • Prevents batch uploads from blocking real-time production data
    • Heap-based priority queue with AsyncQueueAdapter
    • Zero code changes - set priority in config or GUI
  • Security Features ✅
    • IPC authentication with shared secrets
    • Converter process isolation and sandboxing
    • Safe file handling with atomic writes and locking
    • Rate limiting on IPC connections
  • Configuration & Versioning ✅
    • Protocol version tracking (2.0+)
    • Config schema versioning (1.0→2.0 auto-upgrade)
    • Queue size limits and concurrent upload control
  • Connection management with multi-instance support
  • Converter configuration and management
  • Report queue management
  • GUI Mode: Qt-based desktop application (Windows, macOS, Linux)
  • Service Mode: Background Windows service or Unix daemon
  • Headless Mode: CLI and HTTP API for servers, Raspberry Pi, embedded systems

Installation

From PyPI (Recommended)

# Install core API library only
pip install pywats-api

# Install with GUI client (requires Qt)
pip install pywats-api[client]

# Install headless client (no Qt - for Raspberry Pi, servers)
pip install pywats-api[client-headless]

Native Installers

Pre-built installers are available on the Releases page:

Platform	Installer	Notes
Windows	.msi	Silent install: msiexec /i pyWATS-x.x.x.msi /quiet
macOS	.dmg / .pkg	Universal Binary (Intel + Apple Silicon)
Linux	.deb / .rpm	Ubuntu/Debian and RHEL/Rocky/Alma
Linux	AppImage	Portable, no install required

See deployment/README.md for building installers from source.

From Source (Development)

# Clone the repository
git clone https://github.com/olreppe/pyWATS.git
cd pyWATS

# Create virtual environment
python -m venv .venv
.venv\Scripts\activate  # Windows
source .venv/bin/activate  # Linux/Mac

# Install in development mode
pip install -e ".[dev]"

Configuration

Create a configuration with your WATS credentials:

from pywats import pyWATS

api = pyWATS(
    base_url="https://your-server.wats.com",
    token="your_base64_encoded_token"
)

Or use environment variables:

WATS_BASE_URL=https://your-server.wats.com
WATS_AUTH_TOKEN=your_base64_encoded_token

Quick Start

from pywats import pyWATS, WATSFilter

# Initialize API
api = pyWATS(
    base_url="https://your-server.wats.com",
    token="your_token"
)

# Test connection
if api.test_connection():
    print(f"Connected! Server version: {api.get_version()}")

# Get products (sync)
products = api.product.get_products()
for p in products:
    print(f"{p.part_number}: {p.name}")

Async Quick Start

For high-performance applications with concurrent requests:

import asyncio
from pywats import AsyncWATS

async def main():
    async with AsyncWATS(
        base_url="https://your-server.wats.com",
        token="your_token"
    ) as api:
        # Concurrent requests - much faster!
        products, assets, version = await asyncio.gather(
            api.product.get_products(),
            api.asset.get_assets(top=10),
            api.analytics.get_version()
        )
        print(f"Fetched {len(products)} products, {len(assets)} assets")

asyncio.run(main())

Performance Optimization Example

Use caching, batching, and connection pooling for high-performance applications:

import asyncio
from pywats import AsyncWATS
from pywats.core.coalesce import coalesce_map

async def main():
    async with AsyncWATS(
        base_url="https://your-server.wats.com",
        token="your_token"
    ) as api:
        # Built-in caching for static data (95% reduction in server calls)
        processes = await api.process.get_processes()  # First call - from server
        processes = await api.process.get_processes()  # Second call - from cache (100x faster!)
        
        # Check cache statistics
        stats = api.process.cache_stats
        print(f"Cache hit rate: {stats['hit_rate']:.1%}")
        
        # Batch processing with automatic concurrency control
        serial_numbers = [f"SN-{i:05d}" for i in range(1000)]
        
        async def fetch_unit(sn: str):
            return await api.production.get_unit(sn, "WIDGET-001")
        
        # Process 1000 units in batches (5-10x faster than sequential)
        units = await batch_map(
            items=serial_numbers,
            func=fetch_unit,
            batch_size=50,
            max_concurrent=10
        )
        print(f"Processed {len(units)} units")

asyncio.run(main())

See PERFORMANCE_OPTIMIZATIONS.md for complete guide.

Query Reports

# Query recent reports (OData filter)
headers = api.report.query_uut_headers(
    odata_filter="partNumber eq 'WIDGET-001'",
    top=10
)

# Or use helper methods
headers = api.report.get_headers_by_serial("SN-12345")
headers = api.report.get_todays_headers()

Enable Debug Logging

from pywats import pyWATS, enable_debug_logging

# Quick debug mode - shows all library operations
enable_debug_logging()

# Or configure logging your way
import logging
logging.basicConfig(level=logging.INFO)
logging.getLogger('pywats').setLevel(logging.DEBUG)

# Now use the API with detailed logging
api = pyWATS(base_url="...", token="...")

See LOGGING_STRATEGY.md for comprehensive logging documentation.

Running the GUI Client

Important: The GUI is for configuration only. You must run the service first:

# Start the background service (runs 24/7)
python -m pywats_client service --instance-id default

# Then launch GUI for configuration (in a separate terminal)
python -m pywats_client gui --instance-id default

Or simply run:

# Default behavior: starts service
python -m pywats_client

# Then launch GUI when needed
python -m pywats_client gui

Service Installation (Production)

For production deployments, install pyWATS as a system service that auto-starts on boot:

Windows:

# Install as Windows Service (requires admin)
python -m pywats_client install-service
net start pyWATS_Service

Linux (systemd):

# Install as systemd service (requires sudo)
sudo python3 -m pywats_client install-service
sudo systemctl start pywats-service

macOS:

# Install as Launch Daemon (requires sudo)
sudo python3 -m pywats_client install-service

# Or as user-level Launch Agent (no sudo)
python3 -m pywats_client install-service --user-agent

See platform-specific guides:

• Windows Service Setup - NSSM/sc.exe installation
• Linux Service Setup - systemd for Ubuntu/Debian/RHEL
• macOS Service Setup - launchd daemon/agent

GUI Configuration

The GUI supports modular tab configuration and logging control:

• Tab Visibility: Show/hide tabs (Software, SN Handler, etc.) based on your needs
• Logging Integration: Automatic PyWATS library logging when debug mode is enabled
• Multiple Instances: Run multiple client instances with separate configurations
• Service Discovery: Automatically discovers and connects to running service instances

See GUI Configuration Guide for detailed setup instructions.

Running Headless (Raspberry Pi, Servers)

For systems without display or Qt support:

# Initialize configuration
pywats-client config init

# Test connection
pywats-client test-connection

# Run service in foreground
python -m pywats_client service

# Run with HTTP control API
python -m pywats_client service --api --api-port 8765

# Install as system service (auto-start on boot)
sudo python3 -m pywats_client install-service

CLI Commands

pywats-client config show          # Show configuration
pywats-client config set key value # Set config value
pywats-client status               # Show service status
pywats-client converters list      # List converters

HTTP Control API

When running with --api, manage the service remotely:

curl http://localhost:8765/status    # Get status
curl http://localhost:8765/config    # Get configuration
curl -X POST http://localhost:8765/restart  # Restart services

See Headless Operation Guide for complete documentation.

Project Structure

pyWATS/
├── src/
│   ├── pywats/              # Core library
│   │   ├── domains/         # Domain services (async + sync wrappers)
│   │   │   ├── analytics/   # Statistics, yield, Unit Flow
│   │   │   ├── asset/       # Equipment tracking, calibration
│   │   │   ├── process/     # Operation types, caching
│   │   │   ├── product/     # Products, revisions, BOMs
│   │   │   ├── production/  # Serial numbers, unit lifecycle
│   │   │   ├── report/      # Test reports, measurements
│   │   │   ├── rootcause/   # Issue tracking, defects
│   │   │   ├── scim/        # User provisioning
│   │   │   └── software/    # Package distribution
│   │   ├── core/            # HTTP client, routes, sync_runner
│   │   ├── models/          # Report models (UUT/UUR)
│   │   ├── async_wats.py    # AsyncWATS main class
│   │   └── sync_wats.py     # SyncWATS wrapper (pyWATS alias)
│   └── pywats_client/       # Client application
│       ├── core/            # Core client functionality
│       ├── gui/             # Qt GUI components (optional)
│       ├── control/         # Headless control (CLI, HTTP API)
│       └── services/        # Background services
├── converters/              # User converter plugins
├── docs/                    # Documentation
├── examples/                # Usage examples by domain
└── pyproject.toml           # Project configuration

Documentation

Official Documentation

Complete guides shipped with the package:

• Documentation Index - Complete documentation overview

Domain API Reference

• Product Domain - Products, revisions, BOMs, box build templates
• Asset Domain - Equipment tracking, calibration, maintenance
• Production Domain - Unit lifecycle, serial numbers, assembly
• Report Domain - Test reports, measurements, step types
• Analytics Domain - Yield analysis, measurements, Unit Flow
• Software Domain - Package management, versioning, distribution
• RootCause Domain - Issue tracking, defect management
• Process Domain - Operation types, caching

Client Documentation

• GUI Configuration - Configure GUI tabs, logging
• Headless Operation - Raspberry Pi, servers, embedded

Additional Resources

• Changelog - Version history and release notes
• Error Catalog - Comprehensive error reference

Testing

# Run all tests
pytest

# Run with coverage
pytest --cov=src --cov-report=html

Contributing

This project is maintained by Virinco AS.

For Maintainers: Releasing a New Beta Version

There is only ONE command to release:

.\scripts\bump.ps1

See RELEASE.md for complete details. Never manually edit versions or create tags.

License

MIT License - see LICENSE for details.

Links

• WATS Website
• Virinco
• GitHub Repository
• Changelog