maqet 0.0.15

M4x0n's QEMU Tool - A Python-based QEMU automation framework for VM running

MAQET

Warning: Most of the code was written using AI. This product is a work in progress and should not be used in production environments under any circumstances.

MAQET (M4x0n's QEMU Tool) is a VM management system that implements unified API generation. Methods decorated with @api_method automatically become CLI commands, Python API methods, and configuration-driven calls.

Quick Start

Installation

pip install maqet

Optional Dependencies:

• psutil - Enhanced process management and validation (recommended)

  pip install psutil
  

  Without psutil, basic PID tracking still works but ownership validation is skipped.

Breaking Changes

v0.1.0: SSH Readiness Checking Removed

MAQET v0.1.0 removes SSH readiness checking to focus on VM infrastructure management.

Removed:

• --wait-for ssh-ready CLI option
• --ssh-port and --ssh-host CLI flags
• ssh_port and ssh_host Python API parameters

Available wait conditions:

• process-started (default): VM runner process is ready
• file-exists: Wait for specific file to exist

Migration: Use standard SSH tools (ssh-keyscan, ssh with retries) for SSH checking. See Migration Guide v0.1.0 for detailed migration instructions and examples.

v0.0.8+: QEMU Vendoring

MAQET v0.0.8+ vendors QEMU Python bindings internally for reliable installation.

Before (v0.0.7 and earlier):

pip install maqet[qemu]  # DON'T USE ANYMORE

Now (v0.0.8+):

pip install maqet  # QEMU bindings included automatically

Why? The official qemu.qmp PyPI package had packaging issues. Vendoring ensures reliable installation across all platforms.

For existing users: Simply run pip install --upgrade maqet. No code changes needed if you use MAQET's API methods.

v0.0.11: License Change

MAQET changed from MIT to GPL-2.0-only due to vendored QEMU code (GPL-2.0).

Impact:

• CLI usage: No impact
• Library usage in GPL-compatible projects: No impact
• Library usage in proprietary/MIT projects: May require license review

See Migration Guide for details.

v0.0.11: Security Improvements

Automatic - no action required:

• Unix socket permissions: Now 0600 (user-only access)
• Path traversal protection: System directories blocked
• Database performance: 100x faster with 100+ VMs

Action required if your config uses system directories:

# Before (blocked in v0.0.11):
storage:
  - file: /etc/disk.qcow2  # ValueError

# After (use user directories):
storage:
  - file: ~/vms/disk.qcow2  # OK

See CHANGELOG for complete details.

Core Concept

Write once, use everywhere. A single method becomes a CLI command, Python API, and configuration option:

@api_method(cli_name="start", description="Start a virtual machine", category="vm")
def start(self, vm_id: str, detach: bool = False):
    """Start a virtual machine."""
    # Single implementation

This automatically creates:

• CLI: maqet start myvm --detach
• Python API: maqet.start("myvm", detach=True)
• Config: VM settings only (no commands in YAML)

Usage

Command Line Interface

# Create a VM
maqet add config.yaml --name myvm

# Start VM
maqet start myvm

# List all VMs
maqet ls

# Check VM status
maqet status myvm

# Execute QMP command
maqet qmp myvm system_powerdown

# Remove VM
maqet rm myvm --force

Python API

from maqet import Maqet
from pathlib import Path

maqet = Maqet()

# Create and start VM from config file
# Accepts both string and Path objects
vm_id = maqet.add(config="config.yaml", name='myvm')  # String path
# Or using pathlib.Path
config_path = Path("~/my-vms/config.yaml").expanduser()
vm_id = maqet.add(config=config_path, name='myvm')

# Start VM
maqet.start(vm_id)

# Manage VM
status = maqet.status(vm_id)
maqet.qmp(vm_id, 'system_powerdown')
maqet.rm(vm_id, force=True)

Configuration Files

# config.yaml - VM configuration only
name: myvm
binary: /usr/bin/qemu-system-x86_64
memory: 4G
cpu: 2
storage:
  - name: hdd
    size: 20G
    type: qcow2
    interface: virtio

# Use configuration file
maqet add config.yaml
maqet start myvm --detach

Configuration Features:

• Deep-merge multiple config files
• Lists are concatenated (storage, network)
• Command-line args override config values
• Full QEMU argument support

See Configuration Guide for details.

Core Commands

Command	Description	Example
add	Create new VM	maqet add config.yaml --name myvm
start	Start VM	maqet start myvm
stop	Stop VM	maqet stop myvm --force
rm	Remove VM	maqet rm myvm --force
ls	List VMs	maqet ls --status running
status	Show VM status	maqet status myvm
apply	Apply configuration	maqet apply myvm --memory 8G
snapshot	Manage snapshots	maqet snapshot myvm create hdd snap1

QMP Commands

Command	Description	Example
qmp keys	Send key combination	maqet qmp keys myvm ctrl alt f2
qmp type	Type text to VM	maqet qmp type myvm "hello world"
qmp screendump	Take screenshot	maqet qmp screendump myvm screenshot.ppm
qmp pause	Pause VM	maqet qmp pause myvm
qmp resume	Resume VM	maqet qmp resume myvm
qmp device-add	Hot-plug device	maqet qmp device-add myvm usb-storage
qmp device-del	Hot-unplug device	maqet qmp device-del myvm usb1

Global Options

Option	Description
-v, --verbose	Increase verbosity: -v=warnings, -vv=info, -vvv=debug (default: errors only)
--maqet-data-dir	Override data directory path
--maqet-config-dir	Override config directory path
--maqet-runtime-dir	Override runtime directory path
--log-file	Enable file logging

Directory Configuration

MAQET supports flexible directory configuration with the following precedence order (highest to lowest):

1. CLI flags (highest priority): --maqet-data-dir, --maqet-config-dir, --maqet-runtime-dir
2. Config file: maqet.conf settings (searched in: ./maqet.conf, ~/.config/maqet/maqet.conf, /etc/maqet/maqet.conf)
3. Environment variables: XDG_DATA_HOME, XDG_CONFIG_HOME, XDG_RUNTIME_DIR
4. XDG defaults (lowest priority): ~/.local/share/maqet, ~/.config/maqet, /run/user/$(id -u)/maqet

Example maqet.conf:

directories:
  data_dir: ~/custom/maqet/data
  config_dir: ~/custom/maqet/config
  runtime_dir: /tmp/maqet-runtime

CLI override example:

# Config file says data_dir is ~/custom/maqet/data
# CLI flag overrides it
maqet ls --maqet-data-dir /tmp/test-data

Documentation

User Guides

• Installation Guide - Installation and setup
• Quick Start Guide - Get started quickly
• Configuration Guide - VM configuration reference
• Storage Lifecycle Management - VM deletion, orphaned storage, and recovery
• Storage Management - QEMU storage types, snapshots, and performance
• CLI Precedence Guide - How CLI flags, config files, and environment variables interact
• Migration Guide - Database migration and version upgrades
• Troubleshooting Guide - Common issues and solutions

Full Documentation

• Documentation Index - Complete documentation portal
• Architecture - Internal architecture and design
• Development - Contributing and development guides
• Deployment - Production deployment
• Reference - Technical references

Architecture

• Unified API System - Single methods generate CLI, Python API, and config
• State Management - SQLite backend with XDG compliance
• QEMU Integration - Full QMP protocol support
• Storage System - QCOW2, Raw, VirtFS support with snapshots

See QEMU Internal Architecture for details.

Development

Running Tests

Maqet uses pytest with parallel execution support for fast testing.

Quick Commands:

# Run all tests in parallel (recommended) - ~54 seconds
pytest -n auto

# Run all tests serially (for debugging) - ~120 seconds
pytest

# Run specific test categories in parallel
pytest -n auto tests/unit/         # Unit tests (~20s)
pytest -n auto tests/integration/  # Integration tests (~25s)
pytest -n auto tests/e2e/          # End-to-end tests (~15s)
pytest -n auto tests/performance/  # Performance tests (~10s)

# Run with coverage
pytest -n auto --cov=maqet --cov-report=html

# Run and stop on first failure
pytest -n auto -x

Test Organization:

• Unit Tests (tests/unit/): Fast, isolated, fully mocked
• Integration Tests (tests/integration/): Real database, mocked processes
• E2E Tests (tests/e2e/): Complete workflows with real components
• Performance Tests (tests/performance/): Benchmarks and regression tests

ProcessTestHarness:

Reliable subprocess testing with /proc stabilization:

from tests.utils.process_harness import ProcessTestHarness

with ProcessTestHarness(["sleep", "60"]) as harness:
    # Process guaranteed ready, /proc populated
    verify_process(harness.pid, ...)
# Automatic cleanup

Connection Pooling:

StateManager uses connection pooling for 10-50x faster database queries:

• Pool of 5 reusable connections for reads
• Dedicated connections for writes (avoid lock contention)
• Thread-safe with SQLite WAL mode

Parallel vs Serial Execution:

Use parallel mode (default):

• Fast test execution (55% faster)
• Simulates concurrent usage patterns
• Recommended for regular development

Use serial mode when:

• Debugging test failures
• Analyzing test output carefully
• Running under debugger (pdb)

Pre-commit Testing:

For fast pre-commit checks, run E2E tests in parallel:

# .git/hooks/pre-commit
#!/bin/bash
pytest -n auto tests/e2e/ --maxfail=1 -q

E2E tests complete in ~15 seconds with parallel execution.

Writing Parallel-Safe Tests:

See tests/PARALLEL_TESTING.md for:

• Pytest-xdist execution model
• Common pitfalls and solutions
• Best practices and patterns
• Debugging strategies

Test Requirements:

Tests require:

• Python 3.12+
• pytest and plugins (installed with pip install -e ".[dev]")
• Optional: QEMU for E2E tests (skipped if not installed)

See tests/README.md for detailed testing documentation and docs/development/TESTING.md for contributing guidelines.

Roadmap

See Roadmap and Future Features for planned improvements.

Features

• Write Once, Use Everywhere - Single method for CLI, API, and config
• XDG Compliant - Follows Linux directory standards
• Production Ready - Security hardened, tested, robust error handling
• Full QMP Support - Complete QEMU Machine Protocol integration
• Snapshot Management - Create, load, list, and delete snapshots
• Hot-plug Support - Add/remove devices while VM is running

Security

Intended Use: Maqet is a local development tool for single-user workstations. It is NOT designed for multi-tenant environments, production servers, or scenarios where untrusted users have access to the system. Security measures are appropriate for protecting against accidental misuse and basic local threats, not sophisticated attacks from users with system access.

Maqet implements defense-in-depth security for VM operations:

Authentication Secret Protection

• TOCTOU Protection: File descriptor-based atomic operations prevent race conditions
• O_NOFOLLOW: Prevents symlink attacks on secret files
• Permission Validation: Enforces 0600 (user-only) permissions
• Ownership Verification: Ensures secrets owned by current user

Input Validation

• Command Injection Prevention: Shell metacharacter detection
• Path Traversal Prevention: ".." sequence detection
• Argument Injection Prevention: Leading hyphen checks
• Resource Limits: Length and size validation

Security Module

All security-sensitive inputs validated through maqet.security.validation.InputValidator:

• VM IDs and names
• Filesystem paths
• Binary paths

Threat Model

• Attacker cannot read secrets via symlink attacks
• Attacker cannot inject commands via VM IDs
• Attacker cannot escape data directories via path traversal
• Attacker cannot manipulate process arguments

Contributing

Contributions welcome! See Development Guide for contributing guidelines.

License

GNU General Public License v2.0 - see LICENSE file for details.