Skip to main content

Contributors Forks Stargazers Issues MIT License

About The Project

Overview

This package was created by factoring out many reusable code artifacts from my various projects over a number of years. Since this work was not a part of a group effort, the test coverage is predictably abysmal :raised_eyebrow: and Python documentation notably absent :expressionless:. For each of my projects, which derive from a common Docker application found here, this package provides some specific and relatively powerful features to enable rapid offshoots for new ideas.

The package is organized under src/tailucas_pylib/ with the following key modules:

  • Core Modules:

    • __init__.py: Application bootstrap with logging, locale, configuration (via app.conf), and device name setup. Logs to syslog (UDP) or stdout/stderr. Optionally changes working directory to WORK_DIR.
    • creds.py: 1Password credential management supporting both 1Password Connect Server and 1Password Service Account modes. Fetches secrets from environment variables or container secrets (/run/secrets).
    • flags.py: Feature flag checking backed by a 1Password credential item.
  • Application Framework:

    • app.py: Thread base class with ZMQ relay (ZmqRelay) and worker (ZmqWorker) patterns for inter-thread communication via ZeroMQ.
    • threads.py: Thread nanny with shutdown tracking, graceful termination, and lingering socket cleanup.
    • handler.py: Context manager (exception_handler) that handles ZMQ connectivity, ContextTerminated, ResourceWarning, and general exceptions with optional OpenTelemetry exception tracing and graceful shutdown.
    • process.py: Signal handler (SignalHandler) with subprocess execution (exec_cmd) helpers.
  • Communication:

    • rabbit.py: RabbitMQ integration with MQConnection, ZMQListener (consumes RabbitMQ messages and forwards to ZMQ), and RabbitMQRelay (bridges ZMQ to RabbitMQ).
    • zmq.py: ZeroMQ socket creation, lifecycle management (Closable), and graceful context teardown.
  • Utilities:

    • data.py: Builds MessagePack payloads with timestamp and optional data for IPC.
    • datetime.py: Timezone-aware timestamp creation, ISO formatting, and Unix timestamp conversion.
    • device.py: Dataclass data model (Device) for device state with optional fields.
    • bluetooth.py: Bluetooth adaptor detection and device ping via hcitool / l2ping.
  • AWS Integration (aws/):

    • __init__.py: Manages cached Boto3 sessions with STS role assumption, credential retrieval from 1Password.
    • metrics.py: Posts CloudWatch count metrics with dimensions.
  • Command-line Tools (tools/):

    • config_interpol.py: Interpolates a config file with environment variables or an overlay config, outputting the resolved values.
    • cred_tool.py: Fetches and displays a 1Password credential read from stdin.
    • yaml_interpol.py: Processes YAML templates, substituting values from configuration sections.
    • aws_configure.py: Generates AWS CLI configuration files from 1Password-stored credentials.

(back to top)

Built With

Technologies that help make this package useful:

1Password Amazon AWS uv Python RabbitMQ ZeroMQ

Also:

GitHub

Core Technologies:

  • Python 3.11+ - Primary runtime
  • 1Password Connect/Service Account - Credential management
  • ZeroMQ - Inter-process communication
  • RabbitMQ - Message queuing
  • MessagePack - Binary serialization
  • Pydantic - Data validation and modeling
  • OpenTelemetry - Error tracking and monitoring

AWS Integration:

  • Boto3 - AWS SDK
  • CloudWatch - Metrics and monitoring
  • Simple Workflow Service - Workflow orchestration (deprecated)

Development Tools:

  • uv - Dependency management

(back to top)

Getting Started

Here is some detail about the intended use of this package.

Prerequisites

A Python project or runtime environment. Since this project is already initialized with [uv][vu-url] dependency management, I recommend that you continue to use it. Beyond the Python dependencies defined in the configuration, the package init carries a hardcoded dependency on 1Password in order to function. Unless you want this and are effectively extending my base project, you're likely better off forking this package and cutting out what you do not need.

Installation

This package is published to the Python Package index from time to time.

(back to top)

Usage

I have various projects that use this tool chain. For example, my Base Project which can be run stand-alone but also serves as my Docker base image from which other projects are derived.

(back to top)

Testing

Tests use pytest with coverage. Some tests exercise live services (1Password Connect Server or Service Account, AWS session setup) and expect the OP_CONNECT_HOST, OP_CONNECT_TOKEN, OP_VAULT and OP_SERVICE_ACCOUNT_TOKEN environment variables. For deterministic results regardless of the ambient environment, the suite pins TZ=UTC and APP_NAME=test itself (see conftest.py).

Quick run (all extras)

uv sync --all-extras
uv run pytest

Extras matrix

Optional dependencies are organized as extras (aws, creds, monitoring, mq). Using a Hatch matrix, the full test suite also runs against every combination of these extras: 32 environments, each installing the package with only that combination's dependencies.

# list the 32 test environments
uv run hatch env show
# run the whole matrix (stops at the first failing environment)
uv run hatch run test:run
# run a single combination
uv run hatch run test.aws-no-creds-no-monitoring-mq:run
# remove all cached test environments
uv run hatch env prune

Tests that require an unavailable optional dependency skip themselves via pytest.importorskip, so every combination runs as much of the suite as its installed extras allow. Per-environment coverage data is written to .coverage.<env-name>.

Note that a full matrix run executes the live 1Password tests many times over, and the 1Password service-account API rate limit is shared across them. An occasional RateLimitExceeded failure during a full run is transient; re-run the affected environment to confirm.

(back to top)

License

Distributed under the MIT License. See LICENSE for more information.

(back to top)

Acknowledgments

(back to top)

Metadata

Release files for tailucas-pylib 0.8.3

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for tailucas-pylib 0.8.3
File Size Uploaded
tailucas_pylib-0.8.3.tar.gz 23.2 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for tailucas-pylib 0.8.3
File Interpreter ABI Platform
tailucas_pylib-0.8.3-py3-none-any.whl Python 3 none any Details

Total release size: 52.6 kB

Release files / tailucas_pylib-0.8.3.tar.gz

Download URL tailucas_pylib-0.8.3.tar.gz
Size 23.2 kB
Tags Source
SHA-256 checksum
How to use checksums
af0500c41541b7d4b40df590ec35e4d18bcb6c529f8717a38d838b11b453fd5f
BLAKE2b-256 checksum
How to use checksums
325ea6bbde7ba177339d3523f7fdbfb4b900dccc7c4b324c17c5b3d3a450944d
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.12.7 {"installer":{"name":"uv","version":"0.12.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release files / tailucas_pylib-0.8.3-py3-none-any.whl

Download URL tailucas_pylib-0.8.3-py3-none-any.whl
Size 29.3 kB
Tags Python 3
SHA-256 checksum
How to use checksums
b7cca4c76aaf4d63dc6a168d0021fb52a2bd4d2155ec0d25ec7585ded1ed8132
BLAKE2b-256 checksum
How to use checksums
0024aad46f8186f2727882b6a0acd08a57b25df21ccffbf49d50c561cc4fcdc4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via uv/0.12.7 {"installer":{"name":"uv","version":"0.12.7","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

Release history Release notifications | RSS feed

0.8.4

2 release files

This release

0.8.3 This release

2 release files

0.8.2

2 release files

0.8.1

2 release files

0.8.0

2 release files

0.7.8

2 release files

0.7.7

2 release files

0.7.6

2 release files

0.7.5

2 release files

0.7.4

2 release files

0.7.3

2 release files

0.7.2

2 release files

0.7.1

2 release files

0.7.0

2 release files

0.6.6

2 release files

0.6.5

2 release files

0.6.4

2 release files

0.6.3

2 release files

0.6.2

2 release files

0.6.1

2 release files

0.6.0

2 release files

0.5.9

2 release files

0.5.8

2 release files

0.5.7

2 release files

0.5.6

2 release files

0.5.5

2 release files

0.5.4

2 release files

0.5.3

2 release files

0.5.2

2 release files

0.5.1

2 release files

0.5.0

2 release files

0.4.9

2 release files

0.4.8

2 release files

0.4.7

2 release files

0.4.6

2 release files

0.4.5

2 release files

0.4.4

2 release files

0.4.3

2 release files

0.4.2

2 release files

0.4.1

2 release files

0.4.0

2 release files

0.3.9

2 release files

0.3.8

2 release files

0.3.7

2 release files

0.3.6

2 release files

0.3.5

2 release files

0.3.4

2 release files

0.3.3

2 release files

0.3.2

2 release files

0.3.1

2 release files

0.3.0

2 release files

0.2.9

2 release files

0.2.8

2 release files

0.2.7

2 release files

0.2.6

2 release files

0.2.5

2 release files

0.2.4

2 release files

0.2.3

2 release files

0.2.2

2 release files

0.2.1

2 release files

0.2.0

2 release files

0.1.9

2 release files

0.1.8

2 release files

0.1.7

2 release files

0.1.6

2 release files

0.1.5

2 release files

0.1.4

2 release files

0.1.3

2 release files

0.1.2

2 release files

0.1.1

2 release files

0.0.9

2 release files

0.0.8

2 release files

0.0.7

2 release files

0.0.6

2 release files

0.0.5

2 release files

0.0.4

2 release files

0.0.3

2 release files

0.0.2

2 release files

0.0.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page