Contributors Forks Stargazers Issues MIT License

About The Project

Overview

This package was created by factoring out many reusable code artifacts from my various projects over a number of years. Since this work was not a part of a group effort, the test coverage is predictably abysmal :raised_eyebrow: and Python documentation notably absent :expressionless:. For each of my projects, which derive from a common Docker application found here, this package provides some specific and relatively powerful features to enable rapid offshoots for new ideas.

The package is organized under src/tailucas_pylib/ with the following key modules:

  • Core Modules:

    • __init__.py: Application bootstrap with logging, locale, configuration (via app.conf), and device name setup. Logs to syslog (UDP) or stdout/stderr. Optionally changes working directory to WORK_DIR.
    • creds.py: 1Password credential management supporting both 1Password Connect Server and 1Password Service Account modes. Fetches secrets from environment variables or container secrets (/run/secrets).
    • flags.py: Feature flag checking backed by a 1Password credential item.

  • Application Framework:

    • app.py: Thread base class with ZMQ relay (ZmqRelay) and worker (ZmqWorker) patterns for inter-thread communication via ZeroMQ.
    • threads.py: Thread nanny with shutdown tracking, graceful termination, Cronitor monitoring integration, and lingering socket cleanup.
    • handler.py: Context manager (exception_handler) that handles ZMQ connectivity, ContextTerminated, ResourceWarning, and general exceptions with optional Sentry reporting and graceful shutdown.
    • process.py: Signal handler (SignalHandler) with subprocess execution (exec_cmd) helpers.

  • Communication:

    • rabbit.py: RabbitMQ integration with MQConnection, ZMQListener (consumes RabbitMQ messages and forwards to ZMQ), and RabbitMQRelay (bridges ZMQ to RabbitMQ).
    • zmq.py: ZeroMQ socket creation, lifecycle management (Closable), and graceful context teardown.

  • Utilities:

    • data.py: Builds MessagePack payloads with timestamp and optional data for IPC.
    • datetime.py: Timezone-aware timestamp creation, ISO formatting, and Unix timestamp conversion.
    • device.py: Pydantic data model (Device) for device state with optional fields.
    • bluetooth.py: Bluetooth adaptor detection and device ping via hcitool / l2ping.

  • AWS Integration (aws/):

    • __init__.py: Manages cached Boto3 sessions with STS role assumption, credential retrieval from 1Password.
    • metrics.py: Posts CloudWatch count metrics with dimensions.

  • Command-line Tools (tools/):

    • config_interpol.py: Interpolates a config file with environment variables or an overlay config, outputting the resolved values.
    • cred_tool.py: Fetches and displays a 1Password credential read from stdin.
    • yaml_interpol.py: Processes YAML templates, substituting values from configuration sections.
    • aws_configure.py: Generates AWS CLI configuration files from 1Password-stored credentials.

(back to top)

Built With

Technologies that help make this package useful:

1Password Amazon AWS uv Python RabbitMQ Sentry ZeroMQ

Also:

GitHub

Core Technologies:

  • Python 3.11+ - Primary runtime
  • 1Password Connect/Service Account - Credential management
  • ZeroMQ - Inter-process communication
  • RabbitMQ - Message queuing
  • MessagePack - Binary serialization
  • Pydantic - Data validation and modeling
  • Sentry - Error tracking and monitoring

AWS Integration:

  • Boto3 - AWS SDK
  • CloudWatch - Metrics and monitoring
  • Simple Workflow Service - Workflow orchestration (deprecated)

Development Tools:

  • uv - Dependency management
  • Cronitor - Cron job monitoring

(back to top)

Getting Started

Here is some detail about the intended use of this package.

Prerequisites

A Python project or runtime environment. Since this project is already initialized with [uv][vu-url] dependency management, I recommend that you continue to use it. Beyond the Python dependencies defined in the configuration, the package init carries hardcoded dependencies on Sentry and 1Password in order to function. Unless you want these and are effectively extending my base project, you're likely better off forking this package and cutting out what you do not need.

Installation

This package is published to the Python Package index from time to time.

(back to top)

Usage

I have various projects that use this tool chain. For example, my Base Project which can be run stand-alone but also serves as my Docker base image from which other projects are derived.

(back to top)

Testing

Tests use pytest with coverage. Some tests exercise live services (1Password Connect Server or Service Account, AWS session setup) and expect the OP_CONNECT_HOST, OP_CONNECT_TOKEN, OP_VAULT and OP_SERVICE_ACCOUNT_TOKEN environment variables. For deterministic results regardless of the ambient environment, the suite pins TZ=UTC and APP_NAME=test itself (see conftest.py).

Quick run (all extras)

uv sync --all-extras
uv run pytest

Extras matrix

Optional dependencies are organized as extras (aws, creds, dto, monitoring, mq). Using a Hatch matrix, the full test suite also runs against every combination of these extras: 32 environments, each installing the package with only that combination's dependencies.

# list the 32 test environments
uv run hatch env show
# run the whole matrix (stops at the first failing environment)
uv run hatch run test:run
# run a single combination
uv run hatch run test.aws-no-creds-dto-no-monitoring-mq:run
# remove all cached test environments
uv run hatch env prune

Tests that require an unavailable optional dependency skip themselves via pytest.importorskip, so every combination runs as much of the suite as its installed extras allow. Per-environment coverage data is written to .coverage.<env-name>.

Note that a full matrix run executes the live 1Password tests many times over, and the 1Password service-account API rate limit is shared across them. An occasional RateLimitExceeded failure during a full run is transient; re-run the affected environment to confirm.

(back to top)

License

Distributed under the MIT License. See LICENSE for more information.

(back to top)

Acknowledgments

(back to top)

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.8.2)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for tailucas from gravatar.com tailucas

Requires

Python >=3.11

Provides Extra

aws creds dto monitoring mq

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tailucas_pylib-0.7.3.tar.gz (21.7 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

tailucas_pylib-0.7.3-py3-none-any.whl (27.6 kB view details)

Uploaded Python 3

File details

Details for the file tailucas_pylib-0.7.3.tar.gz.

File metadata

  • Download URL: tailucas_pylib-0.7.3.tar.gz
  • Upload date:
  • Size: 21.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.11.26 {"installer":{"name":"uv","version":"0.11.26","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for tailucas_pylib-0.7.3.tar.gz
Algorithm Hash digest
SHA256 c0bece17b69c69c4efa2277f2410932923a081a4bb32132e4b1b3923470533b1
MD5 ec7e9221c814cb8a4227f9276a6e0a20
BLAKE2b-256 371731c2c2b77ab6d5e5a7e56d98e1af0fcc49acbdb03f8d2f887085b98f4eab

See more details on using hashes here.

File details

Details for the file tailucas_pylib-0.7.3-py3-none-any.whl.

File metadata

  • Download URL: tailucas_pylib-0.7.3-py3-none-any.whl
  • Upload date:
  • Size: 27.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.11.26 {"installer":{"name":"uv","version":"0.11.26","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for tailucas_pylib-0.7.3-py3-none-any.whl
Algorithm Hash digest
SHA256 5090927c07e51983a800c74f4331b45add8c3725816d382c210e12f1c1f64ed9
MD5 799b8455543c981546a2e9fea9739ac6
BLAKE2b-256 fcc7f23b5fb600b285c628aec65252e37268f025fa9e305ed46f86f7fc23b9b8

See more details on using hashes here.

Release history Release notifications | RSS feed

0.8.2

2 files

0.8.1

2 files

0.8.0

2 files

0.7.8

2 files

0.7.7

2 files

0.7.6

2 files

0.7.5

2 files

0.7.4

2 files

This release

0.7.3 This release

2 files

0.7.2

2 files

0.7.1

2 files

0.7.0

2 files

0.6.6

2 files

0.6.5

2 files

0.6.4

2 files

0.6.3

2 files

0.6.2

2 files

0.6.1

2 files

0.6.0

2 files

0.5.9

2 files

0.5.8

2 files

0.5.7

2 files

0.5.6

2 files

0.5.5

2 files

0.5.4

2 files

0.5.3

2 files

0.5.2

2 files

0.5.1

2 files

0.5.0

2 files

0.4.9

2 files

0.4.8

2 files

0.4.7

2 files

0.4.6

2 files

0.4.5

2 files

0.4.4

2 files

0.4.3

2 files

0.4.2

2 files

0.4.1

2 files

0.4.0

2 files

0.3.9

2 files

0.3.8

2 files

0.3.7

2 files

0.3.6

2 files

0.3.5

2 files

0.3.4

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1

2 files

0.3.0

2 files

0.2.9

2 files

0.2.8

2 files

0.2.7

2 files

0.2.6

2 files

0.2.5

2 files

0.2.4

2 files

0.2.3

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.9

2 files

0.1.8

2 files

0.1.7

2 files

0.1.6

2 files

0.1.5

2 files

0.1.4

2 files

0.1.3

2 files

0.1.2

2 files

0.1.1

2 files

0.0.9

2 files

0.0.8

2 files

0.0.7

2 files

0.0.6

2 files

0.0.5

2 files

0.0.4

2 files

0.0.3

2 files

0.0.2

2 files

0.0.1

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page