Contributors Forks Stargazers Issues MIT License

About The Project

Overview

This package was created by factoring out many reusable code artifacts from my various projects over a number of years. Since this work was not a part of a group effort, the test coverage is predictably abysmal :raised_eyebrow: and Python documentation notably absent :expressionless:. For each of my projects, which derive from a common Docker application found here, this package provides some specific and relatively powerful features to enable rapid offshoots for new ideas.

The package is organized under src/tailucas_pylib/ with the following key modules:

  • Core Modules:

    • __init__.py: Application bootstrap with logging, locale, configuration (via app.conf), and device name setup. Logs to syslog (UDP) or stdout/stderr. Optionally changes working directory to WORK_DIR.
    • creds.py: 1Password credential management supporting both 1Password Connect Server and 1Password Service Account modes. Fetches secrets from environment variables or container secrets (/run/secrets).
    • flags.py: Feature flag checking backed by a 1Password credential item.

  • Application Framework:

    • app.py: Thread base class with ZMQ relay (ZmqRelay) and worker (ZmqWorker) patterns for inter-thread communication via ZeroMQ.
    • threads.py: Thread nanny with shutdown tracking, graceful termination, Cronitor monitoring integration, and lingering socket cleanup.
    • handler.py: Context manager (exception_handler) that handles ZMQ connectivity, ContextTerminated, ResourceWarning, and general exceptions with optional Sentry reporting and graceful shutdown.
    • process.py: Signal handler (SignalHandler) with subprocess execution (exec_cmd) helpers.

  • Communication:

    • rabbit.py: RabbitMQ integration with MQConnection, ZMQListener (consumes RabbitMQ messages and forwards to ZMQ), and RabbitMQRelay (bridges ZMQ to RabbitMQ).
    • zmq.py: ZeroMQ socket creation, lifecycle management (Closable), and graceful context teardown.

  • Utilities:

    • data.py: Builds MessagePack payloads with timestamp and optional data for IPC.
    • datetime.py: Timezone-aware timestamp creation, ISO formatting, and Unix timestamp conversion.
    • device.py: Pydantic data model (Device) for device state with optional fields.
    • bluetooth.py: Bluetooth adaptor detection and device ping via hcitool / l2ping.

  • AWS Integration (aws/):

    • __init__.py: Manages cached Boto3 sessions with STS role assumption, credential retrieval from 1Password.
    • metrics.py: Posts CloudWatch count metrics with dimensions.

  • Command-line Tools (tools/):

    • config_interpol.py: Interpolates a config file with environment variables or an overlay config, outputting the resolved values.
    • cred_tool.py: Fetches and displays a 1Password credential read from stdin.
    • yaml_interpol.py: Processes YAML templates, substituting values from configuration sections.
    • aws_configure.py: Generates AWS CLI configuration files from 1Password-stored credentials.

(back to top)

Built With

Technologies that help make this package useful:

1Password Amazon AWS uv Python RabbitMQ Sentry ZeroMQ

Also:

GitHub

Core Technologies:

  • Python 3.11+ - Primary runtime
  • 1Password Connect/Service Account - Credential management
  • ZeroMQ - Inter-process communication
  • RabbitMQ - Message queuing
  • MessagePack - Binary serialization
  • Pydantic - Data validation and modeling
  • Sentry - Error tracking and monitoring

AWS Integration:

  • Boto3 - AWS SDK
  • CloudWatch - Metrics and monitoring
  • Simple Workflow Service - Workflow orchestration (deprecated)

Development Tools:

  • uv - Dependency management
  • Cronitor - Cron job monitoring

(back to top)

Getting Started

Here is some detail about the intended use of this package.

Prerequisites

A Python project or runtime environment. Since this project is already initialized with [uv][vu-url] dependency management, I recommend that you continue to use it. Beyond the Python dependencies defined in the configuration, the package init carries hardcoded dependencies on Sentry and 1Password in order to function. Unless you want these and are effectively extending my base project, you're likely better off forking this package and cutting out what you do not need.

Installation

This package is published to the Python Package index from time to time.

(back to top)

Usage

I have various projects that use this tool chain. For example, my Base Project which can be run stand-alone but also serves as my Docker base image from which other projects are derived.

(back to top)

Testing

Tests use pytest with coverage. Some tests exercise live services (1Password Connect Server or Service Account, AWS session setup) and expect the OP_CONNECT_HOST, OP_CONNECT_TOKEN, OP_VAULT and OP_SERVICE_ACCOUNT_TOKEN environment variables. For deterministic results regardless of the ambient environment, the suite pins TZ=UTC and APP_NAME=test itself (see conftest.py).

Quick run (all extras)

uv sync --all-extras
uv run pytest

Extras matrix

Optional dependencies are organized as extras (aws, creds, dto, monitoring, mq). Using a Hatch matrix, the full test suite also runs against every combination of these extras: 32 environments, each installing the package with only that combination's dependencies.

# list the 32 test environments
uv run hatch env show
# run the whole matrix (stops at the first failing environment)
uv run hatch run test:run
# run a single combination
uv run hatch run test.aws-no-creds-dto-no-monitoring-mq:run
# remove all cached test environments
uv run hatch env prune

Tests that require an unavailable optional dependency skip themselves via pytest.importorskip, so every combination runs as much of the suite as its installed extras allow. Per-environment coverage data is written to .coverage.<env-name>.

Note that a full matrix run executes the live 1Password tests many times over, and the 1Password service-account API rate limit is shared across them. An occasional RateLimitExceeded failure during a full run is transient; re-run the affected environment to confirm.

(back to top)

License

Distributed under the MIT License. See LICENSE for more information.

(back to top)

Acknowledgments

(back to top)

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (0.8.2)

1 maintainer

PyPI data
Data sourced directly from PyPI's database.
Avatar for tailucas from gravatar.com tailucas

Requires

Python >=3.11

Provides Extra

aws creds dto monitoring mq

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tailucas_pylib-0.7.5.tar.gz (21.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

tailucas_pylib-0.7.5-py3-none-any.whl (27.4 kB view details)

Uploaded Python 3

File details

Details for the file tailucas_pylib-0.7.5.tar.gz.

File metadata

  • Download URL: tailucas_pylib-0.7.5.tar.gz
  • Upload date:
  • Size: 21.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.12.1 {"installer":{"name":"uv","version":"0.12.1","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for tailucas_pylib-0.7.5.tar.gz
Algorithm Hash digest
SHA256 e18e06768f9ed1eecf1a0efa1dc417712455ca36b9a9a8e1c9e0817dd877b38b
MD5 cfb627cd8090d2478b39f0c21f45991d
BLAKE2b-256 bc7087b4be2d62e1fc183f7b463925a420de0de782e12ffd67f7f6b0b18865c8

See more details on using hashes here.

File details

Details for the file tailucas_pylib-0.7.5-py3-none-any.whl.

File metadata

  • Download URL: tailucas_pylib-0.7.5-py3-none-any.whl
  • Upload date:
  • Size: 27.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: uv/0.12.1 {"installer":{"name":"uv","version":"0.12.1","subcommand":["publish"]},"python":null,"implementation":{"name":null,"version":null},"distro":{"name":"Ubuntu","version":"26.04","id":"resolute","libc":null},"system":{"name":null,"release":null},"cpu":null,"openssl_version":null,"setuptools_version":null,"rustc_version":null,"ci":null}

File hashes

Hashes for tailucas_pylib-0.7.5-py3-none-any.whl
Algorithm Hash digest
SHA256 a6e66c2db865dbdf716154eaf4b5dbfc23273312a270b48a5e2205c8f554cef9
MD5 1d7ff60c51e9654d1d497a73fb1baf1d
BLAKE2b-256 7f2c0c0e0bab35ae9c63ea3b92331f05295372bc7017a9762441298fdc9384e4

See more details on using hashes here.

Release history Release notifications | RSS feed

0.8.2

2 files

0.8.1

2 files

0.8.0

2 files

0.7.8

2 files

0.7.7

2 files

0.7.6

2 files

This release

0.7.5 This release

2 files

0.7.4

2 files

0.7.3

2 files

0.7.2

2 files

0.7.1

2 files

0.7.0

2 files

0.6.6

2 files

0.6.5

2 files

0.6.4

2 files

0.6.3

2 files

0.6.2

2 files

0.6.1

2 files

0.6.0

2 files

0.5.9

2 files

0.5.8

2 files

0.5.7

2 files

0.5.6

2 files

0.5.5

2 files

0.5.4

2 files

0.5.3

2 files

0.5.2

2 files

0.5.1

2 files

0.5.0

2 files

0.4.9

2 files

0.4.8

2 files

0.4.7

2 files

0.4.6

2 files

0.4.5

2 files

0.4.4

2 files

0.4.3

2 files

0.4.2

2 files

0.4.1

2 files

0.4.0

2 files

0.3.9

2 files

0.3.8

2 files

0.3.7

2 files

0.3.6

2 files

0.3.5

2 files

0.3.4

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1

2 files

0.3.0

2 files

0.2.9

2 files

0.2.8

2 files

0.2.7

2 files

0.2.6

2 files

0.2.5

2 files

0.2.4

2 files

0.2.3

2 files

0.2.2

2 files

0.2.1

2 files

0.2.0

2 files

0.1.9

2 files

0.1.8

2 files

0.1.7

2 files

0.1.6

2 files

0.1.5

2 files

0.1.4

2 files

0.1.3

2 files

0.1.2

2 files

0.1.1

2 files

0.0.9

2 files

0.0.8

2 files

0.0.7

2 files

0.0.6

2 files

0.0.5

2 files

0.0.4

2 files

0.0.3

2 files

0.0.2

2 files

0.0.1

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page