jragbeer_common
Overview
A Python utility library containing common functions and tools used across various projects. This library provides reusable components for:
- Data processing and engineering
- Azure blob storage operations
- Dask distributed computing
- Ubuntu system operations
Features
-
Data Engineering Utilities
- DataFrame manipulation and cleaning
- SQL database operations
- Email notifications
- Date/time processing
- Logging configuration
-
Azure Integration
- Blob storage upload/download
- Parquet file handling
- Container management
- Batch operations
-
Dask Distributed Computing
- Cluster deployment and management
- Worker allocation
- Task scheduling
- Remote execution
-
Ubuntu System Operations
- Remote command execution
- Process management
- System monitoring
- File operations
Installation
- Install
uv(recommended):
pip install uv
- Create and activate a virtual environment:
uv venv
source .venv/bin/activate # Linux/Mac
# or
.venv\Scripts\activate # Windows
- Install the package:
uv pip install jragbeer-common
Development Setup
- Clone the repository:
git clone https://github.com/jragbeer/jragbeer_common.git
cd jragbeer_common
- Create a virtual environment and install dependencies:
uv venv
source .venv/bin/activate
uv pip install -r requirements.txt
- Install development dependencies:
uv pip install -e ".[dev]"
- Install pre-commit hooks:
pre-commit install
Usage
from jragbeer_common import (
jragbeer_common_data_eng,
jragbeer_common_azure,
jragbeer_common_dask,
jragbeer_common_ubuntu
)
# Data Engineering
jragbeer_common_data_eng.parse_date_features(df)
# Azure Operations
jragbeer_common_azure.adls_upload_file("path/to/file", "blob_name")
# Dask Operations
jragbeer_common_dask.deploy_dask_home_setup()
# Ubuntu Operations
jragbeer_common_ubuntu.execute_cmd_ubuntu_sudo("command")
Environment Variables
The following environment variables are required:
# Azure Storage
adls_connection_string="your_connection_string"
adls_container_name="your_container"
# Database
local_db_username="username"
local_db_password="password"
local_db_address="address"
local_db_port="port"
# Cluster Configuration
cluster_server_1_address="address"
cluster_server_1_username="username"
cluster_server_1_password="password"
Building and Distribution
- Build the package:
uv build
- Install locally for testing:
uv pip install dist/jragbeer_common-0.2.0-py3-none-any.whl
Contributing
- Fork the repository
- Create a feature branch
- Make your changes
- Run tests and linting:
uv sync --extra dev
pre-commit run --all-files
uv run pytest
CI runs the same full suite on each push and pull request (see
.github/workflows/test.yml), including tests
marked integration (Docker Postgres, etc.). You need a working Docker daemon
for a green run locally and in GitHub Actions (the runner provides Docker).
Markers (unit, integration, contract) still classify tests in
pyproject.toml; nothing is excluded by default.
Integration tests (Docker Postgres)
Database tests that open PostgreSQL live in
test/db/test_jragbeer_common_db.py and are
marked @pytest.mark.integration. They use the session-scoped container from
test/fixtures/postgresql.py plus the docker
Python client and psycopg2-binary. They run as part of the default
uv run pytest (and in CI).
Optional environment variables (defaults in parentheses):
TEST_POSTGRESQL_PORT— host port mapped to the container (54329)TEST_POSTGRESQL_USER,TEST_POSTGRESQL_PASSWORD,TEST_POSTGRESQL_DATABASE— bootstrap credentials (devuser/devpassword/postgres)TEST_POSTGRESQL_VERSION— image tag afterpostgres:(17.4)
Readiness is checked with TCP connections via psycopg2 from the host (same as SQLAlchemy), not docker exec psql, so you do not need the docker CLI or host postgresql-client for the wait loop.
Future work: tests that hit real services should live under the appropriate
test/<subpackage>/ package, be marked @pytest.mark.integration, and
document required environment variables in the test module docstring.
License
Copyright 2026 Julien Ragbeer
Licensed under the Apache License, Version 2.0
1 maintainer
Metadata
1 maintainer
Release files for jragbeer-common 0.2.68
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | Provenance | |
|---|---|---|---|---|
| jragbeer_common-0.2.68.tar.gz | 41.0 kB | Without provenance | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| jragbeer_common-0.2.68-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 82.2 kB
Release files / jragbeer_common-0.2.68.tar.gz
| Download URL | jragbeer_common-0.2.68.tar.gz |
|---|---|
| Size | 41.0 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
b38a54ba2e984062e885f091cb0762b83d7e50bc1fa1b5cb347e808c889bd5c2
|
|
BLAKE2b-256 checksum How to use checksums |
06160d3e42bbefb7352f7ffc791cf10fe1e60f4e904ee30f49df4b3136957ee2
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.10.17
|
Release files / jragbeer_common-0.2.68-py3-none-any.whl
| Download URL | jragbeer_common-0.2.68-py3-none-any.whl |
|---|---|
| Size | 41.1 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
c901859eefe51f21b1722a38c7b95565f2c2d28af97c067dec52b3f50638648a
|
|
BLAKE2b-256 checksum How to use checksums |
8c4e9601bb3ccf6a0fb28e062b52d1068f39a403c0ada96bab718a6b33d16ca1
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.1.0 CPython/3.10.17
|
Security for jragbeer-common 0.2.68
Provenance
Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.
No provenance found
None of the files in this release were uploaded with provenance metadata.
A note on package security
PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.
To learn more about protecting yourself from malicious actors, visit our package security documentation.
