A package containing Python utilities for use across NYPL
Project description
PythonUtils
This package contains common Python utility classes and functions.
Classes
- Pushing records to Kinesis
- Setting and retrieving a resource in S3
- Decrypting values with KMS
- Encoding and decoding records using a given Avro schema
- Connecting to and querying a MySQL database
- Connecting to and querying a PostgreSQL database using a connection pool
- Connecting to and querying Redshift
Functions
- Reading a YAML config file and putting the contents in os.environ
- Creating a logger in the appropriate format
- Obfuscating a value using bcrypt
Developing locally
In order to use the local version of the package instead of the global version, use a virtual environment. To set up a virtual environment and install all the necessary dependencies, run:
python3 -m venv testenv
source testenv/bin/activate
pip install --upgrade pip
pip install .
pip install '.[tests]'
deactivate && source testenv/bin/activate
Add any new dependencies required by code in the nypl_py_utils
directory to the dependencies
section of pyproject.toml
. Add dependencies only required by code in the tests
directory to the [project.optional-dependencies]
section.
Git workflow
This repo uses the Main-QA-Production git workflow.
main
has the latest and greatest commits, qa
has what's in our QA environment, and production
has what's in our production environment.
Ideal Workflow
- Cut a feature branch off of
main
- Commit changes to your feature branch
- File a pull request against
main
and assign a reviewer (who must be an owner)- In order for the PR to be accepted, it must pass all unit tests, have no lint issues, and update the CHANGELOG (or contain the
Skip-Changelog
label in GitHub)
- In order for the PR to be accepted, it must pass all unit tests, have no lint issues, and update the CHANGELOG (or contain the
- After the PR is accepted, merge into
main
- Merge
main
>qa
- Deploy app to QA on GitHub and confirm it works
- Merge
qa
>production
- Deploy app to production on GitHub and confirm it works
Deployment
The utils repo is deployed as a PyPI package here and as a Test PyPI package for QA purposes here. In order to be deployed, the version listed in pyproject.toml
must be updated. To deploy to Test PyPI, create a new release in GitHub and tag it qa-vX.X.X
. The GitHub Actions deploy-qa workflow will then build and publish the package. To deploy to production PyPI, create a release and tag it production-vX.X.X
.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for nypl_py_utils-0.0.5-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 668d5ad3cc3cc3ed3bbd50b84ed86269854213fad0caef442ac8f5f7964cbec5 |
|
MD5 | a7b2b181f43a69da1eb3cf07340eeb4e |
|
BLAKE2b-256 | 2ebb617b80d999bf2bb18cc90ada46ce5a6335f3d65dc315f92042480872530e |