Skip to main content

data-ml-utils

A utility python package that covers the common libraries we use

Installation

This is an open source library hosted on pypi. Run the following command to install the library.

pip install data-ml-utils --upgrade

Documentation

Head over to https://data-ml-utils.readthedocs.io/en/latest/index.html# to read our library documentation

Feature

Pyathena client initialisation

Almost one liner

import os
from data_ml_utils.pyathena_client.client import PyAthenaClient

os.environ["AWS_ACCESS_KEY_ID"] = "xxx"
os.environ["AWS_SECRET_ACCESS_KEY"] = "xxx" # pragma: allowlist secret
os.environ["S3_BUCKET"] = "xxx"

pyathena_client = PyAthenaClient()

Pyathena client initialisation

Pyathena query

Almost one liner

query = """
    SELECT
        *
    FROM
        dev.example_pyathena_client_table
    LIMIT 10
"""

df_raw = pyathena_client.query_as_pandas(final_query=query)

Pyathena query

MLflow utils

Visit link

More to Come

  • You suggest, raise a feature request issue and we will review!

Tutorials

Pyathena

There is a jupyter notebook to show how to use the package utility package for pyathena: notebook

MLflow utils

There is a jupyter notebook to show how to use the package utility package for mlflow_databricks: notebook

Metadata

Release files for sm_data_ml_utils 1.0.11

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sm_data_ml_utils 1.0.11
File Size Uploaded
sm_data_ml_utils-1.0.11.tar.gz 15.5 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for sm_data_ml_utils 1.0.11
File Interpreter ABI Platform
sm_data_ml_utils-1.0.11-py3-none-any.whl Python 3 none any Details

Total release size: 33.2 kB

Release files / sm_data_ml_utils-1.0.11.tar.gz

Download URL sm_data_ml_utils-1.0.11.tar.gz
Size 15.5 kB
Tags Source
SHA-256 checksum
How to use checksums
cc82241e608f52ea5155f80cda8077726cee7203d8ae5616fb4839ebb5150602
BLAKE2b-256 checksum
How to use checksums
c4e4c261604e161f3bdbe8d29e9a2c4a30a69a0132aa20a41ca6f601ec831846
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/1.8.3 CPython/3.13.5 Darwin/25.5.0

Release files / sm_data_ml_utils-1.0.11-py3-none-any.whl

Download URL sm_data_ml_utils-1.0.11-py3-none-any.whl
Size 17.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
88027e232b719e25cc4c8593c6053ec4dc7cc7c5651aea87b8e459faed4893d4
BLAKE2b-256 checksum
How to use checksums
dd86fb8033a458f78f2afbce4fb6e443b3bbbd2b09c53d963d1f945b1a820bdc
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/1.8.3 CPython/3.13.5 Darwin/25.5.0

Release history Release notifications | RSS feed

This release

1.0.11 This release

2 release files

1.0.10

2 release files

1.0.9

2 release files

1.0.8

2 release files

1.0.7

2 release files

1.0.6

2 release files

1.0.5

2 release files

1.0.4

2 release files

1.0.3

2 release files

1.0.2

2 release files

1.0.1

2 release files

1.0.0

2 release files

0.1.3

2 release files

0.1.2

2 release files

0.1.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page