Skip to main content

data-ml-utils

A utility python package that covers the common libraries we use

Installation

This is an open source library hosted on pypi. Run the following command to install the library.

pip install data-ml-utils --upgrade

Documentation

Head over to https://data-ml-utils.readthedocs.io/en/latest/index.html# to read our library documentation

Feature

Pyathena client initialisation

Almost one liner

import os
from data_ml_utils.pyathena_client.client import PyAthenaClient

os.environ["AWS_ACCESS_KEY_ID"] = "xxx"
os.environ["AWS_SECRET_ACCESS_KEY"] = "xxx" # pragma: allowlist secret
os.environ["S3_BUCKET"] = "xxx"

pyathena_client = PyAthenaClient()

Pyathena client initialisation

Pyathena query

Almost one liner

query = """
    SELECT
        *
    FROM
        dev.example_pyathena_client_table
    LIMIT 10
"""

df_raw = pyathena_client.query_as_pandas(final_query=query)

Pyathena query

MLflow utils

Visit link

More to Come

  • You suggest, raise a feature request issue and we will review!

Tutorials

Pyathena

There is a jupyter notebook to show how to use the package utility package for pyathena: notebook

MLflow utils

There is a jupyter notebook to show how to use the package utility package for mlflow_databricks: notebook

Metadata

Release files for sm_data_ml_utils 1.0.10

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for sm_data_ml_utils 1.0.10
File Size Uploaded
sm_data_ml_utils-1.0.10.tar.gz 15.5 kB Details

Built distribution (wheel)

Table of built distributions (wheels) for sm_data_ml_utils 1.0.10
File Interpreter ABI Platform
sm_data_ml_utils-1.0.10-py3-none-any.whl Python 3 none any Details

Total release size: 33.2 kB

Release files / sm_data_ml_utils-1.0.10.tar.gz

Download URL sm_data_ml_utils-1.0.10.tar.gz
Size 15.5 kB
Tags Source
SHA-256 checksum
How to use checksums
60c88256d78631fb459b99fc7ae3f134ff7ada1f8e13c15a709b84dce9b85d8a
BLAKE2b-256 checksum
How to use checksums
facd64c36f73dcd246b89b0d6c7b955cec413f5208226c20765d61f1d95250a4
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/1.8.3 CPython/3.13.5 Darwin/25.5.0

Release files / sm_data_ml_utils-1.0.10-py3-none-any.whl

Download URL sm_data_ml_utils-1.0.10-py3-none-any.whl
Size 17.7 kB
Tags Python 3
SHA-256 checksum
How to use checksums
77aca4ccf7fd0bc26225e2861c3ddb01bbf5e11844781e9108b5472c36c0d1a5
BLAKE2b-256 checksum
How to use checksums
32387b5b144dd852d08cd58849330e99ec4244e7c870ba0596335e33be6384e8
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via poetry/1.8.3 CPython/3.13.5 Darwin/25.5.0

Release history Release notifications | RSS feed

1.0.11

2 release files

This release

1.0.10 This release

2 release files

1.0.9

2 release files

1.0.8

2 release files

1.0.7

2 release files

1.0.6

2 release files

1.0.5

2 release files

1.0.4

2 release files

1.0.3

2 release files

1.0.2

2 release files

1.0.1

2 release files

1.0.0

2 release files

0.1.3

2 release files

0.1.2

2 release files

0.1.1

2 release files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page