No project description provided
Project description
mlplatformutils
mlplatformutils for observability
Structure
.
|-- LICENSE.txt
|-- README.rst
|-- setup.cfg
|-- setup.py
|-- src
| |-- mlplatformutils
| | |-- __init__.py
| | |-- core
| | |-- |-- __init__.py
| | |-- |-- sparkutils.py
| | |-- |-- platformutils.py
| | |-- |-- pandasutils.py
| | |-- |-- lineagegraph.py
| | |-- |-- app_insights_logger.py
|-- tests
| |-- __init__.py
| |-- core
| |-- |--__init__.py
| |-- |-- sparkutils.py
| |-- |-- platformutils.py
| |-- |-- pandasutils.py
| |-- |-- lineagegraph.py
| |-- |-- app_insights_logger.py
Instructions
install twine - twine is a utility package that is used for publishing Python packages on PyPI python -m pip install twine Build Package - create the source distribution of the package python setup.py sdist Upload Package to PyPI python -m twine upload dist/*
Description
app_insights_logger - Constains telemetrylogger Class with Functions to Manage and Log Telemetry into Azure Application Insights lineagegraph - Contains LineageGraph Class with functions to manage Graph on Azure Cosmos DB enabled with Gremlin
platformutils - Contains platform utility functions to check, install depedencies, check Azure ML Compute
- is_package_installed
- install_pip
- get_environment
- set_environment
- assert_amlcompute
- read_setup_ini
sparkutils - Contains functions to read data from sources such as (Azure Data Lake Gen2, Azure Data Explorer (Kusto), Azure Sql Server) and write (Azure Data Lake Gen2)while ensuring integrated Lineage Graph Logging.
- read_from_adls_gen2
- write_to_adls_gen2
- read_from_kusto
- read_from_azsql
pandasutils - Contains functions to read data from Azure Data Lake Gen2 (from Delta Format or Parquet Format) into Pandas Dataframe without Spark while ensuring integrated Lineage Graph Logging.
- read_from_delta_as_pandas
- read_from_parquet_as_pandas
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.