utilsds

Utilsds is a library that includes classes and functions used in data science projects such as:

  • algorithm:

    • Algorithm: Base class for fitting, training, and getting hyperparameters of machine learning models.

  • data_ops:

    • DataOperations: Handle data operations locally and with Google Cloud services (BigQuery and Cloud Storage).
    • BigQuery operations:
      • load_bq_data: Load data from tables, views, and SQL files.
      • save_bq_view, save_bq_table: Save views and tables.
      • load_bq_procedure: Execute stored procedures.
      • load_bq_details: Get table/view details and schema.
      • delete_bq_data: Delete data with safety confirmations.
      • dry_run: Perform dry runs to estimate query costs.
    • Cloud Storage operations:
      • save_gcs_bucket: Create buckets.
      • load_gcs_details: List blobs in a bucket.
      • save_gcs_file, load_gcs_file: Save and load files (.pkl, .json, .csv, .html, .sql, .parquet).
      • load_gcs_file_metadata: Load GCS blob metadata.
    • Local file operations:
      • save_local_file, load_local_file: Save and load files (.pkl, .json, .csv, .html, .sql, .parquet).
    • Utilities:
      • format_bytes: Format byte size as a human-readable string.

  • data_processing:

    • SkewnessTransformer: Transform skewed data using various methods (IHS, neglog, Yeo-Johnson, quantile).
    • NullReplacer: Replace null values in specified columns with configurable strategies.
    • ColumnDropper: Drop specified columns from a DataFrame.
    • OutliersCleaner: Clean outliers by clipping values outside specified percentile ranges.
    • CategoricalMapper: Map values in categorical columns according to a specified mapping scheme.
    • NumericalMapper: Convert numerical columns to categorical by binning.
    • Encoder: One-hot encode categorical columns in the data.
    • Normalizer: Normalize numerical columns using a provided scaler.
    • DuckDBColumnSQLTransformer: Transform a column using a SQL expression (DuckDB).
    • LabelEncoderTransformer: Label-encode categorical columns (e.g. for LightGBM).

  • data_split:

    • train_test_validation_split: Split data into training, testing, and validation sets.
    • resample_X_y: Resample train data and target column.
    • get_train_test_val_periods: Extract chronological train/test/val periods from a date column.

  • ds_statistics:

    • test_kruskal_wallis: Perform the Kruskal-Wallis statistical test.
    • test_agosto_pearsona: Test for normality using D'Agostino-Pearson test.

  • evaluate:

    • ModelEvaluator: Evaluate models and generate plots for diagnostics.
    • ShapExplainer: Explain model predictions using SHAP values.

  • experiments:

    • VertexExperiment: Manage experiments with Vertex AI.

  • vertex_model_registry:

    • ModelVersionRef: Immutable reference to a resolved model version in Vertex AI Model Registry (alias, GCS paths, Vertex registry version).
      • version_id: GCS folder version id (last segment of the artifact prefix).
      • blob_path: Build blob path inside the bucket.
      • from_artifact_uri: Build reference from a Vertex artifact_uri.
    • VertexModelRegistry: Register and resolve model bundles in Vertex AI Model Registry (GCS + aliases).
      • new_version_id, version_alias: Generate version folder id and immutable audit alias.
      • artifact_uri, gcs_prefix: Build GCS paths for a version.
      • resolve_parent_model: Return parent model resource name for subsequent registrations.
      • build_metadata: Build metadata.json content for a model bundle.
      • upload_model: Upload a model bundle to Model Registry.
      • register_model_bundle: Save artifacts to GCS and register them in Model Registry.
      • get_model_by_alias: Resolve a model version by alias.
      • load_artifact, load_model_artifacts: Load artifacts from GCS for a model alias.
      • promote_alias: Assign target_alias to the version referenced by source_alias.
    • resolve_serving_container_image_uri: Build serving container image URI from COMPONENT_BASE_IMAGE (replace image tag only).

  • optuna:

    • Optuna: Optimize hyperparameters using Optuna.

  • metrics:

    • Metrics: Calculate metrics for both classification and regression models.

  • modeling:

    • Modeling: Manage modeling, metrics, and logging with Vertex AI.

  • Supervised:

    • LazyClassifier: A classifier that automatically trains and evaluates multiple models.
    • LazyRegressor: A regressor that automatically trains and evaluates multiple models.
    • get_card_split: Function to split data into card-like groups.
    • adjusted_rsquared: Calculate adjusted R-squared for regression models.

  • visualization:

    • MetricsPlot: Compare metrics for different parameter values.
    • Radar: Create radar plots for visualizing data.
    • cluster_characteristics: Analyze cluster characteristics.
    • comparison_density: Compare density distributions.
    • elbow_visualisation: Visualize the elbow method for clustering.
    • describe_clusters_metrics: Describe metrics for clusters.
    • category_null_variables: Visualize null variables in categorical data.
    • normal_distr_plots: Visualize normal distribution plots.
    • distplot_limitations: Visualize limitations of distplot.
    • boxplot_limitations: Visualize limitations of boxplot.
    • violinplot_limitations: Visualize limitations of violinplot.
    • countplot_limitations: Visualize limitations of countplot.
    • categorical_variable_perc: Visualize percentage of categorical variables.
    • spearman_correlation: Visualize spearman correlation.
    • calculate_crammers_v: Calculate Crammer's V.

  • what_if_streamlit:

    • centroid_softmax_proba: Compute class probabilities from centroid distances (softmax).
    • get_predict_proba_fn: Return a predict-proba function for sklearn or centroid-softmax mode.
    • ShapSaver: Save SHAP explainer components for lazy loading in what-if analysis.
    • ColumnMetadataGenerator: Generate column metadata from a DataFrame or CSV file.

  • monitoring:

    • mapping: Create column mapping from configuration file for Evidently.
    • test_data: Test data for issues using Evidently test suites.
    • check_data_drift: Check data for drift using Evidently metrics.
    • send_email_with_table: Send email notifications with HTML tables for monitoring alerts.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Latest release

4 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for mwoj from gravatar.com mwoj Avatar for PatrykK from gravatar.com PatrykK Avatar for Piotter from gravatar.com Piotter
+1 more Avatar for Veatec from gravatar.com Veatec

Credits

Author: DS Team

License

MIT License (MIT License)

Requires

Python >=3.12

Provides Extra

dev

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

utilsds-2.0.19.tar.gz (57.5 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

utilsds-2.0.19-py3-none-any.whl (60.8 kB view details)

Uploaded Python 3

File details

Details for the file utilsds-2.0.19.tar.gz.

File metadata

  • Download URL: utilsds-2.0.19.tar.gz
  • Upload date:
  • Size: 57.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.13

File hashes

Hashes for utilsds-2.0.19.tar.gz
Algorithm Hash digest
SHA256 617ed64fad5575166922be6c637e9cf76081fcce49600c7832453118817fe821
MD5 d8baa6f64143dacb0ce5d205c2477327
BLAKE2b-256 c3cfc29fb21c31cc385a6cdcea871b03a6b5103e2bb423fcb457b9ebf11b06e0

See more details on using hashes here.

File details

Details for the file utilsds-2.0.19-py3-none-any.whl.

File metadata

  • Download URL: utilsds-2.0.19-py3-none-any.whl
  • Upload date:
  • Size: 60.8 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.13

File hashes

Hashes for utilsds-2.0.19-py3-none-any.whl
Algorithm Hash digest
SHA256 e9f0ba4304ecf4572babdd133db98b27da6621bf96a2387b3de1fb9536373024
MD5 f17f646e21f9551600fb1f08719a3930
BLAKE2b-256 14cb6ec60b774fc280111b5a0361eff3081bbf041ea3c33aaedc517f2dd06513

See more details on using hashes here.

Release history Release notifications | RSS feed

This release

2.0.19 This release

2 files

2.0.18

2 files

2.0.17

2 files

2.0.16

2 files

2.0.15

2 files

2.0.14

2 files

2.0.13

2 files

2.0.12

2 files

2.0.11

2 files

2.0.10

2 files

2.0.9

2 files

2.0.7

2 files

2.0.6

2 files

2.0.5

2 files

2.0.4

2 files

2.0.3

2 files

2.0.2

2 files

2.0.1

2 files

2.0.0

2 files

1.1.16

2 files

1.1.15

2 files

1.1.14

2 files

1.1.13

2 files

1.1.12

2 files

1.1.11

2 files

1.1.10

2 files

1.1.9

2 files

1.1.8

2 files

1.1.7

2 files

1.1.6

2 files

1.1.5

2 files

1.1.4

2 files

1.1.3

2 files

1.1.2

2 files

1.1.1

2 files

1.1.0

2 files

1.0.14

2 files

1.0.13

2 files

1.0.12

2 files

1.0.11

2 files

1.0.10

2 files

1.0.9

2 files

1.0.8

2 files

1.0.7

2 files

1.0.6

2 files

1.0.5

2 files

1.0.4

2 files

1.0.3

2 files

1.0.2

2 files

1.0.1

2 files

1.0.0

2 files

0.3.5

2 files

0.3.4

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1

2 files

0.3

2 files

0.2

2 files

0.1

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page