utilsds

Utilsds is a library that includes classes and functions used in data science projects such as:

  • algorithm:

    • Algorithm: Base class for fitting, training, and getting hyperparameters of machine learning models.

  • data_ops:

    • DataOperations: Handle data operations locally and with Google Cloud services (BigQuery and Cloud Storage).
    • BigQuery operations:
      • load_bq_data: Load data from tables, views, and SQL files.
      • save_bq_view, save_bq_table: Save views and tables.
      • load_bq_procedure: Execute stored procedures.
      • load_bq_details: Get table/view details and schema.
      • delete_bq_data: Delete data with safety confirmations.
      • dry_run: Perform dry runs to estimate query costs.
    • Cloud Storage operations:
      • save_gcs_bucket: Create buckets.
      • save_gcs_file, load_gcs_file: Save and load files (.pkl, .json, .csv, .html, .sql).
    • Local file operations:
      • save_local_file, load_local_file: Save and load files (.pkl, .json, .csv, .html, .sql).

  • data_processing:

    • SkewnessTransformer: Transform skewed data using various methods (IHS, neglog, Yeo-Johnson, quantile).
    • NullReplacer: Replace null values in specified columns with configurable strategies.
    • ColumnDropper: Drop specified columns from a DataFrame.
    • OutliersCleaner: Clean outliers by clipping values outside specified percentile ranges.
    • CategoricalMapper: Map values in categorical columns according to a specified mapping scheme.
    • NumericalMapper: Convert numerical columns to categorical by binning.
    • Encoder: One-hot encode categorical columns in the data.
    • Normalizer: Normalize numerical columns using a provided scaler.

  • data_split:

    • train_test_validation_split: Split data into training, testing, and validation sets.
    • resample_X_y: resample train data and target column.

  • ds_statistics:

    • test_kruskal_wallis: Perform the Kruskal-Wallis statistical test.
    • test_agosto_pearsona: Test for normality using D'Agostino-Pearson test.

  • evaluate:

    • ModelEvaluator: Evaluate models and generate plots for diagnostics.
    • ShapExplainer: Explain model predictions using SHAP values.

  • experiments:

    • VertexExperiment: Manage experiments with Vertex AI.

  • optuna:

    • Optuna: Optimize hyperparameters using Optuna.

  • metrics:

    • Metrics: Calculate metrics for both classification and regression models.

  • modeling:

    • Modeling: Manage modeling, metrics, and logging with Vertex AI.

  • Supervised:

    • LazyClassifier: A classifier that automatically trains and evaluates multiple models.
    • LazyRegressor: A regressor that automatically trains and evaluates multiple models.
    • get_card_split: Function to split data into card-like groups.
    • adjusted_rsquared: Calculate adjusted R-squared for regression models.

  • visualization:

    • MetricsPlot: Compare metrics for different parameter values.
    • Radar: Create radar plots for visualizing data.
    • cluster_characteristics: Analyze cluster characteristics.
    • comparison_density: Compare density distributions.
    • elbow_visualisation: Visualize the elbow method for clustering.
    • describe_clusters_metrics: Describe metrics for clusters.
    • category_null_variables: Visualize null variables in categorical data.
    • normal_distr_plots: Visualize normal distribution plots.
    • distplot_limitations: Visualize limitations of distplot.
    • boxplot_limitations: Visualize limitations of boxplot.
    • violinplot_limitations: Visualize limitations of violinplot.
    • countplot_limitations: Visualize limitations of countplot.
    • categorical_variable_perc: Visualize percentage of categorical variables.
    • spearman_correlation: Visualize spearman correlation.
    • calculate_crammers_v: Calculate Crammer's V.

  • what_if_streamlit:

    • ShapSaver: Save SHAP explainer components for lazy loading in what-if analysis.
    • ColumnMetadataGenerator: Generate column metadata from a DataFrame or CSV file.

  • monitoring:

    • mapping: Create column mapping from configuration file for Evidently.
    • test_data: Test data for issues using Evidently test suites.
    • check_data_drift: Check data for drift using Evidently metrics.
    • send_email_with_table: Send email notifications with HTML tables for monitoring alerts.

Key dates

PyPI data
Data sourced directly from PyPI's database.
  • Released:
Newer release available (2.0.19)

4 maintainers

PyPI data
Data sourced directly from PyPI's database.
Avatar for mwoj from gravatar.com mwoj Avatar for PatrykK from gravatar.com PatrykK Avatar for Piotter from gravatar.com Piotter
+1 more Avatar for Veatec from gravatar.com Veatec

Credits

Author: DS Team

License

MIT License (MIT License)

Requires

Python >=3.12

Provides Extra

dev

Classifiers

Report project as malware

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

utilsds-2.0.18.tar.gz (56.0 kB view details)

Uploaded Source

Built Distribution

If you're not sure about the file name format, learn more about wheel file names.

utilsds-2.0.18-py3-none-any.whl (60.0 kB view details)

Uploaded Python 3

File details

Details for the file utilsds-2.0.18.tar.gz.

File metadata

  • Download URL: utilsds-2.0.18.tar.gz
  • Upload date:
  • Size: 56.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.13

File hashes

Hashes for utilsds-2.0.18.tar.gz
Algorithm Hash digest
SHA256 fe877275738d231785eaeba5d16075c69850e57dc87333316f47046c2c212757
MD5 c17537f0e722dbf51ad50e386600441d
BLAKE2b-256 dbab132b9c7cfc6575467c530431c096de5ec05fc0d5c3df14558170e80f66d6

See more details on using hashes here.

File details

Details for the file utilsds-2.0.18-py3-none-any.whl.

File metadata

  • Download URL: utilsds-2.0.18-py3-none-any.whl
  • Upload date:
  • Size: 60.0 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/6.2.0 CPython/3.12.13

File hashes

Hashes for utilsds-2.0.18-py3-none-any.whl
Algorithm Hash digest
SHA256 a1edbf73de8cc10a4482b18838c13aab861fa7bbf0a8f7394b4769642c0394cf
MD5 feb249c8ca3ee45c120fb13bb54e1f4b
BLAKE2b-256 77bfe73a7b946d3a8ae929750a5e1bcddf2926946a526351d14450f74dbfa1e4

See more details on using hashes here.

Release history Release notifications | RSS feed

2.0.19

2 files

This release

2.0.18 This release

2 files

2.0.17

2 files

2.0.16

2 files

2.0.15

2 files

2.0.14

2 files

2.0.13

2 files

2.0.12

2 files

2.0.11

2 files

2.0.10

2 files

2.0.9

2 files

2.0.7

2 files

2.0.6

2 files

2.0.5

2 files

2.0.4

2 files

2.0.3

2 files

2.0.2

2 files

2.0.1

2 files

2.0.0

2 files

1.1.16

2 files

1.1.15

2 files

1.1.14

2 files

1.1.13

2 files

1.1.12

2 files

1.1.11

2 files

1.1.10

2 files

1.1.9

2 files

1.1.8

2 files

1.1.7

2 files

1.1.6

2 files

1.1.5

2 files

1.1.4

2 files

1.1.3

2 files

1.1.2

2 files

1.1.1

2 files

1.1.0

2 files

1.0.14

2 files

1.0.13

2 files

1.0.12

2 files

1.0.11

2 files

1.0.10

2 files

1.0.9

2 files

1.0.8

2 files

1.0.7

2 files

1.0.6

2 files

1.0.5

2 files

1.0.4

2 files

1.0.3

2 files

1.0.2

2 files

1.0.1

2 files

1.0.0

2 files

0.3.5

2 files

0.3.4

2 files

0.3.3

2 files

0.3.2

2 files

0.3.1

2 files

0.3

2 files

0.2

2 files

0.1

2 files

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page