utilsds
Utilsds is a library that includes classes and functions used in data science projects such as:
-
algorithm:
Algorithm: Base class for fitting, training, and getting hyperparameters of machine learning models.
-
data_ops:
DataOperations: Handle data operations locally and with Google Cloud services (BigQuery and Cloud Storage).- BigQuery operations:
load_bq_data: Load data from tables, views, and SQL files.save_bq_view,save_bq_table: Save views and tables.load_bq_procedure: Execute stored procedures.load_bq_details: Get table/view details and schema.delete_bq_data: Delete data with safety confirmations.dry_run: Perform dry runs to estimate query costs.
- Cloud Storage operations:
save_gcs_bucket: Create buckets.load_gcs_details: List blobs in a bucket.save_gcs_file,load_gcs_file: Save and load files (.pkl, .json, .csv, .html, .sql, .parquet).load_gcs_file_metadata: Load GCS blob metadata.
- Local file operations:
save_local_file,load_local_file: Save and load files (.pkl, .json, .csv, .html, .sql, .parquet).
- Utilities:
format_bytes: Format byte size as a human-readable string.
-
data_processing:
SkewnessTransformer: Transform skewed data using various methods (IHS, neglog, Yeo-Johnson, quantile).NullReplacer: Replace null values in specified columns with configurable strategies.ColumnDropper: Drop specified columns from a DataFrame.OutliersCleaner: Clean outliers by clipping values outside specified percentile ranges.CategoricalMapper: Map values in categorical columns according to a specified mapping scheme.NumericalMapper: Convert numerical columns to categorical by binning.Encoder: One-hot encode categorical columns in the data.Normalizer: Normalize numerical columns using a provided scaler.DuckDBColumnSQLTransformer: Transform a column using a SQL expression (DuckDB).LabelEncoderTransformer: Label-encode categorical columns (e.g. for LightGBM).
-
data_split:
train_test_validation_split: Split data into training, testing, and validation sets.resample_X_y: Resample train data and target column.get_train_test_val_periods: Extract chronological train/test/val periods from a date column.
-
ds_statistics:
test_kruskal_wallis: Perform the Kruskal-Wallis statistical test.test_agosto_pearsona: Test for normality using D'Agostino-Pearson test.
-
evaluate:
ModelEvaluator: Evaluate models and generate plots for diagnostics.ShapExplainer: Explain model predictions using SHAP values.
-
experiments:
VertexExperiment: Manage experiments with Vertex AI.
-
vertex_model_registry:
ModelVersionRef: Immutable reference to a resolved model version in Vertex AI Model Registry (alias, GCS paths, Vertex registry version).version_id: GCS folder version id (last segment of the artifact prefix).blob_path: Build blob path inside the bucket.from_artifact_uri: Build reference from a Vertexartifact_uri.
VertexModelRegistry: Register and resolve model bundles in Vertex AI Model Registry (GCS + aliases).new_version_id,version_alias: Generate version folder id and immutable audit alias.artifact_uri,gcs_prefix: Build GCS paths for a version.resolve_parent_model: Return parent model resource name for subsequent registrations.build_metadata: Buildmetadata.jsoncontent for a model bundle.upload_model: Upload a model bundle to Model Registry.register_model_bundle: Save artifacts to GCS and register them in Model Registry.get_model_by_alias: Resolve a model version by alias.load_artifact,load_model_artifacts: Load artifacts from GCS for a model alias.promote_alias: Assigntarget_aliasto the version referenced bysource_alias.
resolve_serving_container_image_uri: Build serving container image URI fromCOMPONENT_BASE_IMAGE(replace image tag only).
-
optuna:
Optuna: Optimize hyperparameters using Optuna.
-
metrics:
Metrics: Calculate metrics for both classification and regression models.
-
modeling:
Modeling: Manage modeling, metrics, and logging with Vertex AI.
-
Supervised:
LazyClassifier: A classifier that automatically trains and evaluates multiple models.LazyRegressor: A regressor that automatically trains and evaluates multiple models.get_card_split: Function to split data into card-like groups.adjusted_rsquared: Calculate adjusted R-squared for regression models.
-
visualization:
MetricsPlot: Compare metrics for different parameter values.Radar: Create radar plots for visualizing data.cluster_characteristics: Analyze cluster characteristics.comparison_density: Compare density distributions.elbow_visualisation: Visualize the elbow method for clustering.describe_clusters_metrics: Describe metrics for clusters.category_null_variables: Visualize null variables in categorical data.normal_distr_plots: Visualize normal distribution plots.distplot_limitations: Visualize limitations of distplot.boxplot_limitations: Visualize limitations of boxplot.violinplot_limitations: Visualize limitations of violinplot.countplot_limitations: Visualize limitations of countplot.categorical_variable_perc: Visualize percentage of categorical variables.spearman_correlation: Visualize spearman correlation.calculate_crammers_v: Calculate Crammer's V.
-
what_if_streamlit:
centroid_softmax_proba: Compute class probabilities from centroid distances (softmax).get_predict_proba_fn: Return a predict-proba function for sklearn or centroid-softmax mode.ShapSaver: Save SHAP explainer components for lazy loading in what-if analysis.ColumnMetadataGenerator: Generate column metadata from a DataFrame or CSV file.
-
monitoring:
mapping: Create column mapping from configuration file for Evidently.test_data: Test data for issues using Evidently test suites.check_data_drift: Check data for drift using Evidently metrics.send_email_with_table: Send email notifications with HTML tables for monitoring alerts.
Release files for utilsds 2.0.20
For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.
Source distribution (sdist)
| File | Size | Uploaded | |
|---|---|---|---|
| utilsds-2.0.20.tar.gz | 57.8 kB | Details |
Built distribution (wheel)
| File | Interpreter | ABI | Platform | Reset |
|---|---|---|---|---|
| utilsds-2.0.20-py3-none-any.whl | Python 3 | none | any | Details |
Total release size: 118.6 kB
Release files / utilsds-2.0.20.tar.gz
| Download URL | utilsds-2.0.20.tar.gz |
|---|---|
| Size | 57.8 kB |
| Tags | Source |
|
SHA-256 checksum How to use checksums |
3d85ceb130b904c4fc27b8c9846647837f093f86f114149e5036f19f2f3abfcb
|
|
BLAKE2b-256 checksum How to use checksums |
cc3ad4db2caaf1c7e33afe487746db1fe243a7676f5d848855d3f66faf20f19b
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.13
|
Release files / utilsds-2.0.20-py3-none-any.whl
| Download URL | utilsds-2.0.20-py3-none-any.whl |
|---|---|
| Size | 60.8 kB |
| Tags | Python 3 |
|
SHA-256 checksum How to use checksums |
fd2e94ace76687e7dabbbc5e9c74bc360e9a2e7c1bb93e3039e2562e51495eb3
|
|
BLAKE2b-256 checksum How to use checksums |
1bc21d893279de38f9320265c362759baef299f8bbdedca587c4f237f53a2159
|
| Upload date | |
|
Uploaded using Trusted Publishing? What is trusted publishing? |
No |
| Uploaded via |
twine/6.2.0 CPython/3.12.13
|
