ydata-sdk 3.2.0

YData allows to use the *Data-Centric* tools from the YData ecosystem to accelerate AI development

YData SDK

---

🚀 YData Package 🎉 Data quality and Synthetic Data for everyone!

ydata-sdk is here! Create an account and start using the best worldwide provider of data quality and synthetic data tools today! Read more about the benchmarks.

---

Documentation | More on YData

Overview

The YData SDK is the leading Python package for data analytics, data science, data engineering and AI development. The solution runs locally and includes a set of integrated components for data ingestion, standardized data quality evaluation, data improvement, and synthetic data generation.

Synthetic data can be used as Machine Learning performance enhancer, to augment or mitigate the presence of bias in real data. Furthermore, it can be used as a Privacy Enhancing Technology, to enable data-sharing initiatives or even to fuel testing environments.

Under the YData SDK hood, you can find a set of algorithms and metrics based on statistics and deep learning based techniques, that will help you to accelerate your data preparation.

Modules

YData SDK is composed by the following main modules:

• Connectors & Data Sources

  • Built-in connectors for easy integration with existing data sources. It supports several storage types, such as filesystems, RDBMS, Data warehouses, Lakehouses and much more. Check the list of connectors in the documentation.
  • SDK’s Datasources run on top of Dask, which allows it to deal with not only small workloads but also larger volumes of data.

• Data Profiling

  • Automatically generate the visualizations you need for your EDA.
  • A set of metrics and algorithms summarizes datasets quality in three main dimensions: warnings, univariate analysis and a multivariate perspective.

• Synthetic Data Generation

  • Generate synthetic data with a few lines of code and streamlite your data access, overcome data privacy concerns and enrich your datastes.
  • Simplified interface to train a generative model and learn in a data-driven manner the behavior, the patterns and original data distribution. Optimize your model for privacy or utility use-cases.
  • From a trained synthesizer, you can generate synthetic samples as needed and parametrise the number of records needed.

• Synthetic Data quality Report

  • An extensive synthetic data quality report that measures 3 dimensions: privacy, utility and fidelity of the generated data. The report can be downloaded in PDF format for ease of sharing and compliance purposes or as a JSON to enable the integration in data flows.

Features

YData SDK has several festures and for the complete set of features, please refer to the documentation. Among the most popular you can find:

• Generate a Profile Report
  • The most comprehensive and easy to use profile report / exploratory data analysis.
• Generate Synthetic Data
  • Generate either Tabular or Time-Series data using the best models worldwide!
• Generate a Comparison Report
  • Compare two datasets with a single line of code. Let the visualizations do the work for you.

Supported data formats

• Tabular The RegularSynthesizer is perfect to synthesize high-dimensional data, that is time-independent with high quality results.
• Time-Series The TimeSeriesSynthesizer is perfect to synthesize both regularly and not evenly spaced time-series, from smart-sensors to stock.

Installation

From PyPi

python -m pip install ydata-sdk

From Anaconda

conda install ydata-sdk

After Installation

We depend on spacy models for some parts and as spaCy documentation, you have to manually install them.

python -m pip install https://github.com/explosion/spacy-models/releases/download/en_core_web_lg-3.8.0/en_core_web_lg-3.8.0-py3-none-any.whl