Skip to main content

Synthetic dataset insights.

Project description

Dataset Insights

PyPI python PyPI version Downloads Tests License

Unity Dataset Insights is a python package for downloading, parsing and analyzing synthetic datasets generated using the Unity Perception package.


Dataset Insights maintains a pip package for easy installation. It can work in any standard Python environment using pip install datasetinsights command.

Getting Started

Dataset Statistics

We provide a sample notebook to help you load synthetic datasets generated using Perception package and visualize dataset statistics. We plan to support other sample Unity projects in the future.

Dataset Download

You can download the datasets from HTTP(s), GCS, and Unity simulation projects using the 'download' command from CLI or API.


datasetinsights download \
  --source-uri=<xxx> \


UnitySimulationDownloader downloads a dataset from Unity Simulation.

from import UnitySimulationDownloader

dest = "~/data"
access_token = "XXX"
downloader = UnitySimulationDownloader(access_token=access_token), output=dest)

GCSDatasetDownloader downloads a dataset from GCS location.

from import GCSDatasetDownloader

source_uri=gs://url/to/ or gs://url/to/folder
dest = "~/data"
downloader = GCSDatasetDownloader(), output=dest)

HTTPDatasetDownloader downloads a dataset from any HTTP(S) location.

from import HTTPDatasetDownloader

dest = "~/data"
downloader = HTTPDatasetDownloader(), output=dest)

Dataset Explore

You can explore the dataset schema by using following API:

Unity Perception

AnnotationDefinitions and MetricDefinitions loads synthetic dataset definition tables and return a dictionary containing the definitions.

from datasetinsights.datasets.unity_perception import AnnotationDefinitions,
annotation_def = AnnotationDefinitions(data_root=dest, version="my_schema_version")
definition_dict = annotation_def.get_definition(def_id="my_definition_id")

metric_def = MetricDefinitions(data_root=dest, version="my_schema_version")
definition_dict = metric_def.get_definition(def_id="my_definition_id")

Captures loads synthetic dataset captures tables and return a pandas dataframe with captures and annotations columns.

from datasetinsights.datasets.unity_perception import Captures
captures = Captures(data_root=dest, version="my_schema_version")
captures_df = captures.filter(def_id="my_definition_id")

Metrics loads synthetic dataset metrics table which holds extra metadata that can be used to describe a particular sequence, capture or annotation and return a pandas dataframe with captures and metrics columns.

from datasetinsights.datasets.unity_perception import Metrics
metrics = Metrics(data_root=dest, version="my_schema_version")
metrics_df = metrics.filter_metrics(def_id="my_definition_id")


You can use the pre-build docker image unitytechnologies/datasetinsights to run similar commands.


You can find the API documentation on readthedocs.


Please let us know if you encounter a bug by filing an issue. To learn more about making a contribution to Dataset Insights, please see our Contribution page.


Dataset Insights is licensed under the Apache License, Version 2.0. See LICENSE for the full license text.


If you find this package useful, consider citing it using:

    title={Unity {D}ataset {I}nsights Package},
    author={{Unity Technologies}},

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for datasetinsights, version 1.1.1
Filename, size File type Python version Upload date Hashes
Filename, size datasetinsights-1.1.1-py3-none-any.whl (1.5 MB) File type Wheel Python version py3 Upload date Hashes View
Filename, size datasetinsights-1.1.1.tar.gz (1.5 MB) File type Source Python version None Upload date Hashes View

Supported by

AWS AWS Cloud computing Datadog Datadog Monitoring Facebook / Instagram Facebook / Instagram PSF Sponsor Fastly Fastly CDN Google Google Object Storage and Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Salesforce Salesforce PSF Sponsor Sentry Sentry Error logging StatusPage StatusPage Status page