Skip to main content

MIT tools to work with D3M datasets.

Project description

“mit-d3m” An open source project from Data to AI Lab at MIT.

Travis PyPi Shield



MIT tools to work with D3M datasets.



mit-d3m has been developed and tested on Python 3.5, 3.6 and 3.7

Also, although it is not strictly required, the usage of a virtualenv is highly recommended in order to avoid interfering with other software installed in the system where mit-d3m is run.

These are the minimum commands needed to create a virtualenv using python3.6 for mit-d3m:

pip install virtualenv
virtualenv -p $(which python3.6) mit-d3m-venv

Afterwards, you have to execute this command to have the virtualenv activated:

source mit-d3m-venv/bin/activate

Remember about executing it every time you start a new console to work on mit-d3m!

Install with pip

After creating the virtualenv and activating it, we recommend using pip in order to install mit-d3m:

pip install mit-d3m

This will pull and install the latest stable release from PyPi.

Install from source

Alternatively, with your virtualenv activated, you can clone the repository and install it from source by running make install on the stable branch:

git clone
cd mit-d3m
git checkout stable
make install

For development, you can use make install-develop instead in order to install all the required dependencies for testing and code linting.



  • Support downloading public datasets withouth requiring an AWS account
  • Imporoved error messages
  • Internal code cleanup
  • Initial unit test suite


  • disk_usage fails on macOS/BSD with human=True
  • Support latest d3m schema versions
  • Support python 3.7+


  • Set the d3mIndex column as a DataFrame column in the TabularLoader


  • Removal of unnecessary dependencies
  • Separate tabular into single_table and multi_table
  • Improve datasets stats collection.


  • D3M Dataset parser
  • Loaders by Data Modality
  • Dataset Stats generator
  • Dataset Configuration generator
  • Metrics functions
  • MongoDB connector

Project details

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mit-d3m-0.2.1.tar.gz (53.2 kB view hashes)

Uploaded source

Built Distribution

mit_d3m-0.2.1-py2.py3-none-any.whl (17.0 kB view hashes)

Uploaded py2 py3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page