An open source project from Data to AI Lab at MIT.

mit-d3m

• License: MIT
• Documentation: https://HDI-Project.github.io/mit-d3m/
• Homepage: https://github.com/HDI-Project/mit-d3m

Overview

MIT tools to work with D3M datasets.

Install

Requirements

mit-d3m has been developed and tested on Python 3.5, 3.6 and 3.7

Also, although it is not strictly required, the usage of a virtualenv is highly recommended in order to avoid interfering with other software installed in the system where mit-d3m is run.

These are the minimum commands needed to create a virtualenv using python3.6 for mit-d3m:

pip install virtualenv
virtualenv -p $(which python3.6) mit-d3m-venv

Afterwards, you have to execute this command to have the virtualenv activated:

source mit-d3m-venv/bin/activate

Remember about executing it every time you start a new console to work on mit-d3m!

Install with pip

After creating the virtualenv and activating it, we recommend using pip in order to install mit-d3m:

pip install mit-d3m

This will pull and install the latest stable release from PyPi.

Install from source

Alternatively, with your virtualenv activated, you can clone the repository and install it from source by running make install on the stable branch:

git clone git@github.com:HDI-Project/mit-d3m.git
cd mit-d3m
git checkout stable
make install

For development, you can use make install-develop instead in order to install all the required dependencies for testing and code linting.

History

0.2.1

• Support downloading public datasets withouth requiring an AWS account
• Imporoved error messages
• Internal code cleanup
• Initial unit test suite

0.2.0

• disk_usage fails on macOS/BSD with human=True
• Support latest d3m schema versions
• Support python 3.7+

0.1.2

• Set the d3mIndex column as a DataFrame column in the TabularLoader

0.1.1

• Removal of unnecessary dependencies
• Separate tabular into single_table and multi_table
• Improve datasets stats collection.

0.1.0

• D3M Dataset parser
• Loaders by Data Modality
• Dataset Stats generator
• Dataset Configuration generator
• Metrics functions
• MongoDB connector