Skip to main content

Chainer IO library

Project description

ChainerIO

ChainerIO is an IO abstraction library for Chainer, optimized for deep learning training with batteries included. It supports

  • Filesystem API abstraction with unified error semantics,
  • Explicit user-land caching system,
  • IO performance tracing and metrics stats, and
  • Fileset container utilities to save metadata.

Dependency

  • HDFS client and libhdfs for HDFS access
  • Python 3

Installation and Document build

Installation

$ git clone https://github.com/chainer/chainerio.git
$ cd chainerio
$ pip install .

Documentation

$ cd chainerio/docs
$ make html
$ open build/html/index.html

Test

$ cd chainerio
$ pip install .[test]
$ pytest

How to use

Please refer to the Documentation for more information about the usage. Also you can find some examples in examples directory for usage in Chainer training script examples.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for chainerio, version 0.1.1
Filename, size File type Python version Upload date Hashes
Filename, size chainerio-0.1.1-py3-none-any.whl (24.3 kB) File type Wheel Python version py3 Upload date Hashes View hashes
Filename, size chainerio-0.1.1.tar.gz (18.1 kB) File type Source Python version None Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page