Skip to main content

Chainer IO library

Project description

ChainerIO

ChainerIO is an IO abstraction library for Chainer, optimized for deep learning training with batteries included. It supports

  • Filesystem API abstraction with unified error semantics,
  • Explicit user-land caching system,
  • IO performance tracing and metrics stats, and
  • Fileset container utilities to save metadata.

Dependency

  • HDFS client and libhdfs for HDFS access
  • Python 3

Installation and Document build

Installation

$ git clone https://github.com/chainer/chainerio.git
$ cd chainerio
$ pip install .

Documentation

$ cd chainerio/docs
$ make html
$ open build/html/index.html

Test

$ cd chainerio
$ pip install .[test]
$ pytest

How to use

Please refer to the Documentation for more information about the usage. Also you can find some examples in examples directory for usage in Chainer training script examples.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

chainerio-0.1.2.tar.gz (18.3 kB view details)

Uploaded Source

Built Distribution

chainerio-0.1.2-py3-none-any.whl (24.5 kB view details)

Uploaded Python 3

File details

Details for the file chainerio-0.1.2.tar.gz.

File metadata

  • Download URL: chainerio-0.1.2.tar.gz
  • Upload date:
  • Size: 18.3 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.0.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.7.4

File hashes

Hashes for chainerio-0.1.2.tar.gz
Algorithm Hash digest
SHA256 12fde0a959e27ba2702b89712e7dc3e22e0a57e58e2ad6cf69880dc5ca900db7
MD5 4102d05ff4485bf18b45442c44c4b179
BLAKE2b-256 170416e88a4f479d0e7cee5a6d36416417470cd1bc9c79f86855abbe36dbd6e0

See more details on using hashes here.

Provenance

File details

Details for the file chainerio-0.1.2-py3-none-any.whl.

File metadata

  • Download URL: chainerio-0.1.2-py3-none-any.whl
  • Upload date:
  • Size: 24.5 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.23.0 setuptools/46.0.0 requests-toolbelt/0.9.1 tqdm/4.43.0 CPython/3.7.4

File hashes

Hashes for chainerio-0.1.2-py3-none-any.whl
Algorithm Hash digest
SHA256 d613518413e3a2965954ad9418b266669781eed69e43b0c3c2a1035dd46ac35a
MD5 df1f59b26b467c9b08e7a448c2a451a8
BLAKE2b-256 dcd462d10e7a48a57276053f02e151bef04560ed3cd9fa46f0cc16b858ef78d4

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page