Skip to main content

AI Benchmark is an open source python library for evaluating AI performance of various hardware platforms, including CPUs, GPUs and TPUs.

Project description

AI Benchmark Alpha is an open source python library for evaluating AI performance of various hardware platforms, including CPUs, GPUs and TPUs. The benchmark is relying on TensorFlow machine learning library, and is providing a lightweight and accurate solution for assessing inference and training speed for key Deep Learning models.

In total, AI Benchmark consists of 42 tests and 19 sections provided below:

  1. MobileNet-V2  [classification]
  2. Inception-V3  [classification]
  3. Inception-V4  [classification]
  4. Inception-ResNet-V2  [classification]
  5. ResNet-V2-50  [classification]
  6. ResNet-V2-152  [classification]
  7. VGG-16  [classification]
  8. SRCNN 9-5-5  [image-to-image mapping]
  9. VGG-19  [image-to-image mapping]
  10. ResNet-SRGAN  [image-to-image mapping]
  11. ResNet-DPED  [image-to-image mapping]
  12. U-Net  [image-to-image mapping]
  13. Nvidia-SPADE  [image-to-image mapping]
  14. ICNet  [image segmentation]
  15. PSPNet  [image segmentation]
  16. DeepLab  [image segmentation]
  17. Pixel-RNN  [inpainting]
  18. LSTM  [sentence sentiment analysis]
  19. GNMT  [text translation]

For more information and results, please visit the project website: http://ai-benchmark.com/alpha

Installation Instructions

The benchmark requires TensorFlow machine learning library to be present in your system.

On systems that do not have Nvidia GPUs, run the following commands to install AI Benchmark:

pip install tensorflow
pip install ai-benchmark

If you want to check the performance of Nvidia graphic cards, run the following commands:

pip install tensorflow-gpu
pip install ai-benchmark

Note 1: If Tensorflow is already installed in your system, you can skip the first command.

Note 2: For running the benchmark on Nvidia GPUs, NVIDIA CUDA and cuDNN libraries should be installed first. Please find detailed instructions here.

Getting Started

To run AI Benchmark, use the following code:

from ai_benchmark import AIBenchmark
benchmark = AIBenchmark()
results = benchmark.run()

Alternatively, on Linux systems you can type ai-benchmark in the command line to start the tests.

To run inference or training only, use benchmark.run_inference() or benchmark.run_training().

Advanced settings

AIBenchmark(use_CPU=None, verbose_level=1):

use_CPU={True, False, None}:   whether to run the tests on CPUs  (if tensorflow-gpu is installed)

verbose_level={0, 1, 2, 3}:   run tests silently | with short summary | with information about each run | with TF logs

benchmark.run(precision="normal"):

precision={"normal", "high"}:   if high is selected, the benchmark will execute 10 times more runs for each test.


Additional Notes and Requirements

GPU with at least 2GB of RAM is required for running inference tests / 4GB of RAM for training tests.

The benchmark is compatible with both TensorFlow 1.x and 2.x versions.

Contacts

Please contact andrey@vision.ee.ethz.ch for any feedback or information.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

ai-benchmark-0.1.2.tar.gz (21.4 MB view details)

Uploaded Source

Built Distributions

ai_benchmark-0.1.2-py3-none-any.whl (21.5 MB view details)

Uploaded Python 3

ai_benchmark-0.1.2-py2-none-any.whl (21.5 MB view details)

Uploaded Python 2

File details

Details for the file ai-benchmark-0.1.2.tar.gz.

File metadata

  • Download URL: ai-benchmark-0.1.2.tar.gz
  • Upload date:
  • Size: 21.4 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.23.4 CPython/3.5.3

File hashes

Hashes for ai-benchmark-0.1.2.tar.gz
Algorithm Hash digest
SHA256 759ae01af1f8f1fecacf73b3313c9722d37274f778a0e842feab8c935263580c
MD5 c9ae2ad78d1c71b40e14df6e56d9c00a
BLAKE2b-256 999e6685285db14f407d5061e6022f96400f6fe958a70ba320472178151ded4b

See more details on using hashes here.

File details

Details for the file ai_benchmark-0.1.2-py3-none-any.whl.

File metadata

  • Download URL: ai_benchmark-0.1.2-py3-none-any.whl
  • Upload date:
  • Size: 21.5 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.23.4 CPython/3.5.3

File hashes

Hashes for ai_benchmark-0.1.2-py3-none-any.whl
Algorithm Hash digest
SHA256 e97fd54be8e62227a10bf4fad48ba3bf7d116604e11767f0c6307680760dff98
MD5 c3c4aa880cf6164876324b66291ce7bf
BLAKE2b-256 e0702f4581a0b48ffedcb555251c02476fb19a519335a4da63b4f9795b03716d

See more details on using hashes here.

File details

Details for the file ai_benchmark-0.1.2-py2-none-any.whl.

File metadata

  • Download URL: ai_benchmark-0.1.2-py2-none-any.whl
  • Upload date:
  • Size: 21.5 MB
  • Tags: Python 2
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/1.13.0 pkginfo/1.5.0.1 requests/2.21.0 setuptools/41.0.1 requests-toolbelt/0.9.1 tqdm/4.23.4 CPython/3.5.3

File hashes

Hashes for ai_benchmark-0.1.2-py2-none-any.whl
Algorithm Hash digest
SHA256 4f26560f2627d4681892525d7bd057d02b7aebbfee2c54527d5e13129601617c
MD5 ed217c05ab83f7a305c445c7c009266b
BLAKE2b-256 0a34e1f4dfd4713a01e114d1dbb28e9af164054177b870bc92a7c8ea8e6ce0b7

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page