clana

clana is a toolkit for classifier analysis. One key contribution of clana is Confusion Matrix Ordering (CMO) as explained in chapter 5 of Analysis and Optimization of Convolutional Neural Network Architectures. It is a technique that can be applied to any multi-class classifier and helps to understand which groups of classes are most similar.

Installation

The recommended way to install clana is:

$ pip install clana --user

If you want the latest version:

$ git clone https://github.com/MartinThoma/clana.git; cd clana
$ pip install -e . --user

Usage

clana --help
Usage: clana [OPTIONS] COMMAND [ARGS]...

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  distribution   Get the distribution of classes in a dataset.
  get-cm         Calculate the confusion matrix (CSV inputs).
  get-cm-simple  Calculate the confusion matrix (one label per...
  visualize      Optimize confusion matrix.

The visualize command gives you images like this:

Confusion Matrix after Confusion Matrix Ordering of the WiLI-2018 dataset

Data distribution

$ clana distribution --gt gt.csv --labels labels.csv [--out out/] [--long]

prints one line per label, e.g.

60% cat (56789 elements)
20% dog (12345 elements)
 5% mouse (1337 elements)
 1% tux (314 elements)

If --out is specified, it creates a horizontal bar chart. The first bar is the most common class, the second bar is the second most common class, ...

It uses the short labels, except --long is added to the command.

Metrics

$ clana metrics --gt gt.csv --preds preds.csv

gives the following metrics by

Line 1: Accuracy
Line 2: Precision
Line 3: Recall
Line 4: F1-Score
Line 5: Mean accuracy

Visualizations

See visualizations

Development

Check tests with tox.

Release files for clana 0.3.2

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for clana 0.3.2
File	Size	Uploaded
clana-0.3.2.tar.gz	16.1 kB	Sep 12, 2019	Details

Built distribution (wheel)

Table of built distributions (wheels) for clana 0.3.2
File	Interpreter	ABI	Platform	Reset
clana-0.3.2-py3-none-any.whl 18.6 kB Sep 12, 2019	Python 3	none	any	Details

Total release size: 34.8 kB

Release files / clana-0.3.2.tar.gz

Download URL	clana-0.3.2.tar.gz
Size	16.1 kB
Tags	Source
SHA-256 checksum How to use checksums	`5711d448f4d9bdd95069fc23f058010444e3c6b0b43d80cf920498a18a0d6994`
BLAKE2b-256 checksum How to use checksums	`75bb544eb0b4af5469f99841f7bf95bd5ae90e77bac4d593549f9292d52e6e76`
Upload date	Sep 12, 2019
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/1.11.0 pkginfo/1.4.2 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.8`

Release files / clana-0.3.2-py3-none-any.whl

Download URL	clana-0.3.2-py3-none-any.whl
Size	18.6 kB
Tags	Python 3
SHA-256 checksum How to use checksums	`011d8aba30d8d6b3c577ea8d3e266895f7c10caf5f2907b2d9d2e5a58e9ace92`
BLAKE2b-256 checksum How to use checksums	`10dcfc99c71aef5d7ebd4ab4d56ccf666ceca730b788c134952881da83957da0`
Upload date	Sep 12, 2019
Uploaded using Trusted Publishing? What is trusted publishing?	No
Uploaded via	`twine/1.11.0 pkginfo/1.4.2 requests/2.22.0 setuptools/41.0.1 requests-toolbelt/0.8.0 tqdm/4.28.1 CPython/3.6.8`