Skip to main content

A library for pytorch layer testing.

Project description

tencheck

tencheck provides a simple set of utilities for analyzing and validating the properties of layers of deep neural nets.

It's typically quite difficult to validate that a layer "behaves properly" (oftentimes, the final barometer is simply how well a model performs with the layer included), and many brittle unit tests have been written involving randomly instantiated tensors and is_close checks. We believe a good "first line of defense" for neural nets is to create a suite of properties that can be asserted about a layer, while requiring minimal effort per layer in order to do so.

We think there are two aspects of property-based testing that are quite useful to take inspiration from:

  • Automatically generating inputs (and generating inputs of variable sizes and values to elucidate properties of interest).
  • Evaluating properties based on the maintenance of invariants instead of attempting to exactly match values (which is particularly difficult to interpret in deep neural nets).

However, an important difference is that the properties of interest are generally fairly generic and often shared between layers, while the input generation strategies are pretty similar (they're all tensors). So the focus of tencheck is to provide:

  • An (attempted) universal input generation harness.
  • A variety of interesting properties.
  • Three modalities: assertion, analysis, and profiling.

The following requirements need to be met for tencheck to work:

  • Your layers are implemented in torch.
  • The .forward() method is annotated with jaxtyping.

Backlog

  • For profiling, use a grid of input sizes to generate performance curves.
  • Pick a flop counter and use for profiling
  • Tensor container types include more options like dataclasses.
  • Auto-generate simple hyperparameters for layer instantiation.
  • Refine dtype mapping and coherence.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

tencheck-0.0.4.tar.gz (11.5 kB view details)

Uploaded Source

Built Distribution

tencheck-0.0.4-py3-none-any.whl (11.7 kB view details)

Uploaded Python 3

File details

Details for the file tencheck-0.0.4.tar.gz.

File metadata

  • Download URL: tencheck-0.0.4.tar.gz
  • Upload date:
  • Size: 11.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.11.4

File hashes

Hashes for tencheck-0.0.4.tar.gz
Algorithm Hash digest
SHA256 6c42d61da716917a70f57db4074eb9ca3082c56458e5a79f4b180273849a31d9
MD5 89ffb237d434901c94d59f0f352da885
BLAKE2b-256 45109a27bd617c76c89863ab940229f3d7671c5b2356e26989ab4067cb7facf3

See more details on using hashes here.

File details

Details for the file tencheck-0.0.4-py3-none-any.whl.

File metadata

  • Download URL: tencheck-0.0.4-py3-none-any.whl
  • Upload date:
  • Size: 11.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.1.1 CPython/3.11.4

File hashes

Hashes for tencheck-0.0.4-py3-none-any.whl
Algorithm Hash digest
SHA256 1fdb121ae2af14848d6dd6a18349c95b51101680f055beaf5b292615c7a13158
MD5 929ac2b069713eaefdbafd9e7f1cfbc5
BLAKE2b-256 b9084fd9861deaff70ce1bf22271769bd417aa894aee3af5cb4b0a6db7c89157

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page