Skip to main content

NLP Test: Deliver Safe & Effective Language Models

Project's Website • Key Features • How To Use • Community Support • Contributing • Mission • License

screenshot

Project's Website

Take a look at our official page for user documentation and examples: nlptest.org

Key Features

  • Generate and execute more than 50 distinct types of tests only with 1 line of code
  • Test all aspects of model quality: robustness, bias, representation, fairness and accuracy.​
  • Automatically augment training data based on test results (for select models)​
  • Support for popular NLP frameworks for NER and Text-Classifcation: Spark NLP, Hugging Face & Transformers.
  • Support for testing LLMS ( OpenAI, Cohere, AI21, Hugging Face Inference API and Azure-OpenAI LLMs) for question answering and summarization task.

How To Use

# Install nlptest
!pip install nlptest

# Import and create a Harness object
from nlptest import Harness
h = Harness(task='ner', model='dslim/bert-base-NER', hub='huggingface')

# Generate test cases, run them and view a report
h.generate().run().report()

Note For more extended examples of usage and documentation, head over to nlptest.org

Community Support

  • Slack For live discussion with the NLP Test community, join the #nlptest channel
  • GitHub For bug reports, feature requests, and contributions
  • Discussions To engage with other community members, share ideas, and show off how you use NLP Test!

Mission

While there is a lot of talk about the need to train AI models that are safe, robust, and fair - few tools have been made available to data scientists to meet these goals. As a result, the front line of NLP models in production systems reflects a sorry state of affairs.

We propose here an early stage open-source community project that aims to fill this gap, and would love for you to join us on this mission. We aim to build on the foundation laid by previous research such as Ribeiro et al. (2020), Song et al. (2020), Parrish et al. (2021), van Aken et al. (2021) and many others.

John Snow Labs has a full development team allocated to the project and is committed to improving the library for years, as we do with other open-source libraries. Expect frequent releases with new test types, tasks, languages, and platforms to be added regularly. We look forward to working together to make safe, reliable, and responsible NLP an everyday reality.

Contributing

We welcome all sorts of contributions:

  • Ideas
  • Feedback
  • Documentation
  • Bug reports
  • Development and testing

Feel free to clone the repo and submit pull-requests! You can also contribute by simply opening an issue or discussion in this repo.

Contributors

We would like to acknowledge all contributors of this open-source community project.

License

NLP Test is released under the Apache License 2.0, which guarantees commercial use, modification, distribution, patent use, private use and sets limitations on trademark use, liability and warranty.

Metadata

Release files for nlptest 1.5.0

For a detailed explanation of source distributions (sdists) and built distributions (wheels), please see the package formats documentation.

Source distribution (sdist)

Source distribution for nlptest 1.5.0
File Size Uploaded Provenance Actions
nlptest-1.5.0.tar.gz 59.5 MB Without provenance Details for nlptest-1.5.0.tar.gz

Built distribution (wheel)

Table of built distributions (wheels) for nlptest 1.5.0
File Interpreter ABI Platform Actions
nlptest-1.5.0-py3-none-any.whl Python 3 none any Details for nlptest-1.5.0-py3-none-any.whl

Total release size: 119.2 MB

Release files / nlptest-1.5.0.tar.gz

Download URL nlptest-1.5.0.tar.gz
Size 59.5 MB
Tags Source
SHA-256 checksum
How to use checksums
556ac72ef3e8a1c1851bbfe8d22e4312f01e7607d6d3a44775279766744ff9e2
BLAKE2b-256 checksum
How to use checksums
8760a076c56666dc4b4151c8b0bd2e92e197d1c9478eaf4927370df545e9863f
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.2 CPython/3.10.9

Release files / nlptest-1.5.0-py3-none-any.whl

Download URL nlptest-1.5.0-py3-none-any.whl
Size 59.7 MB
Tags Python 3
SHA-256 checksum
How to use checksums
9af341d89f0561479f3b5229e1b207c2a252c8149ddc68908b6cebd8d4890111
BLAKE2b-256 checksum
How to use checksums
a4e19cedcec0e256fcedfac3c42ce0ad8e2a1cef61ad782c0e14bb890f962b85
Upload date
Uploaded using Trusted Publishing?
What is trusted publishing?
No
Uploaded via twine/4.0.2 CPython/3.10.9

Security for nlptest 1.5.0

Provenance

Provenance describes where a file came from. On PyPI, provenance is shared via attestations, which provide a verifiable record of the build or publishing details. View details, limitations and caveats.

No provenance found

None of the files in this release were uploaded with provenance metadata.

A note on package security

PyPI is a repository for distributing packages, not a service for auditing them. While third-party security researchers and our staff do their best to detect and remove malicious packages, PyPI cannot guarantee the quality, security or suitability of any given package.

To learn more about protecting yourself from malicious actors, visit our package security documentation.

Release history Release notifications | RSS feed

This release

1.5.0 This release

2 release files No provenance

Anthropic, PBC Visionary sponsor Bloomberg Visionary sponsor Hudson River Trading Visionary sponsor Meta Visionary sponsor NVIDIA Visionary sponsor Microsoft Sustainability sponsor Depot Continuous Integration AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Sentry Error logging StatusPage Status page