Skip to main content

Intel AI Lab NLP and NLU research model library

Project description



A Deep Learning NLP/NLU library by Intel® AI Lab

GitHub Website GitHub release

Overview | Models | Installation | Examples | Documentation | Tutorials | Contributing

NLP Architect is an open source Python library for exploring state-of-the-art deep learning topologies and techniques for optimizing Natural Language Processing and Natural Language Understanding Neural Networks.

Overview

NLP Architect is an NLP library designed to be flexible, easy to extend, allow for easy and rapid integration of NLP models in applications and to showcase optimized models.

Features:

  • Core NLP models used in many NLP tasks and useful in many NLP applications

  • Novel NLU models showcasing novel topologies and techniques

  • Optimized NLP/NLU models showcasing different optimization algorithms on neural NLP/NLU models

  • Model-oriented design:

    • Train and run models from command-line.
    • API for using models for inference in python.
    • Procedures to define custom processes for training, inference or anything related to processing.
    • CLI sub-system for running procedures
  • Based on optimized Deep Learning frameworks:

  • Essential utilities for working with NLP models - Text/String pre-processing, IO, data-manipulation, metrics, embeddings.

Installing NLP Architect

We recommend to install NLP Architect in a new python environment, to use python 3.6+ with up-to-date pip, setuptools and h5py.

Install using pip

Install core library only

pip install nlp-architect

Install from source (Github)

Includes core library, examples, solutions and tutorials:

git clone https://github.com/NervanaSystems/nlp-architect.git
cd nlp-architect
pip install -e .  # install in developer mode

Running Examples and Solutions

To run provided examples and solutions please install the library with [all] flag which will install extra packages required. (requires installation from source)

pip install .[all]

Models

NLP models that provide best (or near) in class performance:

Natural Language Understanding (NLU) models that address semantic understanding:

Optimizing NLP/NLU models and misc. optimization techniques:

Solutions (End-to-end applications) using one or more models:

Documentation

Full library documentation of NLP models, algorithms, solutions and instructions on how to run each model can be found on our website.

NLP Architect library design philosophy

NLP Architect is a model-oriented library designed to showcase novel and different neural network optimizations. The library contains NLP/NLU related models per task, different neural network topologies (which are used in models), procedures for simplifying workflows in the library, pre-defined data processors and dataset loaders and misc utilities. The library is designed to be a tool for model development: data pre-process, build model, train, validate, infer, save or load a model.

The main design guidelines are:

  • Deep Learning framework agnostic
  • NLP/NLU models per task
  • Different topologies used in models
  • Showcase End-to-End applications (Solutions) utilizing one or more NLP Architect model
  • Generic dataset loaders, textual data processing utilities, and miscellaneous utilities that support NLP model development (loaders, text processors, io, metrics, etc.)
  • Procedures for defining processes for training, inference, optimization or any kind of elaborate script.
  • Pythonic API for using models for inference
  • Extensive model documentation and tutorials

Note

NLP Architect is an active space of research and development; Throughout future releases new models, solutions, topologies and framework additions and changes will be made. We aim to make sure all models run with Python 3.6+. We encourage researchers and developers to contribute their work into the library.

Citing

If you use NLP Architect in your research, please use the following citation:

@misc{izsak_peter_2018_1477518,
  title        = {NLP Architect by Intel AI Lab},
  month        = nov,
  year         = 2018,
  doi          = {10.5281/zenodo.1477518},
  url          = {https://doi.org/10.5281/zenodo.1477518}
}

Disclaimer

The NLP Architect is released as reference code for research purposes. It is not an official Intel product, and the level of quality and support may not be as expected from an official product. NLP Architect is intended to be used locally and has not been designed, developed or evaluated for production usage or web-deployment. Additional algorithms and environments are planned to be added to the framework. Feedback and contributions from the open source and NLP research communities are more than welcome.

Contact

Contact the NLP Architect development team through Github issues or email: nlp_architect@intel.com

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

nlp-architect-0.5.4.tar.gz (203.0 kB view details)

Uploaded Source

Built Distribution

nlp_architect-0.5.4-py3-none-any.whl (308.7 kB view details)

Uploaded Python 3

File details

Details for the file nlp-architect-0.5.4.tar.gz.

File metadata

  • Download URL: nlp-architect-0.5.4.tar.gz
  • Upload date:
  • Size: 203.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/46.0.0 requests-toolbelt/0.9.1 tqdm/4.41.1 CPython/3.7.7

File hashes

Hashes for nlp-architect-0.5.4.tar.gz
Algorithm Hash digest
SHA256 6e2ce4f3b6ecbd3436b01121cfe37a7bf627ea96be071957278ba33055458173
MD5 616ec1f1639b6c3e7ccb0e4ce2932e0c
BLAKE2b-256 af31e112432448decf9666fd029097fc13e791bc92a6772698e75bb7d114d994

See more details on using hashes here.

File details

Details for the file nlp_architect-0.5.4-py3-none-any.whl.

File metadata

  • Download URL: nlp_architect-0.5.4-py3-none-any.whl
  • Upload date:
  • Size: 308.7 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.1.1 pkginfo/1.5.0.1 requests/2.22.0 setuptools/46.0.0 requests-toolbelt/0.9.1 tqdm/4.41.1 CPython/3.7.7

File hashes

Hashes for nlp_architect-0.5.4-py3-none-any.whl
Algorithm Hash digest
SHA256 5ac9b4257b4f0475360c97895c203cdec379f98c5dfb6b14c7130cc1f0f59371
MD5 64270239031b7238eeac384abf12bf54
BLAKE2b-256 4af5cbe2673e5c5c5a6a94f66f32cd4f41770897c0b03b686c1a81744933a69f

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page