Skip to main content

John Snow Labs NLU provides state of the art algorithms for NLP&NLU with hundreds of pretrained models in 60+ languages. It enables swift and simple development and research with its powerful Pythonic and Keras inspired API. It is powerd by John Snow Labs powerful Spark NLP library.

Project description

NLU: The Power of Spark NLP, the Simplicity of Python

John Snow Labs' NLU is a Python library for applying state-of-the-art text mining, directly on any dataframe, with a single line of code. As a facade of the award-winning Spark NLP library, it comes with hundreds of pretrained models in tens of languages - all production-grade, scalable, and trainable.

Project's Website

Take a look at our official Spark NLU page: https://nlu.johnsnowlabs.com/ for user documentation and examples

NLU in action

Features

  • Tokenization
  • Trainable Word Segmentation
  • Stop Words Removal
  • Token Normalizer
  • Document Normalizer
  • Stemmer
  • Lemmatizer
  • NGrams
  • Regex Matching
  • Text Matching
  • Chunking
  • Date Matcher
  • Sentence Detector
  • Deep Sentence Detector (Deep learning)
  • Dependency parsing (Labeled/unlabeled)
  • Part-of-speech tagging
  • Sentiment Detection (ML models)
  • Spell Checker (ML and DL models)
  • Word Embeddings (GloVe and Word2Vec)
  • BERT Embeddings (TF Hub models)
  • ELMO Embeddings (TF Hub models)
  • ALBERT Embeddings (TF Hub models)
  • XLNet Embeddings
  • Universal Sentence Encoder (TF Hub models)
  • BERT Sentence Embeddings (42 TF Hub models)
  • Sentence Embeddings
  • Chunk Embeddings
  • Unsupervised keywords extraction
  • Language Detection & Identification (up to 375 languages)
  • Multi-class Sentiment analysis (Deep learning)
  • Multi-label Sentiment analysis (Deep learning)
  • Multi-class Text Classification (Deep learning)
  • Neural Machine Translation
  • Text-To-Text Transfer Transformer (Google T5)
  • Named entity recognition (Deep learning)
  • Easy TensorFlow integration
  • GPU Support
  • Full integration with Spark ML functions
  • +710 pre-trained models in +192 languages!
  • +450 pre-trained pipelines in +192 languages!
  • Multi-lingual NER models: Arabic, Chinese, Danish, Dutch, English, Finnish, French, German, Hewbrew, Italian, Japanese, Korean, Norwegian, Persian, Polish, Portuguese, Russian, Spanish, Swedish, and Urdu.

Getting Started with NLU

To get your hands on the power of NLU, you just need to install it via pip and ensure Java 8 is installed and properly configured. Checkout Quickstart for more infos

pip install nlu pyspark==2.4.7

Loading and predict with any model in 1 line python

import nlu 
nlu.load('sentiment').predict('I love NLU! <3') 

Loading and predict with multiple models in 1 line

Get 6 different embeddings in 1 line and use them for downstream data science tasks!

nlu.load('bert elmo albert xlnet glove use').predict('I love NLU! <3') 

What kind of models does NLU provide?

NLU provides everything a data scientist might want to wish for in one line of code!

  • NLU provides everything a data scientist might want to wish for in one line of code!
  • 1000 + pre-trained models
  • 100+ of the latest NLP word embeddings ( BERT, ELMO, ALBERT, XLNET, GLOVE, BIOBERT, ELECTRA, COVIDBERT) and different variations of them
  • 50+ of the latest NLP sentence embeddings ( BERT, ELECTRA, USE) and different variations of them
  • 100+ Classifiers (NER, POS, Emotion, Sarcasm, Questions, Spam)
  • 300+ Supported Languages
  • Summarize Text and Answer Questions with T5
  • Labeled and Unlabeled Dependency parsing
  • Various Text Cleaning and Pre-Processing methods like Stemming, Lemmatizing, Normalizing, Filtering, Cleaning pipelines and more

Classifiers trained on many different different datasets

Choose the right tool for the right task! Whether you analyze movies or twitter, NLU has the right model for you!

  • trec6 classifier
  • trec10 classifier
  • spam classifier
  • fake news classifier
  • emotion classifier
  • cyberbullying classifier
  • sarcasm classifier
  • sentiment classifier for movies
  • IMDB Movie Sentiment classifier
  • Twitter sentiment classifier
  • NER pretrained on ONTO notes
  • NER trainer on CONLL
  • Language classifier for 20 languages on the wiki 20 lang dataset.

Utilities for the Data Science NLU applications

Working with text data can sometimes be quite a dirty Job. NLU helps you keep your hands clean by providing lots of components that take away data engineering intensive tasks.

  • Datetime Matcher
  • Pattern Matcher
  • Chunk Matcher
  • Phrases Matcher
  • Stopword Cleaners
  • Pattern Cleaners
  • Slang Cleaner

Where can I see NLUs entire offer?

Checkout the NLU Namespace for everything that NLU has to offer!

Supported Data Types

  • Pandas DataFrame and Series
  • Spark DataFrames
  • Modin with Ray backend
  • Modin with Dask backend
  • Numpy arrays
  • Strings and lists of strings

Checkout the following notebooks for examples on how to work with NLU.

NLU Demos on Datasets

NLU component examples

Checkout the following notebooks for examples on how to work with NLU.

NLU Training Examples

Binary Class Text Classification training

Multi Class Text Classification training

Multi Label Text Classification training

Named Entity Recognition training (NER)

Part of Speech tagger training (POS)

NLU Applications Examples

NLU Demos on Datasets

NLU examples grouped by component

The following are Collab examples which showcase each NLU component and some applications.

Named Entity Recognition (NER)

Part of speech (POS)

Sequence2Sequence

Classifiers

Word Embeddings

Sentence Embeddings

Sentence Embeddings

Dependency Parsing

Text Pre Processing and Cleaning

Chunkers

Matchers

Need help?

Simple NLU Demos

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for nlu, version 1.1.3
Filename, size File type Python version Upload date Hashes
Filename, size nlu-1.1.3-py3-none-any.whl (158.2 kB) File type Wheel Python version py3 Upload date Hashes View
Filename, size nlu-1.1.3.tar.gz (88.4 kB) File type Source Python version None Upload date Hashes View

Supported by

Pingdom Pingdom Monitoring Google Google Object Storage and Download Analytics Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page