Skip to main content
This is a pre-production deployment of Warehouse. Changes made here affect the production instance of PyPI (
Help us improve Python packaging - Donate today!

A library for topic modeling and browsing

Project Description

TOM (TOpic Modeling) is a Python 3 library for topic modeling and browsing, licensed under the MIT license. Its objective is to allow for an efficient analysis of a text corpus from start to finish, via the discovery of latent topics. To this end, TOM features functions for preparing and vectorizing a text corpus. It also offers a common interface for two topic models (namely LDA using either variational inference or Gibbs sampling, and NMF using alternating least-square with a projected gradient method), and implements three state-of-the-art methods for estimating the optimal number of topics to model a corpus. What is more, TOM constructs an interactive Web-based browser that makes it easy to explore a topic model and the related corpus.


Vector space modeling

  • Feature selection based on word frequency
  • Weighting
    • tf
    • tf-idf

Topic modeling

  • Latent Dirichlet Allocation
    • Standard variational Bayesian inference (Latent Dirichlet Allocation. Blei et al, 2003)
    • Online variational Bayesian inference (Online learning for Latent Dirichlet Allocation. Hoffman et al, 2010)
    • Collapsed Gibbs sampling (Finding scientific topics. Griffiths & Steyvers, 2004)
  • Non-negative Matrix Factorization (NMF)
    • Alternating least-square with a projected gradient method (Projected gradient methods for non-negative matrix factorization. Lin, 2007)

Estimating the optimal number of topics

  • Stability analysis (How many topics? Stability analysis for topic models. Greene et al, 2014)
  • Spectral analysis (On finding the natural number of topics with latent dirichlet allocation: Some observations. Arun et al, 2010)
  • Consensus-based analysis (Metagenes and molecular pattern discovery using matrix factorization. Brunet et al, 2004)


We recommend you to install Anaconda ( which will automatically install most of the required dependencies (i.e. pandas, numpy, scipy, scikit-learn, matplotlib, flask). You should then install the lda module (pip install lda). Eventually, clone or download this repo and run the following command:

python install

Or, install it directly from PyPi:

pip install tom_lib


We provide two sample programs, (which shows you how to load and prepare a corpus, estimate the optimal number of topics, infer the topic model and then manipulate it) and (which shows you how to generate a topic model browser to explore a corpus), to help you get started using TOM.

Load and prepare a textual corpus

The following code snippet shows how to load a corpus of French documents and vectorize them using tf-idf with unigrams.

corpus = Corpus(source_file_path='input/raw_corpus.csv',
print('corpus size:', corpus.size)
print('vocabulary size:', len(corpus.vocabulary))
print('Vector representation of document 0:\n', corpus.vector_for_document(0))

Instantiate a topic model and infer topics

It is possible to instantiate a NMF or LDA object then infer topics.


topic_model = NonNegativeMatrixFactorization(corpus)

LDA (using either the standard variational Bayesian inference or Gibbs sampling):

topic_model = LatentDirichletAllocation(corpus)
topic_model.infer_topics(num_topics=15, algorithm='variational')
topic_model = LatentDirichletAllocation(corpus)
topic_model.infer_topics(num_topics=15, algorithm='gibbs')

Instantiate a topic model and estimate the optimal number of topics

Here we instantiate a NMF object, then generate plots with the three metrics for estimating the optimal number of topics.

topic_model = NonNegativeMatrixFactorization(corpus)
viz = Visualization(topic_model)
                       tao=10, step=1,

Save/load a topic model

To allow reusing previously learned topics models, TOM can save them on disk, as shown below.

utils.save_topic_model(topic_model, 'output/NMF_15topics.tom')
topic_model = utils.load_topic_model('output/NMF_15topics.tom')

Topic model browser: screenshots

Topic cloud

### Topic details ### Document details

Release History

This version
History Node


History Node


History Node


History Node


Download Files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Filename, Size & Hash SHA256 Hash Help File Type Python Version Upload Date
(819.5 kB) Copy SHA256 Hash SHA256
Source None Jun 24, 2016

Supported By

Elastic Elastic Search Pingdom Pingdom Monitoring Dyn Dyn DNS Sentry Sentry Error Logging CloudAMQP CloudAMQP RabbitMQ Heroku Heroku PaaS Kabu Creative Kabu Creative UX & Design Fastly Fastly CDN DigiCert DigiCert EV Certificate Google Google Cloud Servers