Topica: fast, all-purpose topic modeling for Python

topica is a fast, all-purpose topic-modeling library for Python, built for computational social scientists who want to go from a column of text to publishable results in one workflow. It brings together models usually split across JVM tools like MALLET and R packages like stm, more than two dozen in all (LDA, STM, CTM, plus neural, dynamic, and embedding-based models), each paired with the validation, covariate-effect, and reporting tools reviewers expect. Where general toolkits like Gensim or BERTopic give you topics, topica is built around the question social scientists ask of them next: how topic prevalence and content relate to covariates, with reference-validated models and reproducible fits. It installs as a single wheel that needs only NumPy and pandas: no JVM, no PyTorch.

pip install topica

Quick start

Point topica at a DataFrame and read the topics. This runs exactly as written, on a bundled example dataset, right after install:

import topica

df = topica.datasets.load_gadarian()          # bundled; loads offline
corpus = topica.from_dataframe(
    df, text_col="open.ended.response", stopwords=topica.ENGLISH_STOPWORDS
)

model = topica.LDA(num_topics=5, seed=42)
model.fit(corpus)                             # sensible defaults; no tuning required
print(topica.summary(model))                  # top words per topic

from_dataframe keeps your metadata aligned to the documents that survive pruning, so the same corpus feeds a structural topic model that relates topic prevalence to a covariate, with a well-calibrated hypothesis test:

prevalence = corpus.metadata[["treatment"]]   # a numeric DataFrame goes straight in

stm = topica.STM(num_topics=5, seed=42)
stm.fit(corpus, prevalence, prevalence_names=["treatment"])

draws  = topica.posterior_theta_samples(stm, nsims=30, seed=0)
effect = topica.estimate_effect(draws, prevalence, feature_names=["treatment"])

Your own data is one line away: pass pandas.read_csv("yours.csv") to from_dataframe. See the getting-started guide and the worked examples for analyses end to end.

Fits are reproducible and validated: the variational models are identical to the bit, the samplers reproduce from a fixed seed and thread count, and every model is checked against its reference implementation (R stm, MALLET, keyATM, and more).

The core needs only NumPy and pandas. Optional extras add features without weighing it down: topica[viz] (matplotlib plots), topica[formula] (R-style formulas), topica[polars] (Polars frames), and topica[llm] (LLM labels and embeddings, OpenAI or local via ollama).

Models

Starting out? LDA for general topics, STM to relate topics to covariates, HDP to let the data choose the number of topics, and BERTopic or CombinedTM for embedding-based topics. The full roster follows.

Every model exposes the same shape: fit(docs, …), then topic_word (φ), doc_topic (θ), top_words(n), and save/load, so one diagnostic, labeling, and effect-estimation stack applies to all of them and a new model inherits it for free. The embedding-based models take document vectors from any embedder (sentence-transformers, an API, or a local model such as ollama; no PyTorch or UMAP/numba in the wheel). Full guides: the models and embedding topics.

Diagnostics & analysis

Model-agnostic: they work on any fitted model's topic_word/doc_topic:

• Quality: coherence (u_mass, c_v, c_uci, c_npmi; co-occurrence counting in the Rust core), exclusivity, topic_diversity, quality_frontier
• Labeling: label_topics (prob / FREX / lift / score), frex, relevance, find_thoughts, topic_table, summary
• Validation: word_intrusion, document_intrusion, bootstrap_stability, search_k
• Reliability: select_model (fit many seeds) and ensemble (combine runs into a consensus more reliable than any single fit — cluster/align/stable methods, the last a gensim EnsembleLda port)
• Comparison: fighting_words (weighted log-odds) for contrasting corpora
• Covariate effects: estimate_effect (method of composition, cluster-robust SEs, GLM links), topic_correlation, and the design helpers one_hot, spline, and interaction (all top level; they build covariate bases for any model's design matrix); posterior_theta_samples draws θ for the logistic-normal models (STM/CTM)
• Preprocessing: tokenize, learn_phrases / apply_phrases, split_documents, the Corpus class

See diagnostics and covariate effects.

Performance

topica runs on a parallel Rust core. It is several times faster than R stm — the single-threaded field standard — for the structural and other variational models, and it matches the hand-tuned compiled samplers core for core: parity with Java MALLET on plain LDA and with the C++ keyATM on keyword models. Fit to convergence (both at the same emtol, spectral start), on real corpora:

topica also fits in about a quarter of R stm's memory (≈180MB against ≈675MB at 5,000 documents). For the approximate parallel Gibbs samplers the multithreaded speedup grows with corpus size: the per-sweep count-table merge is fixed overhead, so larger corpora amortize it over more sampling work. LDA's eight-core speedup over MALLET runs about 3× at 2,000 documents and reaches ~4× at 5,000.

Every fit is reproducible from a fixed seed and validated against its reference. See Benchmarks for the full methodology; reproduce the structural-model table with python benchmarks/bench_stm_convergence.py and the size-varying LDA curve with python benchmarks/speed_vs_size.py.

Install from source

pip install maturin
git clone https://github.com/nealcaren/topica && cd topica
python -m venv .venv && source .venv/bin/activate
maturin develop --release --features python

Requires numpy >= 1.21. Use --release (the debug build is much slower).

Acknowledgements

Topica was inspired by a post from David Mimno about porting Java MALLET to Rust. As a long-time Python user, I had long been jealous of the topic-modeling tools available in other languages; this seemed like an opportunity to make those capabilities easier to use in Python for me and for others.

As such, Topica stands on a generation of open topic-modeling research and code. Each entry below lists the reference, its authors and year, and the topica class(es) it underlies; the other models are Rust ports or reimplementations, validated against these reference implementations.

• MALLET (McCallum, 2002) — LDA, DMR, LabeledLDA: the SparseLDA sampler, Dirichlet-multinomial regression, and hyperparameter optimization. LDA began as a port of David Mimno's RustMallet (Apache-2.0) and follows its SparseLDA sampler and fixed-point optimizer closely, but uses its own RNG (PCG), so it is not byte-identical to RustMallet. Against Java MALLET (also a different RNG) it recovers the same topics on a planted corpus (cosine 1.000)
• stm (Roberts, Stewart & Tingley, 2019) — STM, CTM, SAGE: variational EM, estimateEffect, searchK, FREX, spectral initialization, and the method of composition
• sts (Chen & Mankad, 2024) — STS: the Structural Topic and Sentiment-Discourse model — the joint prevalence/sentiment Laplace E-step and the Poisson topic-word M-step, validated against the package
• lda-c / ctm-c / dtm and hdp (Blei lab, 2006–2007) — CTM, DTM, HDP: the CTM, Dynamic Topic Model, and HDP samplers
• gensim (Řehůřek & Sojka, 2010) — DTM, ensemble: the coherence-pipeline conventions (the coherence_type= API and default sliding windows; the measures themselves are Röder et al. 2015 and Mimno et al. 2011), the LdaSeqModel DTM reference, and the EnsembleLda (CBDBSCAN stable-topic) method ported for ensemble(method="stable")
• tomotopy (bab2min, 2020) — API conventions (summary, the short-text models), and GDMR (generalized DMR; Lee & Song, 2020), validated against its GDMRModel
• scikit-learn (Pedregosa et al., 2011) — NMF: the multiplicative-update solver (Lee & Seung, 2001) and the NNDSVD initialization (Boutsidis & Gallopoulos, 2008), validated against sklearn.decomposition.NMF (BSD-3-Clause); and LSA: latent semantic analysis / indexing (Deerwester et al., 1990), validated against sklearn.decomposition.TruncatedSVD (BSD-3-Clause) including its svd_flip sign convention. The numerics are reimplemented in Rust; the randomized truncated SVD shared by both (it seeds NMF's NNDSVD and is the LSA factorization itself) follows Halko et al. (2011).
• keyATM (Eshima, Imai & Sasaki, 2024) — KeyATM: the base, covariate, and dynamic models, the information-theory token weighting, and the Chib (1998) change-point HMM, validated against the package
• seededlda (Watanabe, 2023) — SeededLDA: the corpus-frequency-scaled seed prior (count × weight × 100), validated against the package's seed matrix and seeded topics
• LightLDA (Yuan et al., 2015) — LDA: the alias-table Metropolis-Hastings sampler
• GSDMM (Yin & Wang, 2014) — GSDMM: the movie-group-process mixture for short text
• BTM (Yan, Guo, Lan & Cheng, 2013; R package by Jan Wijffels) — BTM: the biterm co-occurrence topic model for short text
• Polylingual Topic Models (Mimno, Wallach, Naradowsky, Smith & McCallum, 2009) — PolylingualLDA: LDA over aligned document tuples that share one topic distribution, giving topics aligned across many languages; validated against MALLET's PolylingualTopicModel as a black-box oracle
• DiscLDA (Lacoste-Julien, Sha & Jordan, 2008) — DiscLDA: discriminative LDA with per-class and shared topic blocks; the fixed block-transform variant, validated against the paper's 20 Newsgroups feature-classification result (no reference implementation exists, so it is paper-derived)
• ProdLDA / AVITM (Srivastava & Sutton, 2017) — ProdLDA: autoencoding variational inference and the product-of-experts word model
• SCHOLAR (Card, Tan & Smith, 2018; reference dallascard/scholar, Apache-2.0) — Scholar: metadata in a ProdLDA VAE — a covariate-dependent topic-prevalence prior (neural STM prevalence), an optional supervised label head (neural sLDA), and optional content/topic-covariate word deviations (neural SAGE), on topica's ProdLDA backbone, validated against the reference as a numerical oracle
• BERTopic (Grootendorst, 2022) and Top2Vec (Angelov, 2020) — BERTopic, Top2Vec: the embedding-clustering pipeline, class-based TF-IDF, and the reduce → cluster → represent design
• ETM (Dieng, Ruiz & Blei, 2020) — ETM: the Embedded Topic Model (per-document variational EM and an amortized VAE)
• DETM (Dieng, Ruiz & Blei, 2019) — DETM: the Dynamic Embedded Topic Model (structured amortized variational inference with a hand-coded LSTM)
• FASTopic (Wu et al., 2024) — FASTopic: the optimal-transport topic model
• contextualized-topic-models (Bianchi et al., MIT) — CombinedTM (Bianchi, Terragni & Hovy, 2021) and ZeroShotTM (Bianchi, Nozza & Hovy, 2021): ProdLDA encoders that read a contextual document embedding, alongside or in place of the bag of words
• quanteda.textmodels (Benoit et al., 2018) — Wordfish: the Slapin & Proksch (2008) Poisson scaling model, validated against its textmodel_wordfish (the recovered scale and the analytic position standard errors both match at correlation 1.00 on a corpus sampled from the model)
• tbip (Vafa, Naidu & Blei, 2020) — TBIP: Text-Based Ideal Points; the official implementation is TensorFlow 1.x, so topica reimplements the published model and its mean-field variational inference, validated against an independent PyTorch reference
• TensorLy TLDA (Kangaslahti, Ebanks, Kossaifi, Liu, Alvarez & Anandkumar, 2026) — TensorLDA: online tensor latent Dirichlet allocation via second- and third-order moments. The Rust implementation is experimental; see the TensorLDA validation record for its current evidence and limitations.
• partyembed (Rheault & Cochrane, 2020) — PartyEmbeddings: party embeddings via a PV-DM paragraph-vector model with party-period metadata tags, placed by PCA of the learned party vectors. The reference builds on gensim's Doc2Vec; topica reimplements the PV-DM negative-sampling training in Rust (from Mikolov et al. 2013 and Le & Mikolov 2014) and is validated against that Doc2Vec scale (correlation 1.00 on a planted ordering)
• CLNTM (Nguyen & Luu, 2021) — the InfoNCE contrastive regularization on topic vectors offered by the contrastive= flag on the VAE models
• WHAI / Weibull-Dirichlet VAE (Zhang et al., 2018; Burkhardt & Kramer, 2019) — the Weibull-reparameterized Dirichlet prior offered by prior="dirichlet" on the VAE models
• Neural variational topic models with alternative priors (Miao, Grefenstette & Blunsom, 2017; Nalisnick & Smyth, 2017) — the Gaussian stick-breaking prior offered by prior="stick_breaking" on the VAE models
• TopicGPT (Pham et al., NAACL 2024, MIT) — TopicGPT: the generate / refine / assign prompt flow for LLM-driven topic discovery

The embedding-native models build on two pure-Rust crates: petal-clustering for HDBSCAN and umap-rs for the optional UMAP reducer, both BLAS-free.

Full citations for every model and reference implementation, and how to cite topica, are on the Citing page.

License

Apache-2.0 — see LICENSE.