bnc-lookup 1.0.6

Static Hash-Based Lookup for BNC Terms

BNC Lookup

Is this token a word? O(1) answer. No setup. No dependencies.

A simple question deserves a simple answer. This library gives you instant yes/no validation against 669,000 word forms from the British National Corpus.

Quick Start

pip install bnc-lookup

from bnc_lookup import is_bnc_term

# That's it. Start validating.
is_bnc_term('the')          # True
is_bnc_term('however')      # True
is_bnc_term('nonetheless')  # True
is_bnc_term('xyzabc123')    # False

# Handles plurals automatically
is_bnc_term('computers')    # True

# Case insensitive
is_bnc_term('THE')          # True

Features

• Zero Dependencies - Pure Python, no external packages
• Zero I/O - No filesystem access, no database queries
• Zero Setup - No corpus downloads or configuration
• Microsecond Lookups - O(1) dictionary access
• Smart Plurals - Automatically checks singular forms
• Simple API - One function does it all

The Problem This Solves

In NLP, you frequently need to answer the question: "Is this token a real word?"

Not "what does it mean?" Not "give me synonyms." Just: is this a word?

is_bnc_term('computer')	is_bnc_term('asdfgh')
True	False

That's it. O(1) response. No ambiguity.

Why BNC?

The British National Corpus isn't an academic wordlist (too narrow). It's not a web scrape (too noisy). It's not slang (too ephemeral).

It's a 100-million-word corpus of real British English collected from written and spoken sources between 1991-1994. Books, newspapers, academic papers, conversations. The BNC frequency list captures ~669,000 unique word forms actually used by native speakers.

If a token passes the BNC test, you can be confident it's a word that real people actually use.

When to Use This

• Tokenization filtering: Keep real words, discard garbage
• Input validation: Reject nonsense in user input
• NLP preprocessing: Filter candidates before expensive operations
• Spell-check pre-filtering: Quick reject obvious non-words before fuzzy matching
• Data cleaning: Identify malformed or corrupted text

What This Doesn't Do

• No definitions, synonyms, or semantic relationships (use spaCy for that)
• No frequency counts or rankings (just yes/no)
• No spell-checking or suggestions (just existence check)

Documentation

For detailed usage, performance benchmarks, and advanced features, see the API Documentation.

Development

git clone https://github.com/craigtrim/bnc-lookup.git
cd bnc-lookup
make install  # Install dependencies
make test     # Run tests
make all      # Full build pipeline

See API Documentation for detailed development information.

License

This package is dual-licensed:

• Software: MIT License
• BNC Data: BNC User Licence

See LICENSE for complete terms.

Attribution

This package contains data derived from the British National Corpus frequency lists:

BNC frequency lists compiled by Adam Kilgarriff. Source: https://www.kilgarriff.co.uk/BNClists/all.num.gz

The British National Corpus, version 3 (BNC XML Edition). 2007. Distributed by Bodleian Libraries, University of Oxford, on behalf of the BNC Consortium.

Note: This is a static snapshot of BNC frequency data. The data is not automatically updated.

See Also

• wordnet-lookup - Similar O(1) lookup using the WordNet lexicon

Links

• Repository: github.com/craigtrim/bnc-lookup
• PyPI: pypi.org/project/bnc-lookup
• BNC: natcorp.ox.ac.uk
• Author: Craig Trim ([e-mail])