ai-starter-kit 0.3.1

Importable modules for learning machine learning and deep learning.

ai-starter-kit

Importable boilerplate for machine learning / deep learning projects — reusable code wrapped around common libraries (like NLTK) so you don't have to rewrite the same setup every time. Install it, import what you need.

Install

pip install ai-starter-kit

Available modules

Module	Description
ask.text	Chains common NLP preprocessing steps (lowercasing, noise/punctuation removal, tokenization, stopword removal, stemming, lemmatization) behind a fluent, resettable TextProcessor API using NLTK.

More modules will be added as the project grows.

Text Processing

Each preprocessing step is a method on TextProcessor that returns self, so they can be called individually, one at a time, or chained together in a single fluent pipeline:

from ask.text import TextProcessor, PartOfSpeech

tp = TextProcessor([
    "Running, runners ran quickly through the running trails.",
    "Check out <b>this</b> amazing offer at https://example.com!! Call 12345 now.",
])

By default, constructing a TextProcessor downloads the NLTK data it needs (punkt, punkt_tab, stopwords, wordnet) the first time, which requires network access and can be slow — or fail outright in offline/sandboxed environments. Pass auto_download=False to skip this (e.g. if you've already fetched the data yourself via nltk.download(), or don't need methods that depend on it).

Individually, one step at a time, resetting back to the original input between each:

tp.reset().tokenize()
tp.reset().to_lowercase().tokenize()
tp.reset().remove_punctuation().tokenize()
tp.reset().tokenize().remove_stopwords()
tp.reset().remove_noise().tokenize()
tp.reset().tokenize().stem()
tp.reset().tokenize().lemmatize(pos=PartOfSpeech.Noun)

Or chained together as a full preprocessing pipeline:

tp.reset().to_lowercase().remove_noise().remove_punctuation().tokenize().remove_stopwords().lemmatize(pos=PartOfSpeech.Verb)

print(tp.tokens)
# [['run', 'runners', 'run', 'quickly', 'run', 'trail'], ['check', 'amaze', 'offer', 'call']]

Methods

TextProcessor holds two pieces of state: .sentences (a list of strings) and .tokens (a list of word lists, populated by .tokenize()). Most methods work on one or the other, and .remove_stopwords(), .stem(), and .lemmatize() require .tokenize() to have been called first — call them too early and they raise ValueError.

Method	Operates on	Description
to_lowercase()	.sentences	Lowercases every sentence.
remove_noise(html=True, urls=True, numbers=True, special_chars=True)	.sentences	Strips HTML tags, URLs, digits, and/or non-letter characters. Each kind of noise can be toggled off.
remove_punctuation()	.sentences	Strips punctuation from each sentence.
tokenize()	.sentences → .tokens	Splits each sentence into a list of word tokens.
remove_stopwords()	.tokens	Drops common stopwords (e.g. "the", "a") from each token list.
stem()	.tokens	Reduces each token to its word stem (Porter stemmer).
lemmatize(pos=PartOfSpeech.Verb)	.tokens	Reduces each token to its dictionary form for the given part of speech.
reset()	both	Restores .sentences to the original constructor input and clears .tokens.

PartOfSpeech is an enum with Noun, Verb, Adjective, and Adverb — pass one to lemmatize(pos=...) to control how words are reduced.

Development

To set up a local environment for running the test suite:

# Windows
powershell -ExecutionPolicy Bypass -File .\scripts\bootstrap.ps1

# macOS/Linux
./scripts/bootstrap.sh

This creates a .venv, installs the package in editable mode, and pulls in dev/test dependencies (nltk, pytest). Then activate it and run the tests:

# Windows
. .\.venv\Scripts\Activate.ps1
pytest -v

# macOS/Linux
source .venv/bin/activate
pytest -v

Contributing

New modules are welcome. Add a package under src/ask/<name>/, cover it with tests, then open a PR.

License

MIT