languagemodels 0.1.1

Simple inference for large language models

Language Models

A Python package providing simple building blocks for exploring natural language processing.

Target Audience

This package is designed to be as simple as possible for learners, educators, and hobbyists exploring how large language models intersect with modern software development. The interfaces to this package are all simple functions using standard types. The complexity of large language models is hidden from view while providing free local inference using light-weight, open models. All included models are free for educational use, no API keys are required, and all inference is performed locally by default.

Installation

This package can be installed using the following command:

pip install languagemodels

Example Usage

Here are some usage examples as Python REPL sessions. This should work in the REPL, notebooks, or in traditional scripts and applications.

Text Completions

>>> import languagemodels as lm

>>> lm.complete("She hid in her room until")
'she was sure she was safe'

Instruction Following

>>> import languagemodels as lm

>>> lm.do("Translate to English: Hola, mundo!")
'Hello, world!'

>>> lm.do("What is the capital of France?")
'paris'

Classification

>>> import languagemodels as lm

>>> lm.classify("Language models are useful", "positive", "negative")
'positive'

Semantic Search

Semantic search is provided to retrieve documents that may provide helpful context from a document store.

>>> import languagemodels as lm

>>> lm.store_doc("Mars is a planet")
>>> lm.store_doc("The sun is hot")
>>> lm.load_doc("What is Mars?")
'Mars is a planet'

Extractive Question Answering

>>> import languagemodels as lm

>>> lm.extract_answer("What color is the ball?", "There is a green ball and a red box")
'green'

External Retrieval

>>> import languagemodels as lm

>>> lm.fetch_wiki('Chemistry')
'Chemistry is the scientific study...

>>> lm.fetch_weather(41.8, -87.6)
'Partly cloudy with a chance of rain...

Misc Text Tools

>>> import languagemodels as lm

>>> get_date()
'Friday, May 12, 2023 at 09:27AM'

Full documentation

System Requirements

This package uses large language models to complete natural language prompts. These models are resource intensive in terms of initial network bandwidth, memory usage, and compute. This package defaults to downloading model weights and computing outputs using the local CPU. The base resource requirements for this are:

	Base (Default)	Recommended
CPU	Any	AVX512 support
GPU	Not required	Not required
RAM	512MB	16GB
Storage	512MB	4GB

This package will use approximately 5GB of network data to download models initially.

Projects Ideas

This package can be used to do the heavy lifting for a number of learning projects:

• Basic chatbot
• Chatbot with information retrieval
• Chatbot with access to real-time information
• Tool use
• Text classification
• Extractive question answering
• Semantic search
• Document question answering

Several example programs and notebooks are included in the examples directory.

Attribution

• CTranslate2
• LaMini-Flan-T5
• Flan-T5