Simple inference for large language models
Project description
Language Models
A Python package providing simple building blocks for exploring natural language processing.
Target Audience
This package is designed to be as simple as possible for learners, educators, and hobbyists exploring how large language models intersect with modern software development. The interfaces to this package are all simple functions using standard types. The complexity of large language models is hidden from view while providing free local inference using light-weight, open models. All included models are free for educational use, no API keys are required, and all inference is performed locally by default.
Installation
This package can be installed using the following command:
pip install languagemodels
Example Usage
Here are some usage examples as Python REPL sessions. This should work in the REPL, notebooks, or in traditional scripts and applications.
Text Completions
>>> import languagemodels as lm
>>> lm.complete("She hid in her room until")
'she was sure she was safe'
Instruction Following
>>> import languagemodels as lm
>>> lm.do("Translate to English: Hola, mundo!")
'Hello, world!'
>>> lm.do("What is the capital of France?")
'paris'
Classification
>>> import languagemodels as lm
>>> lm.classify("Language models are useful", "positive", "negative")
'positive'
Semantic Search
Semantic search is provided to retrieve documents that may provide helpful context from a document store.
>>> import languagemodels as lm
>>> lm.store_doc("Mars is a planet")
>>> lm.store_doc("The sun is hot")
>>> lm.load_doc("What is Mars?")
'Mars is a planet'
Extractive Question Answering
>>> import languagemodels as lm
>>> lm.extract_answer("What color is the ball?", "There is a green ball and a red box")
'green'
External Retrieval
>>> import languagemodels as lm
>>> lm.fetch_wiki('Chemistry')
'Chemistry is the scientific study...
>>> lm.fetch_weather(41.8, -87.6)
'Partly cloudy with a chance of rain...
Misc Text Tools
>>> import languagemodels as lm
>>> get_date()
'Friday, May 12, 2023 at 09:27AM'
System Requirements
This package uses large language models to complete natural language prompts. These models are resource intensive in terms of initial network bandwidth, memory usage, and compute. This package defaults to downloading model weights and computing outputs using the local CPU. The base resource requirements for this are:
Base (Default) | Recommended | |
---|---|---|
CPU | Any | AVX512 support |
GPU | Not required | Not required |
RAM | 512MB | 16GB |
Storage | 512MB | 4GB |
This package will use approximately 5GB of network data to download models initially.
Projects Ideas
This package can be used to do the heavy lifting for a number of learning projects:
- Basic chatbot
- Chatbot with information retrieval
- Chatbot with access to real-time information
- Tool use
- Text classification
- Extractive question answering
- Semantic search
- Document question answering
Several example programs and notebooks are included in the examples
directory.
Attribution
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for languagemodels-0.1.0-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 075d221e2a30e2688dc08fbadaf5225d03c0ce374cd5a0924bba2700d7ed20f0 |
|
MD5 | e1039d828a2c08cb4bd3a64e30bed65c |
|
BLAKE2b-256 | 25cf15166ef4983775c991d7a60d5fe8727a043e16d0810169eb3f6134c606dc |