Skip to main content

An open source framework for in-context learning.

Project description


OverviewInstallationPaperExamplesDocsCitation

version

Overview

OpenICL provides an easy interface for in-context learning, with many state-of-the-art retrieval and inference methods built in to facilitate systematic comparison of LMs and fast research prototyping. Users can easily incorporate different retrieval and inference methods, as well as different prompt instructions into their workflow.

What's News

  • v0.1.8 Support LLaMA and self-consistency

Installation

Note: OpenICL requires Python 3.8+

Using Pip

pip install openicl

Installation for local development:

git clone https://github.com/Shark-NLP/OpenICL
cd OpenICL
pip install -e .

Quick Start

Following example shows you how to perform ICL on sentiment classification dataset. More examples and tutorials can be found at examples

Step 1: Load and prepare data

from datasets import load_dataset
from openicl import DatasetReader

# Loading dataset from huggingface
dataset = load_dataset('gpt3mix/sst2')

# Define a DatasetReader, with specified column names where input and output are stored.
data = DatasetReader(dataset, input_columns=['text'], output_column='label')

Step 2: Define the prompt template (Optional)

from openicl import PromptTemplate
tp_dict = {
    0: "</E>Positive Movie Review: </text>",
    1: "</E>Negative Movie Review: </text>" 
}

template = PromptTemplate(tp_dict, {'text': '</text>'}, ice_token='</E>')

The placeholder </E> and </text> will be replaced by in-context examples and testing input, respectively. For more detailed information about PromptTemplate (such as string-type template) , please see tutorial1.

Step 3: Initialize the Retriever

from openicl import TopkRetriever
# Define a retriever using the previous `DataLoader`.
# `ice_num` stands for the number of data in in-context examples.
retriever = TopkRetriever(data, ice_num=8)

Here we use the popular TopK method to build the retriever.

Step 4: Initialize the Inferencer

from openicl import PPLInferencer
inferencer = PPLInferencer(model_name='distilgpt2')

Step 5: Inference and scoring

from openicl import AccEvaluator
# the inferencer requires retriever to collect in-context examples, as well as a template to wrap up these examples.
predictions = inferencer.inference(retriever, ice_template=template)
# compute accuracy for the prediction
score = AccEvaluator().score(predictions=predictions, references=data.references)
print(score)

Docs

(updating...)

OpenICL Documentation

Citation

If you find this repository helpful, feel free to cite our paper:

@article{wu2023openicl,
  title={OpenICL: An Open-Source Framework for In-context Learning},
  author={Zhenyu Wu, Yaoxiang Wang, Jiacheng Ye, Jiangtao Feng, Jingjing Xu, Yu Qiao, Zhiyong Wu},
  journal={arXiv preprint arXiv:2303.02913},
  year={2023}
}

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

openicl-0.1.8.tar.gz (30.0 kB view details)

Uploaded Source

Built Distribution

openicl-0.1.8-py3-none-any.whl (45.9 kB view details)

Uploaded Python 3

File details

Details for the file openicl-0.1.8.tar.gz.

File metadata

  • Download URL: openicl-0.1.8.tar.gz
  • Upload date:
  • Size: 30.0 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.13

File hashes

Hashes for openicl-0.1.8.tar.gz
Algorithm Hash digest
SHA256 65b03cc90351201c1493726d9b90a1356acf02b1b509ab8d7e89656b4c1bb301
MD5 aaa9840a7a0dbeb8b1134c0c9edcb92a
BLAKE2b-256 45b90b41d78adad75c03a788bfab1fea3cab7aed9fcc7e250fda25d2b6796d7f

See more details on using hashes here.

File details

Details for the file openicl-0.1.8-py3-none-any.whl.

File metadata

  • Download URL: openicl-0.1.8-py3-none-any.whl
  • Upload date:
  • Size: 45.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.13

File hashes

Hashes for openicl-0.1.8-py3-none-any.whl
Algorithm Hash digest
SHA256 46d86b74b922f5ed1f869344f73ed0fa7d37978da8fbfd963ffee0838c1f21b9
MD5 855b05c2e64585f82054f7a45147a7f0
BLAKE2b-256 dcd427e7823d9a022e6caed9a8b50fa1b53f931a5a07d6edde4122cecdb57b25

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page