Skip to main content

No project description provided

Project description

spacy-setfit

This repository contains an easy and intuitive approach to using SetFit in combination with spaCy.

Installation

Before using spaCy with SetFit, make sure you have the necessary packages installed. You can install them using pip:

pip install spacy spacy-setfit

Additionally, you will might want to download a spaCy model, for example:

python -m spacy download en_core_web_sm

Getting Started

To use spaCy with SetFit use the following code:

import spacy

# Create some example data
train_dataset = {
    "inlier": ["This text is about chairs.",
               "Couches, benches and televisions.",
               "I really need to get a new sofa."],
    "outlier": ["Text about kitchen equipment",
                "This text is about politics",
                "Comments about AI and stuff."]
}

# Load the spaCy language model:
nlp = spacy.load("en_core_web_sm")

# Add the "spacy_setfit" pipeline component to the spaCy model, and configure it with SetFit parameters:
nlp.add_pipe("spacy_setfit", config={
    "pretrained_model_name_or_path": "paraphrase-MiniLM-L3-v2",
    "setfit_trainer_args": {
        "train_dataset": train_dataset
    }
})
doc = nlp("I really need to get a new sofa.")
doc.cats
# {'inlier': 0.902350975129, 'outlier': 0.097649024871}

The code above processes the input text with the spaCy model, and the doc.cats attribute returns the predicted categories and their associated probabilities.

That's it! You have now successfully integrated spaCy with SetFit for text categorization tasks. You can further customize and train the model using additional data or adjust the SetFit parameters as needed.

Feel free to explore more features and documentation of spaCy and SetFit to enhance your text classification projects.

setfit_trainer_args

The setfit_trainer_args are a simplified version of the official args from the SetFit library.

Arguments

  • train_dataset (Union[dict, Dataset]): The training dataset to be used by the SetFitTrainer. It can be either a dictionary or a Dataset object.

  • eval_dataset (Union[dict, Dataset], optional): The evaluation dataset to be used by the SetFitTrainer. It can be either a dictionary or a Dataset object. Defaults to None.

  • metric (Union[str, Callable[["Dataset", "Dataset"], Dict[str, float]]], optional): The metric to be used for evaluation. It can be either a string or a callable. Defaults to "accuracy".

  • metric_kwargs (Optional[Dict[str, Any]], optional): Additional keyword arguments to pass to the metric function. Defaults to None.

  • loss_class (losses.CosineSimilarityLoss, optional): The loss function to be used for training. Defaults to losses.CosineSimilarityLoss.

  • num_iterations (int, optional): The number of iterations to train the model. Defaults to 20.

  • num_epochs (int, optional): The number of epochs to train the model. Defaults to 1.

  • learning_rate (float, optional): The learning rate for the optimizer. Defaults to 2e-5.

  • batch_size (float, optional): The batch size for training. Defaults to 16.

  • seed (int, optional): The random seed for reproducibility. Defaults to 42.

  • column_mapping (dict, optional): A mapping dictionary that specifies how to map input columns to model inputs. Defaults to None.

  • use_amp (bool, optional): Whether to use Automatic Mixed Precision (AMP) for training. Defaults to False.

  • warmup_proportion (float, optional): The proportion of training steps to perform linear learning rate warmup for. Defaults to 0.1.

  • distance_metric (Callable, optional): The distance metric to be used for training. Defaults to BatchHardTripletLossDistanceFunction.cosine_distance.

  • margin (float, optional): The margin for the triplet loss function. Defaults to 0.25.

  • samples_per_label (int, optional): The number of samples per label to be used for training. Defaults to 2.

Please note that the above documentation provides an overview of the arguments and their purpose. For more detailed information and usage examples, it is recommended to refer to the official SetFit library documentation or any specific implementation details provided by the library.

Usage

To use the setfit_trainer_args, you can create a dictionary with the desired values for the arguments. Here's an example:

setfit_trainer_args = {
    "train_dataset": train_data,
    "eval_dataset": eval_data,
    "num_iterations": 20,
    "num_epochs": 1,
    "learning_rate": 2e-5,
    "batch_size": 16,
    "seed": 42,
    "column_mapping": column_map,
    "use_amp": False
}

setfit_from_pretrained_args

The setfit_from_pretrained_args are a simplified version of the official args from the SetFit library and Hugging Face transformers.

Arguments

  • pretrained_model_name_or_path (str or Path): This argument specifies the model to be loaded. It can be either:

    • The model_id (string) of a model hosted on the Hugging Face Model Hub, e.g., bigscience/bloom.
    • A path to a directory containing model weights saved using the save_pretrained method of PreTrainedModel, e.g., ../path/to/my_model_directory/.
  • revision (str, optional): The revision of the model on the Hub. It can be a branch name, a git tag, or any commit id. Defaults to the latest commit on the main branch.

  • force_download (bool, optional): Whether to force (re-)downloading the model weights and configuration files from the Hub, overriding the existing cache. Defaults to False.

  • resume_download (bool, optional): Whether to delete incompletely received files and attempt to resume the download if such a file exists. Defaults to False.

  • proxies (Dict[str, str], optional): A dictionary of proxy servers to use by protocol or endpoint. It is used for requests made during the downloading process. For example: proxies = {'http': 'foo.bar:3128', 'http://hostname': 'foo.bar:4012'}

  • token (str or bool, optional): The token to use as HTTP bearer authorization for remote files. By default, it uses the token cached when running huggingface-cli login.

  • cache_dir (str or Path, optional): The path to the folder where cached files are stored.

  • local_files_only (bool, optional): If True, it avoids downloading the file and returns the path to the local cached file if it exists. Defaults to False.

  • model_kwargs (Dict, optional): Additional keyword arguments to pass to the model during initialization.

Please note that the above documentation provides an overview of the arguments and their purpose. For more detailed information and usage examples, it is recommended to refer to the official SetFit library documentation or any specific implementation details provided by the library.

Usage

To use the setfit_from_pretrained_args, you can create a dictionary with the desired values for the arguments. Here's an example:

setfit_from_pretrained_args = {
    'pretrained_model_name_or_path': '',  # str or Path
    'revision': None,  # str, optional
    'force_download': False,  # bool, optional
    'resume_download': False,  # bool, optional
    'proxies': None,  # Dict[str, str], optional
    'token': None,  # str or bool, optional
    'cache_dir': None,  # str or Path, optional
    'local_files_only': False,  # bool, optional
    'model_kwargs': None  # Dict, optional
}

Pretrained SetFit models

You can also use pre-trained SetFit models.

import spacy

# Load the spaCy language model:
nlp = spacy.load("en_core_web_sm")

# Add the "spacy_setfit" pipeline component to the spaCy model
nlp.add_pipe("spacy_setfit", config={
    "pretrained_model_name_or_path": "lewtun/my-awesome-setfit-model",
})
nlp("I really need to get a new sofa.")

Saving and Loading models

You can use the pickle module in Python to save and load instances of the pre-trained pipeline. pickle allows you to serialize Python objects, including custom classes, into a binary format that can be saved to a file and loaded back into memory later. Here's an example of how to save and load using pickle:

import pickle

nlp = ...

# Save nlp pipeline
with open("my_cool_model.pkl", "wb") as file:
    pickle.dump(nlp, file)

# Load nlp pipeline
with open("my_cool_model.pkl", "rb") as file:
    nlp = pickle.load(file)

doc = nlp("I really need to get a new sofa.")
doc.cats
# {'inlier': 0.902350975129, 'outlier': 0.097649024871}

Logo Reference

Quotation by Adrien Coquet from Noun Project

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

spacy_setfit-0.1.5.tar.gz (10.9 kB view details)

Uploaded Source

Built Distribution

spacy_setfit-0.1.5-py3-none-any.whl (12.4 kB view details)

Uploaded Python 3

File details

Details for the file spacy_setfit-0.1.5.tar.gz.

File metadata

  • Download URL: spacy_setfit-0.1.5.tar.gz
  • Upload date:
  • Size: 10.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.18

File hashes

Hashes for spacy_setfit-0.1.5.tar.gz
Algorithm Hash digest
SHA256 dca959fc0f6ef8e4fec309f9bd8c9978f2a6c486ed6fe95356c9e9213bb76156
MD5 d157109d4a8645ab8bc525e7c32efb61
BLAKE2b-256 5525d84d7287d75bca4321b05993101845ddc7983db33217a90e3b6d74b91abc

See more details on using hashes here.

File details

Details for the file spacy_setfit-0.1.5-py3-none-any.whl.

File metadata

  • Download URL: spacy_setfit-0.1.5-py3-none-any.whl
  • Upload date:
  • Size: 12.4 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/4.0.2 CPython/3.9.18

File hashes

Hashes for spacy_setfit-0.1.5-py3-none-any.whl
Algorithm Hash digest
SHA256 9c8bbb486db101fc7879cd0a26ab8ce71addc010c52186022e6b67db5db2467e
MD5 362b4eb792ea499241b684966848c441
BLAKE2b-256 8ba05a56e52ba3f864dd62f0417ac0bbe1117cd29b94d90a6ba09853e78cdf60

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page