BERT model fine-tuned on chilean STEM lessons
Project description
BERT-STEM
BERT model fine-tuned on Science Technology Engineering and Mathematics (STEM) lessons.
Install:
To install from pip:
pip install bertstem
Quickstart
To encode sentences :
from BERT_STEM.BertSTEM import *
bert = BertSTEM()
# Example dataframe with text in spanish
data = {'col_1': [3, 2, 1],
'col_2': ['hola como estan', 'alumnos queridos', 'vamos a hablar de matematicas']}
df = pd.DataFrame.from_dict(data)
# Encode sentences using BertSTEM:
bert._encode_df(df, column='col_2', encoding='sum')
To classify sentences with COPUS models:
from BERT_STEM.BertSTEM import *
# Download BERT for classification (guiding/presenting/administration)
bert_classification = BertSTEMForTextClassification(2, model_name = 'pablouribe/bertstem-copus-guiding')
# Example dataframe with text in spanish
data = {'col_1': [3, 2, 1],
'col_2': ['hola como estan', 'alumnos queridos', 'vamos a hablar de matematicas']}
df = pd.DataFrame.from_dict(data)
# Classify sentences using BertSTEM for COPUS (Guiding):
bert_classification.predict(df,'col_2')
To use it from HuggingFace:
from BERT_STEM.Encode import *
import pandas as pd
import transformers
# Download spanish BERTSTEM:
model = transformers.BertModel.from_pretrained("pablouribe/bertstem")
# Download spanish tokenizer:
tokenizer = transformers.BertTokenizerFast.from_pretrained("dccuchile/bert-base-spanish-wwm-uncased",
do_lower_case=True,
add_special_tokens = False)
# Example dataframe with text in spanish
data = {'col_1': [3, 2, 1],
'col_2': ['hola como estan', 'alumnos queridos', 'vamos a hablar de matematicas']}
df = pd.DataFrame.from_dict(data)
# Encode sentences using BertSTEM:
sentence_encoder(df, model, tokenizer, column = 'col_2', encoding = 'sum')
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
bertstem-0.0.24.tar.gz
(6.7 kB
view hashes)
Built Distribution
Close
Hashes for bertstem-0.0.24-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 41eeb95ffdd5c4acb3e00265b66ec329f9bbdbd40c6a3e17f811ef95b4462472 |
|
MD5 | 495bc697a3b2ce22552883b163d2efea |
|
BLAKE2b-256 | c40b5a41abe4bb2b2504b1a892a4a8b784468e5c479f4dd90c45dfd934e9b7ca |