pyspace is a tool set of data science python functions
Project description
pyspace
installation
git clone -b pyspace_resources https://github.com/sahinbatmaz/resources.git pyspace_resources
cd pyspace_resources
pip install .
pip install pyspace-toolkit
# UPGRADE
pip install pyspace-toolkit --no-deps --force-reinstall
requirements
'sklearn','pandas','numpy',
'matplotlib','seaborn',
'rasa>=1.10.3','tensorflow',
'lightgbm','xgboost',
'spacy>=2.3.0','spacymoji',
'stanza','nlpcube',
'fuzzywuzzy','jellyfish>=0.8.2','fuzzy_sequence_matcher','fastdtw',
'tabulate','tqdm','jsonlines',
'sklearn-hierarchical-classification','JPype1','MiniSom'
text classification
from pyspace.data.analyse import get_cv_predictions
from pyspace.data.metrics import get_confusion_report
# train # pandas DataFrame # required columns : ['Text', 'Intent']
outdf_avg, y_prob_matrix_avg_df = get_cv_predictions(train, mode='text', cv=5, model_repeat_count=3, n_jobs=5, verbose=11,)
# outdf_avg # required columns : ['Text', 'Intent', 'y_pred'], additional columns : ['y_prob', 'y_2nd_pred', ...]
get_confusion_report(outdf_avg, excel_file_path='./dataset_analysis.xlsx', y_prob_matrix_df=None)
text gcn
import pandas as pd
train = pd.read_csv('dataset.csv')
train.columns = ['text', 'label']
from sklearn.model_selection import StratifiedKFold
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)
train_idx, test_idx = list(skf.split(list(train.index), train['label'].values))[4]
train['tot'] = None
train.at[train_idx, 'tot'] = 'train'
train.at[test_idx, 'tot'] = 'test'
from pyspace.nlp.models.text_gcn.text_gcn import TextGCN_TransductiveClassifier
from pyspace.nlp.models.text_gcn.fast_text_gcn_norank import FastTextGCN_InductiveClassifier
fasttextgcn = FastTextGCN_InductiveClassifier(verbose=1, )
fasttextgcn.train(train, validation_ratio=0.0, batch_size=256, epochs=80, learning_rate=0.01)
future work
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distributions
No source distribution files available for this release.See tutorial on generating distribution archives.
Built Distribution
File details
Details for the file pyspace_toolkit-1.1.25.294-py3-none-any.whl
.
File metadata
- Download URL: pyspace_toolkit-1.1.25.294-py3-none-any.whl
- Upload date:
- Size: 347.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.7.1 importlib_metadata/4.10.0 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9
File hashes
Algorithm | Hash digest | |
---|---|---|
SHA256 | 809b972b04ccfe763ac0186188ea9c55024e4bf736a04374f840624cb2dd189a |
|
MD5 | c5050cececb0e3904f5b32a6f17c630e |
|
BLAKE2b-256 | 79f840ddec7afe130f1471e07d62d705398f073952c43df66f11cd75d1fb6a8a |