Skip to main content

pyspace is a tool set of data science python functions

Project description

pyspace

installation

git clone -b pyspace_resources https://github.com/sahinbatmaz/resources.git pyspace_resources
cd pyspace_resources
pip install .

pip install pyspace-toolkit
# UPGRADE
pip install pyspace-toolkit --no-deps --force-reinstall

requirements

    'sklearn','pandas','numpy',
    'matplotlib','seaborn',
    'rasa>=1.10.3','tensorflow',
    'lightgbm','xgboost',
    'spacy>=2.3.0','spacymoji',
    'stanza','nlpcube',
    'fuzzywuzzy','jellyfish>=0.8.2','fuzzy_sequence_matcher','fastdtw',
    'tabulate','tqdm','jsonlines',
    'sklearn-hierarchical-classification','JPype1','MiniSom'

text classification

from pyspace.data.analyse import get_cv_predictions
from pyspace.data.metrics import get_confusion_report

# train # pandas DataFrame # required columns : ['Text', 'Intent']
outdf_avg, y_prob_matrix_avg_df =  get_cv_predictions(train, mode='text', cv=5, model_repeat_count=3, n_jobs=5, verbose=11,)

# outdf_avg # required columns : ['Text', 'Intent', 'y_pred'], additional columns : ['y_prob', 'y_2nd_pred', ...] 
get_confusion_report(outdf_avg, excel_file_path='./dataset_analysis.xlsx', y_prob_matrix_df=None)

text gcn

import pandas as pd
train = pd.read_csv('dataset.csv')
train.columns = ['text', 'label']

from sklearn.model_selection import StratifiedKFold
skf = StratifiedKFold(n_splits=5, shuffle=True, random_state=0)
train_idx, test_idx = list(skf.split(list(train.index), train['label'].values))[4]

train['tot'] = None
train.at[train_idx, 'tot'] = 'train'
train.at[test_idx, 'tot'] = 'test'
from pyspace.nlp.models.text_gcn.text_gcn import TextGCN_TransductiveClassifier
from pyspace.nlp.models.text_gcn.fast_text_gcn_norank import FastTextGCN_InductiveClassifier

fasttextgcn = FastTextGCN_InductiveClassifier(verbose=1, )
fasttextgcn.train(train, validation_ratio=0.0, batch_size=256, epochs=80, learning_rate=0.01)


future work

Project details


Release history Release notifications | RSS feed

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distributions

No source distribution files available for this release.See tutorial on generating distribution archives.

Built Distribution

pyspace_toolkit-1.1.25.294-py3-none-any.whl (347.1 kB view details)

Uploaded Python 3

File details

Details for the file pyspace_toolkit-1.1.25.294-py3-none-any.whl.

File metadata

  • Download URL: pyspace_toolkit-1.1.25.294-py3-none-any.whl
  • Upload date:
  • Size: 347.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.7.1 importlib_metadata/4.10.0 pkginfo/1.8.2 requests/2.26.0 requests-toolbelt/0.9.1 tqdm/4.62.3 CPython/3.9.9

File hashes

Hashes for pyspace_toolkit-1.1.25.294-py3-none-any.whl
Algorithm Hash digest
SHA256 809b972b04ccfe763ac0186188ea9c55024e4bf736a04374f840624cb2dd189a
MD5 c5050cececb0e3904f5b32a6f17c630e
BLAKE2b-256 79f840ddec7afe130f1471e07d62d705398f073952c43df66f11cd75d1fb6a8a

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page