Skip to main content

Data science collective tools

Project description

ForgeBox

Data science comprehensive toolbox 🛠⚔️📦

forgebox logo

Installation

Easy simple installation in 1 line

pip install forgebox

If not specified, you need anaconda3 for most of the tools, python shold be at least >=3.6

Features 🚀 Briefing

This is a tool box with comprehensive utilies, to put it simply, I just hope most of my frequetyly used DIY tools in in place and can be easily installed and imported

Lazy, fast imports 🤯

The following command will import many frequent tools for data science, like pd for pandas, np for numpy, os, json, PIL.Image for image processing

from frogebox.imports import *

No more🚫 following typings

import pandas as pd
import numpy as np
import os
import json
...

Categorical converter

Mapping and converting categorical infomation

from forgebox.category import Category
az = list(map(chr,range(ord("A"), ord("z")+1)))
print(az)
['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I', 'J', 'K', 'L', 'M', 'N', 'O', 'P', 'Q', 'R', 'S', 'T', 'U', 'V', 'W', 'X', 'Y', 'Z', '[', '\\', ']', '^', '_', '`', 'a', 'b', 'c', 'd', 'e', 'f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o', 'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z']
cate_az = Category(az)
cate_az
Category Manager with 58
cate_az.c2i["R"], cate_az.i2c[17]
(17, 'R')
cate_az.c2i[list("ForgeBox")]
array([ 5, 46, 49, 38, 36,  1, 46, 55])
cate_az.i2c[[ 5, 46, 49, 38, 36,  1, 46, 55]]
array(['F', 'o', 'r', 'g', 'e', 'B', 'o', 'x'], dtype='<U1')

Padding missing token

cate_az = Category(az, pad_mst=True)
cate_az.c2i[list("Forge⚡️Box")]
array([ 6, 47, 50, 39, 37,  0,  0,  2, 47, 56])

Get a dataframe of file details under a directory

from forgebox.files import file_detail
file_detail("/Users/xiaochen.zhang/.cache/").sample(5)
<style scoped> .dataframe tbody tr th:only-of-type { vertical-align: middle; }
.dataframe tbody tr th {
    vertical-align: top;
}

.dataframe thead th {
    text-align: right;
}
</style>
path file_type parent depth
36 /Users/xiaochen.zhang/.cache/torch/transformer... json transformers 7
13 /Users/xiaochen.zhang/.cache/torch/transformer... json transformers 7
51 /Users/xiaochen.zhang/.cache/langhuan/task_NER... json task_NER_210121_140513 7
32 /Users/xiaochen.zhang/.cache/torch/transformer... lock transformers 7
58 /Users/xiaochen.zhang/.cache/langhuan/task_Cla... json task_Classify_210128_164710 7

JS style async

from forgebox.asyncing import Async
from time import sleep
def something_time_costing_but_you_dont_want_to_wait(x):
    sleep(x)
    return f"slept for {x} seconds"

def task2_you_will_perfrom_after_the_time_costing_one(x):
    print(f"[result]:\t{x}")
    return 1

print("1111111111")

Async(something_time_costing_but_you_dont_want_to_wait)(2)\
.then(task2_you_will_perfrom_after_the_time_costing_one)\
.catch(print)

print("22222222222")
1111111111
22222222222
[result]:	slept for 2 seconds

HTML in notebook

from forgebox.html import DOM, list_group, list_group_kv

This will map a clear HTML table view of wild tree type json structure/ list

bands = ["police", "headpin", {"ac":"dc"}]
list_group(bands)()
  • police
  • headpin
    • acdc
questions = {
    "question":"answer",
    "another":{
        "deeper question": "answer again"},
    "final":{
        "questions": ["what","is","the","answer", "to",
            ["life", "universe","everything"]]}
}
list_group_kv(questions)()
  • questionanswer
  • another
    • deeper questionanswer again
  • final
    • questions
      • what
      • is
      • the
      • answer
      • to
        • life
        • universe
        • everything

Coding html in python

title = DOM("Title example","h5", kwargs={"style":"color:#3399EE"})
ul = DOM("","ul");
for i in range(5):
    ul = ul.append(DOM(f"Line {i}", "li", kwargs={"style":"color:#EE33DD"}))

title()
ul()
Title example
  • Line 0
  • Line 1
  • Line 2
  • Line 3
  • Line 4

Free style mapping

Works on every value of a complicated dictionary structure (eg. list in dict in list in dict, etc,. 😳)

from forgebox.freemap import FreeMap

# flatten decides if we want to flatten the strucuture
freemap_tool = FreeMap(
    <function/callable applying to every value>,
    <function/callable that filters every value>,
    flatten=True
)

data2 = freemap_tool(data1)

Interactive Widgets

Interactive widgets work with in jupyter notebooks

Search box 🔎 for dataframe

This will create an interactive text input box to search through the pandas dataframe, within the columns you set.

if manual is set to False, the search will respond to each of your key press, it's fast but will suffer terrible user experience if the dataframe is huge in size.

from forgebox.widgets import search_box

search_box(data_df, columns=["col1","col2"], manual=False)

paginate

You can browse through a pandas dataframe like fliping pages 📄.

from forgebox.widgets import paginate

paginate(your_dataframe, page_len=10)

Single button callback

a fully functional page with a single button, this single button is bonded to a function

This is as much code as you need, to build a fully functional interactive page shows sql table from jupyter, that you can:* choose which table to visit* choose how many lines you want to show, (with a slider)

  • configure the where condition with a text box on front end
tablename_list = ["pubmed", "patient", "users", "drugs"]

from forgebox.html import DOM
def show_sql_table(sql_input:str) -> str:
    with engine.connect() as conn:
        df=pd.read_sql(sql_input, con=conn)
    # display the table as html
    DOM(df.to_html(),"div")()

@SingleButton(callback=show_sql_table)
def abc(
    limit:{"typing":int, "default":10, "min":5, "max":20},
    where_condition:{"typing":str, "default": "where 1=1", },
    table:{"typing":list, "options":tablename_list}
):
    return f"SELECT * FROM {table} {where_condition} LIMIT {limit}"

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

forgebox-0.4.18.1.tar.gz (69.8 kB view hashes)

Uploaded Source

Built Distribution

forgebox-0.4.18.1-py3-none-any.whl (91.5 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page