Skip to main content

A few ready-to use python tools for machine learning

Project description

FlashPythonToolbox

A few ready-to use python tools for machine learning

Install

pip install flashtool

Logger (to file)

For every print function, we print a copy to file without changing the whole file.

Logger(filepath = "./log.txt", mode = "w", stdout = None)
  • filepath: output path. Default: "./log.txt"
  • mode: write mode, e.g., w,w+,a,a+. Default: "w"
  • stdout: capture which print source. Default: "sys.stdout"

Example:

from flashtool import Logger
import sys
sys.stdout = Logger("path/to/log/file/log.txt")
sys.stdout = Logger("path/to/log/file/log.txt","w")
sys.stdout = Logger("path/to/log/file/log.txt","w", sys.stdout)

Email Module

Default Sender: gtflashauto@gmail.com. You need a password for this.

# Function api
def send_email(port = 587,password = None,
        sender_email = "gtflashauto@gmail.com",
        smtp_server="smtp.gmail.com",
        receiver_email = None,
        subject="Subject",
        message="Hello! ",
        attachment=None):
    ...

# Example
from flashtool import send_email
send_email(receiver_email="xxx@gmail.com", subject="Hi", message="first email")

Track Job

Track your job. Send you an email when it is done.

Tracking by PID

  • Step 1: find your job pid by ps, e.g., ps aux | grep python
  • Step 2:
from flashtool import trackpid
trackpid(12345,5,"xxx@gmail.com")
trackpid([1357,2468],5,"xxx@gmail.com")

API:

def trackpid(pid, checktime=1, receiver_email=None, sender_email="gtflashauto@gmail.com", msg=""):
  ...
  • pid: pid or pids
  • checktime: check pid every checktime seconds
  • msg: additional message

Pytorch Utils

GPU Usage

import flashtool as flash; flash.check_torch_memory(brief=True)
  • brief: short message (default: Flase)

Auto Script Running

import flashtool as flash;
basescript='script/run.sh'
config={
    "DATA": "subset5"
}
flash.run_script(basescript, config)

Profile Timing Line by Line

from flashtool import line_time_profiler 

@line_time_profiler()
def f(n=1000):
    a = [1 for _ in range(n)]
    b = sum(a)

f()
f(10000)

class C:
    @line_time_profiler()
    def f(self, n=1000):
        a = [1 for _ in range(n)]
        b = sum(a)

ins = C()
ins.f()
ins.f(10000)


@line_time_profiler(stop=True)
def f(n=1000):
    a = [1 for _ in range(n)]
    b = sum(a)

f()

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

flashtool-0.0.10.tar.gz (4.7 kB view details)

Uploaded Source

Built Distribution

flashtool-0.0.10-py3-none-any.whl (7.1 kB view details)

Uploaded Python 3

File details

Details for the file flashtool-0.0.10.tar.gz.

File metadata

  • Download URL: flashtool-0.0.10.tar.gz
  • Upload date:
  • Size: 4.7 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.7.7

File hashes

Hashes for flashtool-0.0.10.tar.gz
Algorithm Hash digest
SHA256 4251325430c3a131a913a3f16ddec9fb96e06f02932d278f7d41e987130683b8
MD5 2a4c497a3f959739f243ba6f76c7310b
BLAKE2b-256 a31bfc6d6636afd7b0a8d3dda6eed7c15615e4eed3ba40b7488434c1d44915a6

See more details on using hashes here.

File details

Details for the file flashtool-0.0.10-py3-none-any.whl.

File metadata

  • Download URL: flashtool-0.0.10-py3-none-any.whl
  • Upload date:
  • Size: 7.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.24.0 setuptools/41.2.0 requests-toolbelt/0.9.1 tqdm/4.48.2 CPython/3.7.7

File hashes

Hashes for flashtool-0.0.10-py3-none-any.whl
Algorithm Hash digest
SHA256 ee9ff9b00f5fe74c1bb1a3a41444966030b5bf35316b23baebe194723656d5e1
MD5 22222df72a62c3aaedac3a96c00cf761
BLAKE2b-256 a61d8e00e398adce201b8f156842838ce47e78a7b73a9e11f05c7b6eaa7923c9

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page