Skip to main content

Save your files with current date time appended; load the newest back without specifying file name everytime

Project description

This is a simple script that has only two functions for one purpose. It allows you to save files with date time appended behind the filename but in front of the extension, like sldt.s(variable, 'filename.csv') will save variable to a file "filename_1912181212.csv". Everytime you want to load it back, use sldt.l('filename.csv'), it will automatically find the newest file.

This is useful when you want to save some intermediate result during running your code. But be aware the saved files will keep growing everytime you run.

Feel free to download/edit this script, and maybe share your version with me!

Installation

pip install sldt

Supported File Types

It supports only 4 common file types now showed below. Basically it just calls like pd.DataFrame.to_csv() as you already familiar.

import sldt
sldt.__version__
'1.0.2'
sldt.SUPPORTED_EXT
['.pkl', '.csv', '.png', '.txt']

Demo

# launch logger to see info when saving and loading files
import logging
logging.getLogger().setLevel(logging.INFO)

anything -> pkl

a_list = ['a', 0.1, False]
# save
sldt.s(a_list, 'output/a_list.pkl')

# save the second file for demo
import time
time.sleep(60)
a_list = ['b', 0.2, True]
sldt.s(a_list, 'output/a_list.pkl')

# load the newest (which is the later one) back
a_list = sldt.l('output/a_list.pkl')
a_list
INFO:root:output/a_list_1912240334.pkl saved
INFO:root:output/a_list_1912240335.pkl saved
INFO:root:output/a_list_1912240335.pkl loaded





['b', 0.2, True]

pandas dataframe -> csv

import pandas as pd
df = pd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
sldt.s(df, 'output/df.csv')

df = sldt.l('output/df.csv')
df
INFO:root:output/df_1912240335.csv saved
INFO:root:output/df_1912240335.csv loaded
col1 col2
0 1 3
1 2 4
you can pass any argument as to `pd.DataFrame.to_csv()`, if there's no arguments in `save_dt`, the default will be `index=False`.

For example, save_dt(df, 'output/df.csv', sep=';')

figure -> png

import seaborn as sns
exercise = sns.load_dataset("exercise")
g = sns.catplot(x="time", y="pulse", hue="kind", data=exercise)

sldt.s(g, 'output/g.png')
INFO:root:output/g_1912240335.png saved

Display it using ![](output/g_1912240037.png) in markdown.

the default arguments will be dpi=600, bbox_inches='tight'. And it will try to close the fig after saving the file.

string -> txt

text = '''some random sentences
here
and there'''
sldt.s(text, 'output/text.txt')

text = sldt.l('output/text.txt')
text
INFO:root:output/text_1912240335.txt saved
INFO:root:output/text_1912240335.txt loaded





'some random sentences\nhere\nand there'

helper functions

you can save your own file type and load it back using append_dt and find_newest

import pickle

output_filename = sldt.append_dt('output/sth.pkl', datetime_format="%y%m%d%H%M")[0]
with open(output_filename, 'wb') as f:
    pickle.dump('some strings', f)
newest_file = sldt.find_newest('output/sth.pkl')[0]
with open(newest_file, 'rb') as f:
    sth = pickle.load(f)
sth
'some strings'

find_newest returns a tuplet (filename, extension)

sldt.find_newest('output/text.txt')
('output/text_1912240335.txt', '.txt')

It can be served as checking whether file is exist

try:
    sldt.find_newest('output/text.txt')
    print('file exist')
except:
    print('no file exist')
file exist

Similarly, load only if you haven't save the needed file

try:
    sldt.l('output/result.csv')
except:
    # do some calculation
    result = pd.DataFrame({'col1': [1, 2], 'col2': [3, 4]})
    # save calculation result
    sldt.s(result, 'output/result.csv')
INFO:root:output/result_1912240335.csv saved

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Files for sldt, version 1.0.4
Filename, size File type Python version Upload date Hashes
Filename, size sldt-1.0.4-py3-none-any.whl (5.1 kB) File type Wheel Python version py3 Upload date Hashes View hashes

Supported by

Elastic Elastic Search Pingdom Pingdom Monitoring Google Google BigQuery Sentry Sentry Error logging AWS AWS Cloud computing DataDog DataDog Monitoring Fastly Fastly CDN DigiCert DigiCert EV certificate StatusPage StatusPage Status page