Skip to main content

utils & misc 4 mlm8s

Project description

mlm8s

miscellaneous for machine-learning in TensorFlow
pip install keras_tuner
pip install mlm8s

Imports:

from mlm8s import ListedPaths, paths2labels, strings2onehot, paths2label_dicts, map_via_dict
from mlm8s import GeneratorDataset, HyperModel, connect
from mlm8s import standardize, normalize, stretch, rotate_deg, flatten, correlate
from mlm8s import create_meshgrid, span_polar_basis
from mlm8s import group_unique
from mlm8s import print_plot_play
from mlm8s import get_sampling_matrix, soft_threshold, deadzone_l1_loss, lasso_l1, norm_l0, huber_loss
from mlm8s import lipschitz_grad_lasso, nesterov_momentum, ista_prox_lasso, fista

Labels from Paths:

from mlm8s import ListedPaths, paths2label_dicts, map_via_dict

### read PATH2DATA and (onehot-)encode by file-containing folder:

paths = ListedPaths(PATH2DATA)*'ogg'
label_dict = paths2label_dicts(paths(), seperators=['/', '.'], indices=[-2, 0])
labels = map_via_dict(paths2labels(paths()), label_dict)

Class - GeneratorDataset:

Enables alternating results of tf.random* -calls, from within the generator of tf.data.Dataset.from_generator*.

kwargs = dict()
kwargs['paths'] = paths
kwargs['label_dict'] = label_dict
kwargs['seperators'] = ['/', '.']
kwargs['indices'] = [-2, 0]

### Feature-Engineering with generator, that can use random variables!!
def engineer_features(paths):
    # use data in path to engineer features
    features = tf.random.uniform(shape=[32, 256, 256, 4])
    return features

kwargs['engineer_features'] = engineer_features

### Creating Features & Labels:
def generate_from_paths(batch_size=1024, **kwargs):
    paths = kwargs['paths']
    seperators = kwargs['seperators']
    indices = kwargs['indices']
    engineer_features = kwargs['engineer_features']
    label_dict = kwargs['label_dict']
    rdm_paths = paths.get_rdm(batch_size)
    features = engineer_features(rdm_paths)
    labels = paths2labels(rdm_paths, seperators, indices)
    labels = map_via_dict(labels, label_dict)
    return features, labels

### Creating tf.data.Dataset, that can generate an infinite number of random batches
###                           from 'generate_from_paths'.
ds = GeneratorDataset(generate_from_paths, batch_size=128, epochs=16, **kwargs)()

for batch in ds.take(1):
    x, y = batch
    print(x.shape, y.shape)

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

mlm8s-1.0.3.3.tar.gz (102.1 kB view details)

Uploaded Source

Built Distribution

mlm8s-1.0.3.3-py3-none-any.whl (107.6 kB view details)

Uploaded Python 3

File details

Details for the file mlm8s-1.0.3.3.tar.gz.

File metadata

  • Download URL: mlm8s-1.0.3.3.tar.gz
  • Upload date:
  • Size: 102.1 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.7.1 CPython/3.11.0 Linux/6.2.0-1018-azure

File hashes

Hashes for mlm8s-1.0.3.3.tar.gz
Algorithm Hash digest
SHA256 b172f4784cfcf23798db7862734f634489d4a1f709b33a5dae6cbff1c12e6924
MD5 0193368d744876cb063d6da1ae520a4c
BLAKE2b-256 d3898805886a6870034c61be896d4087fc00c921382bc88d50a0df28b0971b03

See more details on using hashes here.

File details

Details for the file mlm8s-1.0.3.3-py3-none-any.whl.

File metadata

  • Download URL: mlm8s-1.0.3.3-py3-none-any.whl
  • Upload date:
  • Size: 107.6 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.7.1 CPython/3.11.0 Linux/6.2.0-1018-azure

File hashes

Hashes for mlm8s-1.0.3.3-py3-none-any.whl
Algorithm Hash digest
SHA256 4f8230ac25998880ab63b242662ecbcbace8bbb0ddea8fc9cf536ba8cdafdcbf
MD5 cabc526e492f8cdc109f9fd8969b21ee
BLAKE2b-256 2437bda9c67d23f9d757730227f1d7146e4ae72d8a5fd2c188a334a194ca10c1

See more details on using hashes here.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page