Metamorphic relations library
Project description
Installation
pip install metamorphic_relations
Get Started - MNIST Example
For more information on all classes and functions see https://3200-metamorphic-relations-lib.readthedocs.io/
Create Data Object
Stores the data that the metamorphic relations (MRs) are used on
from keras.datasets import mnist
from metamorphic_relations.Data import Data
MNIST = mnist.load_data()
data = Data(train_x=MNIST[0][0], train_y=MNIST[0][1],
test_x=MNIST[1][0], test_y=MNIST[1][1], max_y=10)
Create Keras Model
A deep learning keras model to be trained for MNIST image classification
from keras import Sequential
from keras.layers import Dense
import keras.layers as layers
MNIST_model = Sequential()
MNIST_model.add(Dense(128, input_shape=MNIST[0][0][0].flatten().shape))
MNIST_model.add(layers.LeakyReLU())
MNIST_model.add(Dense(256))
MNIST_model.add(layers.LeakyReLU())
MNIST_model.add(Dense(256))
MNIST_model.add(layers.LeakyReLU())
MNIST_model.add(Dense(data.max_y))
MNIST_model.compile(optimizer="adam", loss="mse", metrics=['accuracy'])
Create Metamorphic Relations (MRs)
A list of domain specific MRs (DSMRs) is created for the given dataset using the transforms provided by ImageMR
from metamorphic_relations.MR import MR
from metamorphic_relations.ImageMR import ImageMR
DSMRs = []
DSMRs += MR.for_all_labels(lambda x: ImageMR.rotate_transform(x, angle=180), [0, 1, 6, 8, 9], [0, 1, 9, 8, 6], "Rotate 180 degrees")
DSMRs += MR.for_all_labels(lambda x: ImageMR.flip_vertical_transform(x), [0, 1, 3, 8], name="Flip vertical")
DSMRs += MR.for_all_labels(lambda x: ImageMR.flip_horizontal_transform(x), [0, 1, 8], name="Flip horizontal")
The list of Generic MRs (GMRs) for images is provided by ImageMR
GMRs = ImageMR.get_image_GMRs()
Create MRModel
The object from which the MRs can be applied to the data and the model can be trained
from metamorphic_relations.MRModel import MRModel
MR_model = MRModel(data=data, model=MNIST_model,
transform_x=lambda x: x.reshape((x.shape[0], -1)),
GMRs=GMRs, DSMRs=DSMRs)
Use MRModel to Create Additional Data
The list of MRs are converted to MR objects which can be used to create new data
MR_model.DSMRs.perform_MRs_tree(data.train_x, data.train_y, data.max_y)
The maximum number of composite MRs can be increased before perform_MRs_tree() to create more data
MR_model.DSMRs.update_composite(max_composite=2)
Use MRModel to Train Keras Model
Trains the given model and returns F1 scores and models for various configurations
from metamorphic_relations.Results import Results
MR_model.compare_MRs() compares each MR individually using all the training data
results, models = MR_model.compare_MRs()
MR_model.compare_MR_sets() compares each set of MR (No MRs, GMRs, DSMRs, GMRs + DSMRs) using all the training data
results, models = MR_model.compare_MR_sets()
MR_model.compare_MR_sets() compares each set of MR (No MRs, GMRs, DSMRs, GMRs + DSMRs) using increasing proportions of the training data
results, models = MR_model.compare_MR_sets_counts()
Save and Load Results
Result objects can be written to and read from text files
results, _ = MR_model.compare_MR_sets_counts()
results.write_to_file("Output/MNIST_sets_results.txt")
Results.read_from_file("Output/MNIST_sets_results.txt")
Present Results
Using compare_MRs() gives individual results for each MR, these can be tabulated with initial number of training instances, actual number used, train and test F1 score
results, _ = MR_model.compare_MRs()
results.write_to_file("Output/MNIST_individual_best_results.txt")
Results.read_from_file("Output/MNIST_individual_best_results.txt").print_individual()
The results for each set from compare_MR_sets_counts() can be graphed with graph_all()
results, _ = MR_model.compare_MR_sets_counts()
results.write_to_file("Output/MNIST_sets_results.txt")
Results.read_from_file("Output/MNIST_sets_results.txt").graph_all("MNIST")
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for metamorphic_relations-0.1.2.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 9015e825ce98b2bb9b6f5e2fbfcac81d14b90a38cf0b2d260506a0acf5ad20b9 |
|
MD5 | e19bfcca8b060a8f4d56ffc0a71c26ff |
|
BLAKE2b-256 | ff9ad5a83ef5f5063a43f138df04f7a267585d1018d4fad0efa8dbf1f9400f70 |
Hashes for metamorphic_relations-0.1.2-py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | ad1a6482acc79b7b5766b387fc34cd8bbc54ac2948195659c4ce14cdd05b397c |
|
MD5 | e09db32363c3c1ecb6a42db6c0a4f81c |
|
BLAKE2b-256 | ad95e11a593f4eacb8d92dc192ca9b03b735e11fc076641f4b54b0ef01ed5a31 |