DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programs
To use DeepDiagnosis, you need to add our callback as a subclass in your keras.callbacks.py file.
The core principle of our callback is to get a view of the internal states and statistics of the model during training.
Then you can pass our callback DeepDiagnosis()
to the .fit()
method of a model as follows:
callback = keras.callbacks.DeepDiagnosis(inputs, outputs, layer_number, batch_size, startTime)
model = keras.models.Sequential()
model.add(keras.layers.Dense(64))
model.add(keras.layers.Activation(activations.relu))
model.compile(keras.optimizers.SGD(), loss='mse')
model.fit(np.arange(100).reshape(5, 20), np.zeros(5), epochs=10, batch_size=1,
... callbacks=[callback], verbose=0)
The version numbers below were confirmed to work with the current release:
python 3.6.5
Keras 2.2.0
Keras-Applications 1.0.2
Keras-Preprocessing 1.0.1
numpy 1.19.2
pandas 1.1.5
scikit-learn 0.21.2
scipy 1.6.0
tensorflow 1.14.0
If you find this paper useful in your research, please consider citing:
@inproceedings{wardat2021deepdiagnosis,
author={Mohammad Wardat and Breno Dantas Cruz and Wei Le and Hridesh Rajan},
title={DeepDiagnosis: Automatically Diagnosing Faults and Recommending Actionable Fixes in Deep Learning Programs},
booktitle = {ICSE'22: The 44th International Conference on Software Engineering},
location = {Pittsburgh, PA, USA},
month = {May 21-May 29, 2022},
year = {2022},
entrysubtype = {conference}
}
- Contains the source code to extract (.h5) to source code
- Contains the source code of all AUTOTRAINER Models
- Contains the results of DeepDiagnosis from AUTOTRAINER dataset
- Contains the results of the motivating example using AUTOTRAINER
- Contains the experiments of saturated activation for Sigmoid and Tanh
- Contains the result of AUTOTRAINER on normal models with different threshold (accuracy =100%)
- Complete result of Table 6
- this model(MNIST_Normal/double_random_99fe3625-3c58-4766-968e-7d40401237fe) is detected by DeepDiagnosis and the accuracy = 20%
- Contains the results of DeepLocalize from AUTOTRAINER dataset
- Contains the results of UMLUAT from AUTOTRAINER dataset