The aim of this study is to check whether a model with better accuracy and loss values is more robust. As the two models below have a close accuracy, it is difficult to distinguish which is better based on this data alone.
Example of original image and after augmentation
To illustrate the behaviour of the models, we use a radar with data from the dominance results of the Saimple evaluations.
A curve is composed of 10 points (10 points, as 10 output classes). One point corresponds to the classification score of a class. The class with the highest score will be the class selected by the model.
For the purpose of simplified visualisation, the classification score displayed on the radar corresponds to the maximum value that the neural network can respond to in the space of the inputs considered. The colours of the curves are chosen randomly from one radar to another.
The centre of the radar corresponds to the value 0 and its periphery the value 1.
Two examples of evaluations with a picture representing a 3 sign :
In the worst case, no class is dominant, they are all in conflict. In the best case, class 3 is dominant with a score of 1 and the other 9 have a score of 0.
Illustration of the 10 reference classes
Note: Each curve is drawn from a dominance result of an evaluation with a model and one of the 10 images presented above.
The model on the left is less accurate. The curves are less close to the periphery and more close to the centre. Whereas the model on the right illustrates dominances that are well demarcated for each of the figures. We can therefore understand from the analysis carried out with Saimple that the augmented model is much more efficient.
Note: Each curve is derived from a dominance result of an evaluation with a model and one of the 10 images presented earlier.
Graph of max deltas by class and model :
Reminder: The higher the delta max value, the more robust the model isIt can be seen from the evaluation that the values of the max deltas of the "Augmented" model are always higher than those of the "Original" model. Thus the model has been made more robust by the data augmentation. By averaging the ratios of the max deltas over 10 evaluations between the two models, we notice that the max delta of the "Augmented" model is twice as large as that of the "Original" model.
Contact us : firstname.lastname@example.org