Redes residuais : sobre quantidade de camadas e variância intraclasse utilizando aprendizagem não supervisionada

Abstract

Deep Learning is a term in the literature to define different approaches of Deep Neural Networks (i.e. networks that have more than two hidden layers). From this, different network architectures have been proposed in the literature. These networks have been used as a reference for different applications, highlighting networks such as VGG-16, GoogLeNet, and residual networks (ResNets). Residual networks have a topology inspired by the VGG-16 since it uses stacks of convolutional layers of 3 x 3 filters. As the main contribution, it forms residual blocks through shortcut connections to reduce problems of model degradation. This problem increases the model error rate as the network depth increases. From this, ResNets of 20 to 1202 layers were proposed. However, the literature does not portray a pattern of use of these models according to a certain context. Therefore, this work aimed to carry out an agnostic study, which analyzes the performance of five residual networks, as well as the computational cost, for different databases. Overall, 15 datasets were used. ResNet models with 20, 32, 44, 56, and 110 layers were trained and evaluated for each dataset. Also, the performance of these networks was evaluated using F1 measure. Afterward, a significance test was performed using t Student distribution. In this test, the F1 measure of each network was analyzed to find out if there was a significant improvement in performance with increasing depth. However, the experiments indicated no improvement in results despite the increase in computational cost. Besides, the intraclass variance was analyzed for each class of the datasets using the k-means unsupervised learning algorithm. Also, the k-means clusters were evaluated according to the silhouette coefficient. Through this analysis, we discovered that at least one of the classes with lower F1 measure scores have low diversity. Thus, this work performed a set of experiments with ResNet-20 and data augmentation techniques to improve the results. The performance of the ResNet-20 network was superior to all previously evaluated networks. Based on this, the increase in depth of these networks results in high computational cost, which is also proportional to the number of samples in the database. However, this increase is not proportional to improvements in the results of the evaluated network models, resulting in an F1 measure without significant variations, with the use of data augmentation techniques being more effective than increasing the number of model layers.