Detecção e destaque em vídeo de objetos utilizando YOLO

Abstract

In mobile devices and cameras, the image magnification (zoom) functionality is increasingly present, however, it is still an innovation to use object detection to automate this task. Detection is a classic problem related to computer vision that deals with the location of instances of semantic objects in a specific class. In this sense, the objective of this work was to implement a system capable of performing the detection of objects automatically in video, evaluating its operation with acoustic guitar, electric guitar and microphone. After this detection, a new video was generated emphasizing the detected instrument, in order to facilitate the observation of its execution. For that, a system was developed that, using a YOLOV4 model, is able to identify objects and perform a procedure similar to a zoom in on the video. A pipeline was implemented, where the frames are first extracted, and then the detection of a parameterized object in an interval of 12 frames. After detection, the clipping is made following an interpolation methodology to deal with the fluidity of the video, and, finally, a new video is generated from these clippings. Tests were made with different parameters for extracting the frames, using videos retrieved from Youtube, evaluating 4 scenarios for extracting the images. In these tests, the performance of the detection model, the time taken for extraction and the percentage of information excluded in the output video in each scenario were evaluated. For the validation of this pipeline, a methodology was adopted, assuming that detection would work efficiently, to validate the heuristic implemented in the interpolation, the confidence of the model for the 4 extraction scenarios and the behavior of the system when dealing with occlusion problems in the video.. In the validation, a pre-trained model of the YOLOV4 open source neural network with 80 classes was used, performing the detection of arbitrary objects, with cat and dog being the chosen classes. In addition, a customized YOLOV4 model was also trained to be able to perform specific detection of musical instruments with the Imagenet database. Regarding the network results, although not the focus of this work, the average accuracy achieved by the model in the guitar, acoustic guitar and microphone classes was 61.90%, 87.94% and 62.27%, respectively. In the zoom in system, it was possible to notice that the better the extraction parameter, the greater the number of objects detected by the model, as well as the greater precision and quality of detection. There was a small loss of quality from the resolution of the original video, and there was no significant loss of video content due to frame gap in detection. Concluding the analysis of the results obtained, it is possible to affirm that the proposal of the work was successful, therefore, all the presented objectives were reached. For future work, we aim to test new detection models, implement new output video evaluation criteria and parallelize the pipeline steps.