Sincronismo em língua de sinais: uma investigação de técnicas para redução da dissincronia entre o Português e a tradução automática para Língua Brasileira de Sinais

Abstract

The provision of a window with 3D avatars that render automatic translation of audiovisual content into sign languages enables a more significant offering of accessible programming, both on live television and video-on-demand services, while also reducing operational costs, particularly those associated with interpreters. However, additional challenges involve synchronizing the original audio with its equivalent presentation in signs, which is slower than human speech. Studies indicate that the duration of a sign presentation in American Sign Language (ASL) is approximately twice the length of a monosyllabic word. This study aims to investigate methods and strategies to improve the synchronization of signs with the associated video content. One approach considered is the application of automatic sentence summarization (or compression), which aims to produce a shorter paraphrase of a given input sentence, simplifying it to require fewer signs to convey the original content. Another applicable technique is called elastic adjustment, which involves modifying the playback duration of a media object. Elastic adjustment algorithms can adjust the speed of audio by up to 10%, either up or down, while maintaining acceptable audio perception quality. The main hypothesis is that the combination of such methods will allow the time required to interpret content into Brazilian Sign Language (Libras) to avoid desynchronization or loss of context, prioritizing the preservation of the most crucial information related to the content. 30 experiments with automatic summarization of journalistic content were conducted. In some cases, it was possible to display Brazilian Sign Language (Libras) signing using a 3D avatar within the duration of the news segment. Finally, an analysis of the quality of the automatic translation was carried out using the Bilingual Evaluation Understudy (BLEU), Metric for Evaluation of Translation with Explicit ORdering (METEOR) and the Translation Edit Rate (TER) algorithms.