Escala de Sintomas Vocais (ESV-TRI) : avaliação das evidências de validade e predição de desfechos vocais por meio de modelos de aprendizado de máquina

Abstract

Introduction: Vocal assessment requires a multidimensional approach that includes the patient’s perception of their own voice. Self-assessment instruments, such as the Vocal Symptoms Scale (VSS), allow measuring the impact of vocal changes on daily life and monitoring therapeutic progress. The Brazilian version validated through Item Response Theory (VSS-IRT) represents a methodological advancement but lacked additional validity evidence based on relationships with other variables and responsiveness. Thus, the present study aimed to (1) investigate validity evidence based on relationships with other variables and (2) analyze the responsiveness of the VSS-IRT. Methods: The research was structured into two complementary studies. The cross-sectional study included 293 dysphonic participants treated at the Speech-Language Pathology teaching clinic of the Federal University of Paraíba (UFPB). Sociodemographic data, selfassessment measures (VSS-IRT, Voice Handicap Index—VHI-IRT, Voice-Related Quality of Life—V-RQOL-IRT, and Vocal Tract Discomfort Scale—VTDS-IRT), auditory-perceptual judgment (APJ), acoustic analysis, and laryngeal examination were collected. Pearson, polychoric, and tetrachoric correlations, logistic regression with recursive feature elimination (RFE), and machine learning models (Random Forest, Logistic Regression, and Gradient Boosting) with stratified cross-validation were performed. The longitudinal study involved 72 participants divided into an intervention group (IG) and a control group (CG). The IG completed six sessions of voice therapy, and both groups were evaluated using the VSS-IRT, APJ, and acoustic measures at preand post–six weeks. Analyses were conducted using Wilcoxon, Mann–Whitney, and mixed ANOVA tests. Results: The VSS-IRT showed strong positive correlations with VHI-IRT (r = 0.636; p < 0.001), V-RQOL-IRT (r = 0.602; p < 0.001), and VTDS-IRT (r = 0.720; p < 0.001), confirming convergent validity. There was a significant association between the VSS-IRT and the presence of vocal deviation according to the APJ (p = 0.011), and a moderate positive correlation with the overall severity grade (p = 0.025). Weak correlations were found between the VSS-IRT score and acoustic measures (F0sd, F0cv, shimmer, HNR, and CPPS). A moderate positive correlation was observed between the VSS-IRT and the presence of laryngeal structural alterations (p = 0.048). These findings reinforce the concurrent criterion validity of the VSS-IRT in relation to vocal deviation (APJ), acoustic measures, and laryngeal alterations. Clinical groups categorized by vocal deviation severity (p < 0.0001), laryngeal diagnoses (p < 0.001), and type of dysphonia (p = 0.005) showed significant differences in VSS-IRT scores, demonstrating the instrument’s discriminant validity. No differences were observed regarding gender, education level, or being a voice professional. Machine learning models were developed to test the prediction of VSS-IRT scores based on multidimensional voice assessment outcomes. The Random Forest model identified ten acoustic variables as the most predictive of the VSS-IRT score—F0 deviation, HNR, CPPSmax, HNRD, Shimmer.APQ3, LNPSD, F0 quartile 1, GNE3000, F0min, and GNE2000—with 82% accuracy and AUC > 0.80, indicating predictive capability. In the longitudinal study, the IG showed significant reductions in VSS-IRT scores and improvements in seven acoustic measures and in the APJ overall severity grade, whereas the CG remained stable. Mixed ANOVA confirmed a significant interaction between group and time, indicating vocal improvement only in the IG. The VSS-IRT showed 90.91% specificity, 80.70% overall accuracy, and an 85.11% negative predictive value relative to APJ, demonstrating excellent diagnostic performance for ruling out dysphonia and detecting clinically relevant changes. Conclusion: The VSS-IRT demonstrates full validity evidence in its Brazilian Portuguese version. The study provides validity evidence based on external variables and responsiveness. The VSS-IRT shows (1) convergent validity with VHI-IRT, V-RQOL-IRT, and VTDS-IRT; (2) concurrent criterion validity with laryngeal structural alterations, APJ vocal deviation, and acoustic measures; (3) discriminant validity regarding APJ deviation severity, laryngeal diagnostic categories, and dysphonia types; (4) predictive criterion validity between the VSS-IRT, acoustic measures, and APJ vocal deviation; and (5) responsiveness, demonstrated by its ability to detect significant changes in vocal symptoms throughout therapy. This characteristic reinforces its clinical utility by enabling sensitive and accurate monitoring of patient progress and identifying improvements or persistence of vocal symptoms during intervention.