The Effects of Clinical Experience in the Rating of Intelligibility of Phonetically Contrasted Words

Abstract

Purpose: The purpose of this study was to investigate if crowdsourced lay listeners with minimal exposure to child speech and certified clinicians in the field of speech-language pathology experienced in treating speech sound disorders rate speech intelligibility by direct magnitude estimation (DME) differently. Method: Speech recordings of 9 preschool children producing phonetic contrasts were rated by 30 listeners, 15 inexperienced listeners and 15 experienced clinicians, to compare perceptual ratings of child speech using Direct Magnitude Estimation (DME) to determine whether a listener bias exists resulting in the inconsistent subjective rating of intelligibility. Listening judgments were recruited through two crowdsourcing methods, Amazon Mechanical Turk and the ASHA Community sites. Results: The results of this study reinforce the correlation between measures of whole-word accuracy and ratings of intelligibility. It was found that listeners, both inexperienced and experienced with child speech productions, distinguish differences in intelligibility categorically when compared to word production accuracy. A significant difference was not found between DME intelligibility ratings of inexperienced and experienced listeners. Conclusion: Online crowdsourcing for the perceptual rating of child speech intelligibility provides high-quality data consistent with measures of whole word accuracy. Additionally, in this study there was evidence that indicates inexperienced and experienced listeners ratings are in concordance. This novel approach to rating child speech intelligibility increases the ability to obtain laypersons ratings using an ecologically valid approach.