Utilizing ChatGPT for subtitling movies: a comparative analysis of accuracy and fuency with human translators

Abstract

This study conducts a comprehensive examination of English-to-Arabic translations, comparing outputs from human translators with those from ChatGPT 3.5. A corpus was compiled using the "Our Cartoon President" series and its Arabic translations, aligning source texts with target translations. Thirty-two randomly selected examples underwent comparison between ChatGPT 3.5 outputs and human translations sourced from the Subscene website, where the latter were crafted by professional translators. To assess the translations, a survey was administered to English-Arabic translation professors, who rated fluency and accuracy on a 1 to 5 scale. Collected data underwent SPSS analysis, including a T-test to determine statistically significant differences between human and ChatGPT translations. Employing a mixed-methods approach, qualitative insights were drawn from professors' evaluations, while quantitative data stemmed from their ratings. Results indicate that human translation slightly outperformed ChatGPT translation, as demonstrated by both qualitative and quantitative analyses. Notably, the study underscores the continued necessity of post-editing, even with the advent of AI translation tools like ChatGPT, to ensure the highest quality and accuracy in translated content. This study sheds light on the comparative effectiveness of human and ChatGPT translations in English-to-Arabic contexts, emphasizing ChatGPT's potential utility as a subtitling tool to expedite the translation process.