Beyond Single-Document Detection: A Within-Subject Authorship Authentication Approach in the Age of Generative AI

Abstract

The rapid development and integration of generative AI platforms like ChatGPT into daily life has created challenges for verifying authentic human writing in personnel selection contexts and beyond. This dissertation evaluated a within-subject authorship authentication approach that compares known human-authored writing with questioned documents rather than evaluating a single text in isolation. Using Stacked Denoising Autoencoder (SDAE) and Random Forest approaches, this study tested whether same source human-human writing pairs could be distinguished from different source human-AI writing pairs across baseline and evasion conditions. Results provided strong support for this approach. The baseline model performance before the incorporation of common evasion techniques was strong, with the best AUC of .930. Across evasion conditions, best AUC values remained high, ranging from .888 to .994. Stacked denoising autoencoders generally outperformed Random Forest, with average best AUC values of .944 and .935, respectively. Additionally, the SDAE approach also performed better than the commonly used detector GPTZero, which analyzes documents in isolation. These findings suggest that within-subject comparison is more effective at determining authentic authorship when compared to single-document AI detection and may help organizations assess authorship authenticity in written materials. This research has significant implications for personnel selection and the assessment of authentic human capabilities in an increasingly AI-driven environment.