Keywords

LLMs, ChatGPT, deception, keyboard dynamics, mouse tracking, design science

Abstract

Large language models (LLMs) such as OpenAI's GPT-4 have transformed natural language processing with their ability to understand context and generate human-like text. This has led to considerable debate, especially in the education sector, where LLMs can enhance learning but also pose challenges to academic integrity. Detecting AIgenerated content (AIGC) is difficult, as existing methods struggle to keep pace with advancements in generation technology. This research proposes a novel approach to AIGC detection in short essays, using digital behavior capture and follow-up questioning to verify text authorship. We executed a controlled experiment as an initial evaluation to test the prototype system. The results obtained show promise in differentiating between user-authored and AI-generated text. The system design and prototype represent valuable contributions for future research in this area. The solution also provides a novel approach to addressing practical challenges posed by LLMs, particularly in maintaining academic integrity in educational settings.

Original Publication Citation

Wilson, David W., Burnett, Parker, Valacich, Joseph S., & Jenkins, Jeffrey L., “Human or AI? Using Digital Behavior to Verify Essay Authorship” (2023), ICIS 2023 Proceedings, Hyderabad, India.

Document Type

Conference Paper

Publication Date

2023

Publisher

ICIS 2023 Proceedings

Language

English

College

Marriott School of Business

Department

Information Systems Management

University Standing at Time of Publication

Assistant Professor

Download

Share

COinS