Human vs AI prompts: Exploring differences in grammatical choices of writer's intent

Presentation Type

Abstract

Faculty Advisor

Libby Barak

Access Type

Event

Start Date

25-4-2025 10:30 AM

End Date

25-4-2025 11:29 AM

Description

Previous research points to key differences between human and ChatGPT-generated prompts such as rhetorical and lexical choices (Reinhart et al., 2024). This study aims to analyze grammatical differences using Part-of-Speech (POS) as a metric of writer’s intent variation (Biber, 1988). We explored a dataset of human-generated questions from Quora and their ChatGPT-paraphrased prompts (Vorobev, 2023). We tokenized the data and assigned a POS tag for each token while preserving sentence boundaries to keep grammatical structure intact. Then, we extracted POS unigrams, bigrams, and trigrams (e.g. NN, JJ NN, and PRP VBD TO for one, two, and three POS tags respectively) to analyze differences in verb-structure aspects in relation to writer’s intent. Results suggested that humans write in a much more involved style, according to Biber’s Multidimensional Analysis, due to a larger number of present participles and infinitives compared to AI prompts. Human prompts are also more narrative than AI Prompts, as indicated by the commonality of the bigram PRP VBD (Personal-pronoun followed by past-tense participle verb). Meanwhile, AI prompts rely less on context to generate responses, but instead elaborate much more than human writers, e.g. bigrams such as NN NNS concretely explain nouns by pairing them with other nouns. Such bigrams also exemplify the prevalent abstract writing style of AI prompts, while human writing contains more adverbs and modalities which can be attributed to an overt-persuasive style. In current work-in-progress, we pursue a more fine-grained approach to POS distribution to further analyze its role in AI prompt engineering.

Comments

Poster presentation at the 2025 Student Research Symposium.

This document is currently not available here.

Share

COinS
 
Apr 25th, 10:30 AM Apr 25th, 11:29 AM

Human vs AI prompts: Exploring differences in grammatical choices of writer's intent

Previous research points to key differences between human and ChatGPT-generated prompts such as rhetorical and lexical choices (Reinhart et al., 2024). This study aims to analyze grammatical differences using Part-of-Speech (POS) as a metric of writer’s intent variation (Biber, 1988). We explored a dataset of human-generated questions from Quora and their ChatGPT-paraphrased prompts (Vorobev, 2023). We tokenized the data and assigned a POS tag for each token while preserving sentence boundaries to keep grammatical structure intact. Then, we extracted POS unigrams, bigrams, and trigrams (e.g. NN, JJ NN, and PRP VBD TO for one, two, and three POS tags respectively) to analyze differences in verb-structure aspects in relation to writer’s intent. Results suggested that humans write in a much more involved style, according to Biber’s Multidimensional Analysis, due to a larger number of present participles and infinitives compared to AI prompts. Human prompts are also more narrative than AI Prompts, as indicated by the commonality of the bigram PRP VBD (Personal-pronoun followed by past-tense participle verb). Meanwhile, AI prompts rely less on context to generate responses, but instead elaborate much more than human writers, e.g. bigrams such as NN NNS concretely explain nouns by pairing them with other nouns. Such bigrams also exemplify the prevalent abstract writing style of AI prompts, while human writing contains more adverbs and modalities which can be attributed to an overt-persuasive style. In current work-in-progress, we pursue a more fine-grained approach to POS distribution to further analyze its role in AI prompt engineering.