By Joshua Lambert (University of Cincinnati)
Information
As artificial intelligence (AI) becomes increasingly integrated into education, understanding student perceptions of AI-generated support is critical. This pilot study examined how Doctor of Nursing Practice (DNP) students evaluate statistical help from different sources. Using a randomized, blinded, within-subjects design, seven DNP students at a Midwestern R1 university's College of Nursing were recruited via email during the fall 2024 and spring 2025 semesters (IRB ID: 2024-0764). Each participant submitted their own statistical questions related to their capstone projects through a pretest survey administered via REDCap. Questions covered topics such as selecting appropriate analyses, interpreting results, and managing data for their DNP final scholarly projects. Students received a single round of responses (not an ongoing dialogue) from three blinded sources: a statistics professor, a trained doctoral-level graduate assistant, and a custom-built ChatGPT (OpenAI) chatbot. The response order was randomized per a protocol established prior to the study, and all other conditions were held constant. The custom chatbot was created using OpenAI's GPT platform (ChatGPT Plus). The principal investigator trained the chatbot by uploading self-created educational resources (including lecture slides, statistical tips, and prior consultation responses) drawn from over 15 years of combined biostatistics consulting experience with DNP students. After receiving the three blinded responses, students completed a follow-up Likert-scale survey (1–5) in REDCap rating each response on perceived helpfulness, satisfaction, and likelihood of use. Students also guessed which response came from the chatbot. The chatbot received the highest average ratings for helpfulness (M = 4.43) and satisfaction (M = 4.43). Notably, all seven students incorrectly identified the chatbot, attributing its responses to either the graduate assistant or professor. However, responses students believed were AI-generated received consistently lower ratings across all dimensions, suggesting a perceptual bias against AI-generated content. This finding aligns with expectation confirmation theory (Oliver, 1980) and prior work on AI trust in health professions education (Choudhury & Shamszare, 2023; Ragot et al., 2020). These results highlight the need to address bias and trust as AI tools are integrated into academic support. The full study methods and results are published in the Journal of Nursing Education (April 2026; doi: 10.3928/01484834-20260216-01).