Keywords
computer code, computer programming, artificial intelligence tools, bioinformatics
Abstract
Life scientists frequently write computer code when doing research. Computer programming can aid researchers in performing tasks that are not supported by existing tools. Programming can also help researchers to implement analytical logic in a way that documents their steps and thus enables others to repeat those steps. Many educational resources are available to teach computer programming, but this skill remains challenging for many researchers and students to master. Artificial-intelligence tools like OpenAI’s ChatGPT are able to interpret human-language requests to generate code. Accordingly, we evaluated the extent to which this technology might be used to perform programming tasks described by humans. To evaluate ChatGPT, we used requirements specified for 184 programming exercises taught in an introductory bioinformatics course at the undergraduate level. Within 7 or fewer attempts, ChatGPT solved 179 (97.3%) of the exercises. These findings suggest that some educators may need to reconsider how they evaluate students’ programming abilities, and researchers might be able to collaborate with such tools in research settings.
Original Publication Citation
Piccolo SR, Denny P, Luxton-Reilly A, Payne SH, Ridge PG (2023) Evaluating a large language model’s ability to solve programming exercises from an introductory bioinformatics course. PLoS Comput Biol 19(9): e1011511. https://doi.org/10.1371/journal.pcbi.1011511
BYU ScholarsArchive Citation
Piccolo, Stephen R.; Denny, Paul; Luxton-Reilly, Andrew; Payne, Samuel H.; and Ridge, Perry G., "Evaluating a large language model’s ability to solve programming exercises from an introductory bioinformatics course" (2023). Faculty Publications. 7370.
https://scholarsarchive.byu.edu/facpub/7370
Document Type
Peer-Reviewed Article
Publication Date
2023-09-28
Publisher
PLOS
Language
English
College
Life Sciences
Department
Biology
Copyright Status
© 2023 Piccolo et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Copyright Use Information
https://lib.byu.edu/about/copyright/
Included in
Bioinformatics Commons, Programming Languages and Compilers Commons, Software Engineering Commons
