ChatGPT in Computer Science Curriculum Assessment: An analysis of Its Successes and Shortcomings
Abstract
The application of Artificial intelligence for teaching and learning in the academic sphere is a trending subject of interest in computing education. ChatGPT, as an AI-based tool, provides various advantages, such as heightened student involvement, cooperation, accessibility, and availability. This paper addresses the prospects and obstacles associated with utilizing ChatGPT as a tool for learning and assessment in undergraduate Computer Science curriculum in particular to teaching and learning fundamental programming courses. Students having completed the course work for a Data Structures and Algorithms (a sophomore-level course) participated in this study. Two groups of students were given programming challenges to solve within a short period of time. The control group (group A) had access to textbooks and notes of programming courses, however, no Internet access was provided. Group B students were given access to ChatGPT and were encouraged to use it to help solve the programming challenges. The challenge was conducted in a computer lab environment using Programming Contest Control (PC2) environment which is widely used in ACM International Collegiate Programming Contest (ICPC). Each team of students addresses the problem by writing executable code that satisfies a certain number of test cases. Student teams were scored based on their performance in terms of the number of successfully passed test cases. Results show that students using ChatGPT had an advantage in terms of earned scores, however, there were inconsistencies and inaccuracies in the submitted code consequently affecting the overall performance. After a thorough analysis, the paper’s findings indicate that incorporating AI in higher education brings about various opportunities and challenges. Nonetheless, universities can efficiently manage these apprehensions by adopting a proactive and ethical stance toward the implementation of such tools.