Addressing Question Repetition in Academic Assessments: An Empirical Study and Generative AI-Based Solution

Abstract

Frequent reuse of exam questions harms the integrity of assessments and pushes students towards learning by automatism rather than understanding. It is a fundamental problem, yet the scientific literature has paid little attention to it so far. Our research addresses precisely this lack. Initially, we measured the concrete impact of this phenomenon through a survey among 194 actors from the academic community. Faced with this observation, we have developed an automatic question generation system based on the fine-tuned Transformers, in order to ensure a continuous renewal of the proposed topics. The results of the survey confirm the relevance of this approach: 80.4% of respondents state that they regularly encounter questions already seen, a situation that 63.4% of them consider penalising. In addition, 72.7% of participants advocate for an accelerated renewal of evaluation content. To meet this identified need, we designed an automatic question generation (AQG) architecture based on transformers, by fine-tuning the T5-small, T5-base and BART models on the SQuAD dataset. The comparative evaluation, supported by the BLEU, ROUGE and METEOR metrics as well as a multi-domain qualitative semantic analysis, established the constant superiority of the T5-based model over the other approaches (BLEU = 0.1765; ROUGE-1 = 0.5317; METEOR = 0.4085). These findings empirically validate the urgency of renewing assessments and demonstrate the effectiveness of transformer-based systems in ensuring diversity of tests, while easing teachers’ workload. This study thus establishes the pedagogical necessity, as well as the technical feasibility, of an AI-assisted generation of questions in the service of educational equity.