Evaluation of Transformer-Based Large Language Models for Email Spam Detection Using BERT, Phi, and Gemma

  • Ana Clara C. Grassmann São Paulo State University - UNESP
  • Juliana C. Feitosa São Paulo State University - UNESP
  • José Remo F. Brega São Paulo State University - UNESP
  • Kelton A. P. da Costa São Paulo State University - UNESP
Keywords: Large Language Models, Spam Detection, Fine-Tuning, Binary Classification, Cybersecurity

Abstract

In this paper, we study how LLMs based on the transformer architecture work and the possibility of adjusting these models to use only the body of email messages to classify them as spam or ham. The models studied are BERT, Gemma, and Phi. All of them underwent quantization stages, fine-tuning with a real dataset, and evaluation with metrics commonly used in binary classification problems. The Gemma model achieves over 99% accuracy in detecting spam, standing out as the best among the compared models.
Published
2024-12-26
How to Cite
C. Grassmann, A. C., C. Feitosa, J., F. Brega, J. R., & A. P. da Costa, K. (2024). Evaluation of Transformer-Based Large Language Models for Email Spam Detection Using BERT, Phi, and Gemma. Statistics, Optimization & Information Computing, 13(2), 459-473. https://doi.org/10.19139/soic-2310-5070-2267
Section
Research Articles