Project – Explainable Machine Learning for Phishing Detection in Nigerian University Email Systems: A Comparative Evaluation of Detection Accuracy and False-Positive Rates.

CHAPTER ONE

INTRODUCTION

Title: Explainable Machine Learning for Phishing Detection in Nigerian University Email Systems: A Comparative Evaluation of Detection Accuracy and False-Positive Rates

1.1 Background to the Study

The increasing dependence on digital communication technologies has significantly transformed the operations of educational institutions across the world. Universities now rely heavily on electronic mail systems for academic communication, administrative activities, research collaboration, student registration, examination notifications, financial transactions, and information sharing among staff and students. Email has become an essential communication channel because it provides a relatively fast, affordable, and convenient means of transmitting information across different geographical locations. However, the growing dependence on email communication has also exposed universities to various cybersecurity threats, particularly phishing attacks. Phishing is a form of social engineering in which cybercriminals impersonate trusted individuals, institutions, or organisations to deceive recipients into disclosing confidential information, clicking malicious links, downloading harmful attachments, or performing unauthorised transactions. According to Fette, Sadeh and Tomasic (2007), phishing emails exploit the trust and expectations of recipients by presenting fraudulent messages that appear to originate from legitimate sources. Similarly, Hong (2012) explains that phishing remains a major information security problem because attackers continuously manipulate human behaviour and exploit weaknesses in digital communication systems. Consequently, the protection of university email systems has become an important aspect of institutional cybersecurity management.

The nature of phishing attacks has become increasingly sophisticated as cybercriminals continue to adopt techniques that make fraudulent emails difficult to distinguish from legitimate communications. Earlier phishing messages were often characterised by poor grammar, suspicious email addresses, unusual formatting, and obvious requests for sensitive information. However, contemporary phishing attacks may contain professionally written messages, convincing institutional logos, realistic email signatures, familiar names, and links that closely resemble legitimate university websites. Attackers may impersonate university management, lecturers, examination officers, admission departments, financial offices, information technology administrators, and other trusted institutional authorities. Such messages may request students to update their portal information, confirm admission status, pay outstanding fees, reset passwords, or download important academic documents. Fette et al. (2007) demonstrated the importance of combining different email characteristics in detecting phishing messages, while Bergholz et al. (2010) established that phishing detection can benefit from analysing linguistic and structural characteristics of email messages. These developments indicate that traditional approaches based solely on suspicious keywords, blacklists, or manually defined rules may struggle to recognise changing phishing patterns. Therefore, more adaptive and intelligent detection mechanisms are necessary to improve the security of electronic communication systems.

Machine learning has emerged as an important technological approach for addressing cybersecurity challenges, including the identification and classification of phishing emails. Machine learning involves the development of computational models that learn patterns from existing data and use the acquired knowledge to make predictions or classifications on new observations. In phishing detection, machine learning algorithms can analyse different characteristics of email messages, including sender information, subject lines, message content, hyperlinks, attachment indicators, linguistic patterns, and other relevant features. These algorithms can be trained using datasets containing both legitimate and phishing emails to distinguish suspicious messages from genuine communications. Several machine learning techniques have been applied to phishing detection, including Logistic Regression, Decision Tree, Random Forest, Support Vector Machine, Naïve Bayes, and ensemble learning methods. Sahingoz et al. (2019) demonstrated the application of machine learning and natural language processing techniques to phishing email detection, highlighting the importance of extracting meaningful characteristics from email content. Similarly, Almujahid, Haq and Alshehri (2024) conducted a comparative evaluation of machine learning algorithms for phishing website detection, demonstrating the relevance of systematic model comparison in determining detection performance. These developments provide a foundation for investigating how different machine learning algorithms can improve phishing detection within Nigerian university email environments.

Despite the potential of machine learning, the use of highly complex predictive models introduces another important challenge relating to the transparency and interpretability of automated decisions. Some advanced machine learning algorithms operate as black-box systems because their internal decision-making processes are difficult for ordinary users and cybersecurity administrators to understand. Although such models may produce accurate predictions, they do not always provide clear explanations of why a particular email has been classified as phishing or legitimate. This limitation can create difficulties for university information technology departments, especially when they need to investigate suspicious messages, justify security decisions, correct classification errors, and establish confidence in automated detection systems. Explainable Artificial Intelligence (XAI) has therefore become an important area of research concerned with making the decisions of artificial intelligence systems more understandable to human users. Ribeiro, Singh and Guestrin (2016) introduced Local Interpretable Model-Agnostic Explanations (LIME), a technique that explains individual model predictions by approximating the behaviour of complex models around specific observations. Similarly, Lundberg and Lee (2017) developed SHapley Additive exPlanations (SHAP), which uses Shapley values to explain the contribution of individual features to model predictions. These techniques provide opportunities to examine the factors influencing phishing detection decisions and improve the transparency of machine learning applications.

The importance of explainability becomes particularly significant when evaluating the accuracy and reliability of phishing detection systems. Accuracy measures the proportion of correctly classified observations among all observations evaluated by a model. However, accuracy alone may not adequately represent the practical usefulness of a phishing detection system, particularly when the dataset contains different proportions of legitimate and phishing emails. Another important evaluation measure is the false-positive rate, which represents the proportion of legitimate emails incorrectly classified as phishing among all legitimate emails. In a university environment, an excessively high false-positive rate may result in important academic and administrative messages being incorrectly blocked, quarantined, or flagged as suspicious. Such errors could prevent students from receiving examination schedules, admission notifications, scholarship information, assignment instructions, and other essential communications. On the other hand, a system with a high false-negative rate may allow dangerous phishing messages to reach users. Sokolova and Lapalme (2009) emphasise the importance of evaluating classification systems using multiple performance measures rather than relying on a single metric. Consequently, a reliable phishing detection system must achieve a suitable balance between identifying malicious emails and allowing legitimate messages to reach their intended recipients.

In Nigeria, the expansion of digital technologies within the higher education sector has increased the importance of protecting institutional information systems and electronic communication platforms. Nigerian universities increasingly depend on online portals, institutional email addresses, electronic learning environments, digital libraries, student information systems, and other internet-based services to support teaching, learning, research, and administration. These systems frequently involve the exchange of personal information, academic records, institutional correspondence, financial information, and other sensitive data. As a result, phishing attacks targeting university email users could expose students, lecturers, administrators, and institutions to identity theft, account compromise, financial fraud, and unauthorised access to institutional resources. The Federal Republic of Nigeria (2021), through the National Cybersecurity Policy and Strategy, recognises the importance of strengthening cybersecurity capacity, protecting digital infrastructure, and promoting secure participation in the digital environment. Furthermore, the Nigeria Data Protection Act (2023) establishes obligations relating to the protection and lawful processing of personal data. These national developments reinforce the need for Nigerian universities to adopt effective information security measures. However, the implementation of intelligent email protection systems must consider the operational conditions of individual institutions, including the quality of available data, technical infrastructure, cybersecurity expertise, and the reliability of automated detection decisions.

Recent developments in explainable machine learning provide an opportunity to improve phishing detection by combining predictive performance with understandable explanations of model behaviour. Research on explainable phishing detection has demonstrated the potential of combining machine learning algorithms with interpretability techniques such as LIME and SHAP. For example, Pandya et al. (2025) investigated phishing detection using explainable machine learning models and highlighted the importance of incorporating transparency into automated security decisions. Similarly, recent studies have explored the use of explainable feature selection and model interpretation to identify the characteristics that contribute most significantly to phishing classification. These developments suggest that an effective phishing detection framework should not only identify malicious emails but should also provide understandable reasons for its predictions. Nevertheless, the performance of machine learning models may differ depending on the dataset, feature extraction techniques, algorithm selection, and evaluation procedures employed. More importantly, findings obtained from general phishing datasets cannot automatically be assumed to represent the characteristics of email communication within Nigerian universities. There is therefore a need for a context-sensitive comparative evaluation that examines detection accuracy, false-positive rates, and model explainability using relevant email data. It is against this background that this study investigates explainable machine learning for phishing detection in Nigerian university email systems, with particular attention to the comparative evaluation of detection accuracy and false-positive rates.

1.2 Statement of the Problem

The increasing use of electronic mail systems in Nigerian universities has created opportunities for improved academic communication, administrative efficiency, research collaboration, and information sharing. However, the growing dependence on these platforms has also increased the potential exposure of university communities to phishing attacks and other email-related cybersecurity threats. Students, lecturers, administrative personnel, and university management frequently receive messages relating to admissions, registration, examinations, fee payments, academic activities, and institutional announcements. Cybercriminals can exploit these communication channels by creating fraudulent messages that imitate legitimate university correspondence. Such attacks may deceive recipients into revealing login credentials, disclosing personal information, transferring money, or accessing malicious websites. Although traditional email security measures provide some level of protection, the continuous modification of phishing techniques makes it difficult for conventional detection systems to identify every suspicious message. This situation raises concerns about the ability of existing email protection mechanisms to provide reliable security within Nigerian university environments.

Another problem concerns the limitations associated with the accuracy and reliability of existing machine learning approaches to phishing detection. Although different algorithms have been developed to classify phishing and legitimate messages, their performance may vary depending on the nature of the training dataset, selected features, algorithmic structure, and evaluation procedures. A model may demonstrate high overall accuracy while still incorrectly classifying a considerable number of legitimate emails as phishing or allowing some phishing messages to pass through the detection system. In Nigerian universities, the incorrect classification of legitimate emails may disrupt academic and administrative communication, particularly when important messages concerning examinations, admissions, registration, research activities, and financial transactions are blocked or marked as suspicious. Conversely, failure to detect malicious emails may expose users and institutional information systems to security breaches. The problem, therefore, is not simply the development of a machine learning model with high detection accuracy but the identification of models that can achieve effective phishing detection while maintaining an acceptable false-positive rate.

Furthermore, the increasing application of complex machine learning algorithms has introduced concerns regarding the transparency and explainability of automated phishing detection decisions. Some predictive models operate as black-box systems, making it difficult for university information technology administrators and cybersecurity personnel to understand the specific characteristics responsible for classifying an email as malicious. Without understandable explanations, security administrators may find it difficult to verify suspicious classifications, investigate incorrect predictions, and determine whether a model is relying on meaningful phishing indicators or irrelevant characteristics. Although explainability techniques such as LIME and SHAP have been developed to address this challenge, their application and usefulness may differ across machine learning algorithms and datasets. Consequently, there is a need to investigate how explainable machine learning can improve the interpretation of phishing detection decisions and provide meaningful information about the features influencing the classification of university emails.

The central problem addressed by this study is the limited context-specific comparative evidence concerning the performance of explainable machine learning techniques for phishing detection in Nigerian university email systems, particularly in relation to detection accuracy and false-positive rates. Existing research has demonstrated the potential of machine learning and explainable artificial intelligence in identifying phishing activities; however, the performance of these approaches cannot automatically be generalised to Nigerian university email environments without appropriate evaluation. There is therefore a need to comparatively examine selected machine learning algorithms, determine their ability to distinguish phishing emails from legitimate messages, evaluate their detection accuracy and false-positive rates, and establish how explainability techniques can clarify their classification decisions. Addressing these problems will provide useful evidence for identifying more reliable, transparent, and contextually appropriate approaches to strengthening email security in Nigerian universities.

1.3 Objectives of the Study

The main objective of this study is to evaluate the application of explainable machine learning techniques for phishing detection in Nigerian university email systems, with particular emphasis on detection accuracy and false-positive rates.

The specific objectives are to:

1. Examine the application of selected machine learning algorithms in detecting phishing emails within Nigerian university email systems.

2. Comparatively evaluate the detection accuracy of selected machine learning algorithms in distinguishing phishing emails from legitimate emails.

3. Assess and compare the false-positive rates of selected machine learning algorithms in identifying legitimate university emails.

4. Investigate how explainable artificial intelligence techniques, particularly LIME and SHAP, improve the interpretation and transparency of machine learning-based phishing detection decisions.

1.4 Research Questions

The following research questions will guide the study:

1. How can selected machine learning algorithms be applied to detect phishing emails within Nigerian university email systems?

2. What differences exist in the detection accuracy of selected machine learning algorithms in distinguishing phishing emails from legitimate emails?

3. What are the comparative false-positive rates of selected machine learning algorithms in identifying legitimate university emails?

4. How do explainable artificial intelligence techniques, particularly LIME and SHAP, improve the interpretation and transparency of machine learning-based phishing detection decisions?

1.5 Research Hypothesis

The following null hypothesis will be tested at a 0.05 level of significance:

H₀: There is no statistically significant difference in the detection accuracy and false-positive rates of selected explainable machine learning models used for phishing detection in Nigerian university email systems.

1.6 Significance of the Study

This study will be significant to university management, information technology administrators, students, lecturers, cybersecurity researchers, policymakers, and future researchers.

University Management: The study will provide useful information to university management regarding the importance of adopting intelligent and explainable cybersecurity solutions for protecting institutional email systems. Its findings will assist management in understanding the implications of phishing attacks and the need to support effective email security infrastructure.

Information Technology Administrators: The study will provide information on the comparative performance of different machine learning algorithms in phishing detection. It will help university ICT personnel understand the importance of detection accuracy, false-positive rates, and explainability when selecting or implementing automated email security solutions.

Students: Students will benefit from improved awareness of phishing threats and the importance of protecting their institutional email accounts. The study may also contribute to the development of more reliable detection mechanisms that reduce their exposure to fraudulent academic notifications, credential theft, and financial scams.

Lecturers and Academic Staff: The study will highlight the importance of protecting official academic communication channels against impersonation and fraudulent messages. Improved phishing detection can help safeguard communication relating to teaching, examinations, research collaboration, and other academic responsibilities.

Cybersecurity Professionals: The research will provide useful insights into the application of explainable machine learning techniques in cybersecurity. It will demonstrate how LIME and SHAP can be used to understand the features influencing automated phishing detection decisions and support the investigation of classification errors.

Policymakers and Regulatory Institutions: The findings may provide useful information for stakeholders responsible for promoting cybersecurity awareness, digital infrastructure protection, and data security within Nigerian higher education institutions. The study may also support efforts to strengthen institutional cybersecurity practices in line with relevant national policies and regulations.

Future Researchers: The study will serve as a reference for researchers interested in machine learning, explainable artificial intelligence, email security, phishing detection, and cybersecurity applications within higher education institutions. It will also provide a foundation for further investigations involving larger datasets, additional classification algorithms, and more advanced explainability techniques.

1.7 Scope of the Study

The study focuses on the application of explainable machine learning techniques for phishing detection in Nigerian university email systems. It examines selected machine learning algorithms for distinguishing phishing emails from legitimate email messages. The study will comparatively evaluate the algorithms based on detection accuracy and false-positive rates.

The study will also investigate the application of explainable artificial intelligence techniques, particularly Local Interpretable Model-Agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP), to understand the features responsible for the classification decisions of the selected models.

The scope is limited to email-based phishing detection rather than phishing websites, SMS phishing, telephone-based scams, or other forms of cyberattacks. The study will focus on relevant email characteristics, including sender information, message content, subject lines, hyperlinks, and other features that can support phishing classification. Its geographical and institutional context is Nigerian universities.

1.8 Operational Definition of Terms

Explainable Machine Learning: The application of machine learning models alongside techniques that make their predictions and decision-making processes understandable to human users.

Phishing: A cyberattack in which an individual or organisation is deceived through fraudulent electronic messages designed to obtain confidential information, steal credentials, or facilitate unauthorised activities.

Phishing Detection: The process of identifying and classifying suspicious email messages that attempt to deceive recipients or compromise their information.

Machine Learning: A branch of artificial intelligence that enables computer systems to learn patterns from data and use the acquired knowledge to make predictions or classifications.

Detection Accuracy: The proportion of all email messages correctly classified as either phishing or legitimate by a detection model.

False-Positive Rate: The proportion of legitimate emails incorrectly classified as phishing out of all legitimate emails evaluated.

Legitimate Email: A genuine electronic message sent by an authorised individual, organisation, or institution for valid communication purposes.

LIME: Local Interpretable Model-Agnostic Explanations, an explainable artificial intelligence technique used to explain individual predictions made by machine learning models.

SHAP: SHapley Additive exPlanations, a method that explains machine learning predictions by estimating the contribution of individual features to the model’s output.

University Email System: An institutionally managed electronic mail platform used by students, lecturers, administrators, and other authorised members of a university community.

Comparative Evaluation: The systematic assessment of two or more machine learning models using common datasets, evaluation procedures, and performance metrics to determine their relative effectiveness.

1.9 Limitations of the Study

The study may encounter certain limitations during its execution. These include the availability of reliable and adequately labelled phishing and legitimate email datasets, difficulties in obtaining institution-specific email samples, and possible variations in the characteristics of phishing messages across different universities.

Another limitation may arise from the computational resources required to train and evaluate selected machine learning algorithms. Some advanced models may require greater processing capacity and longer training periods. Furthermore, the effectiveness of explainability techniques may depend on the complexity of the selected models and the quality of the extracted email features.

To address these limitations, the study will adopt appropriate data preprocessing procedures, use consistent evaluation criteria, and apply suitable explainability techniques to ensure that the comparative assessment is reliable and meaningful.

Project – Explainable Machine Learning for Phishing Detection in Nigerian University Email Systems: A Comparative Evaluation of Detection Accuracy and False-Positive Rates.