The multiple dimensions of student academic performance include: engagement and participation in the learning process, the student’s previous academic performance and behavior, how the student studies, and the environment in which the student engages in the learning process. Identifying students at risk of underachieving academically enables educational institutions to prepare and implement timely and effective interventions to support the students. Traditional methods of identifying at-risk students mainly depend on teachers’ observations and students’ academic performance and records, and they often lead to the identification of students at risk at a late stage. It is for this reason that several researchers have focused on predicting students’ academic performance using data science and other allied fields.

Educational Analytics combines multiple dimensions of students’ performance in ways that Academic Performance, Behavioral, and Social-Emotional dimensions of students had not been previously integrated. Prediction of students’ performance and achievement in the field of Educational Data Mining has gained increased attention, despite challenges of applying and adapting it to different educational situations. It has also focused on refining methods of classification and risk prediction. Studies have documented the use of varying datasets and different techniques in academic performance and achievement prediction. These studies have shown the need for further and more methodical studies in the field.

The current study explores student characteristics related to different components of the learning process and environment. Factors such as attendance, study hours, and completion of assignments reflect the level of participation and engagement in the learning process. Past grades and failures describe past learning experiences. Other variables included in this study describe the family situation and social environment of a student. These variables describe the student’s context and environment and could be associated with learning opportunities and academic achievements.

Academic risk is defined as the probability of not achieving academic goals and standards. For this study, academic risk is considered at three levels: safe, at-risk, and high-risk. Traditionally, the term ‘academic risk’ is used to describe students who failed or dropped out of a course. Other students with less severe academic concerns are often unrecognized. In the field of early-warning research, the use of machine learning in recognizing at-risk students has gained popularity, and the focus has shifted to the timely implementation of remedial measures [1]. Given the variability in the student data, the statistics and data analysis should take this into account. Both numerical and categorical variables were reported in the student data. To use the variables in classification, certain data transformations are required. In the current study, categorical variables were converted to numerical values. Numerical variables were analyzed after replacing missing values and standardizing the values.

The study investigates several learning strategies using Logistic Regression, Gaussian Naive Bayes, SVC, KNN, Decision Tree, Extra Trees, Random Forest, and Bagging. Different classification behaviors are analyzed by various modeling principles, such as classification by a line, classification by probability, classification by similarity, and classification by a tree. This provides an opportunity to analyze classification behavior without distinguishing the principles of data processing. This kind of analysis is important, since no ML algorithm has been shown to be clearly dominant in the field of educational data prediction [2]. To assess models, several methods have been used, such as classification by a line, by probability, and by a tree. Moreover, to evaluate classification behavior, an analysis of classification by a tree was done using a one-against-the-rest approach. To gain a deep understanding of classification behavior, other methods such as a five-fold cross-validation and analysis of accuracy, precision, recall, and F1-score were performed.

Because classification results should be explained by a human, the influence of each feature on the classification result was analyzed. Interpreting the results allows teachers to better understand the reasoning for classification and provide suggestions for improving student learning [3]. For this research, feature importance was analyzed to improve understanding of the classification of academic risk. While there are studies that explore the use of ML to predict academic performance and dropout using different terminologies and frameworks, there are very few studies that use a unified framework for the classification of academic risk to include behavioral and contextual factors to identify Safe, At-Risk, and High-Risk students. The current study proposes and implements a comprehensive machine learning framework to classify levels of academic risk and to identify characteristics of students based on that classification.

The problem statement can be addressed as follows:

There are several applications of ML and EDM to detect at-risk students, including students who experience academic underachievement and students who leave the education system. These studies have assessed and integrated behavioral, social, and demographic factors and employed different classification techniques. Prediction of student dropout and changes in academic status are closely related and have been studied extensively. In one study, the researchers formulated a system to predict student dropout using the integration of different engineering and scientific techniques in ML and responded to class imbalance using SMOTE. Other studies have focused on the integration of different techniques in classification and employed different approaches to address class imbalance and dimensionality. These studies have laid the foundation for the use of data science and integration of different techniques to address class imbalance and other challenges that hinder the identification of at-risk students [4].

Evaluation of traditional classification techniques has been studied as well. For example, a study evaluated the performance of Decision Trees and Random Forests in predicting students' academic performance [5]. The authors suggested that evaluation procedures should be standardized when comparing the performance of different classification techniques. Data balance has been an important consideration for predicting educational outcomes. The effect of SMOTE data augmentation has been studied by [6] for a range of educational datasets, and the authors reported that resampling techniques greatly impacted the classification results. Similarly, [7] performed classification for predicting secondary school graduation using Logistic Regression, Random Forest, XGBoost, and SVM and reported that implementation details and choices of preprocessing techniques influenced the results significantly. The above-mentioned studies suggest that a combination of appropriate data processing and model selection is critical for developing high-quality educational prediction systems.

One of the recent advances in the area of educational predictive systems is presented by [8], where the authors proposed an LSTM autoencoder-based framework for educational decision-making. Used a range of classification and regression models for postgraduate students' academic outcome prediction and concluded that the integration of various types of student data (e.g., academic, demographic, and pre-enrollment data) enabled them to capture various types of student performance and behavior. These studies demonstrate the application of predictive learning to capture different educational outcomes using advanced and complex architectures [9].

Other research focuses on student-related issues of a broader nature and on other methodological issues [10]. One study used different ML methods to create models that assess the depression risk of university students using variables related to academics, finance, and social relationships. This study [11] illustrates that, with ML, student variables beyond academics can be analyzed. A systematic review focused on prediction studies of various clusters and discussed the challenges of using ML in different institutions. Another study reported that, in the context of ML and educational research, the use of different algorithms for the same task can lead to inconsistent results and, therefore, different conclusions, and this is particularly valid when the performance of the models is analyzed without sufficient justification.

The same type of ML research focused on prediction of academic performance has been extended to other fields [12]. For instance, one study used different ML methods for classification and comparison in the field of finance, and reported that, although the data may be related to different fields, the use of different ML methods can help address the same problem in different ways. Other studies [13] have also extended research focused on predicting student dropout to other areas beyond academic performance, and reported the use of ML methods for predictions and integrative, interpretable, and exploratory analyses.

Research on ML algorithms indicates that no one algorithm covers all territories in the field of education. Research focused on the prediction of academic performance (PAP) is comprehensive, large-scale, and addresses the differences in datasets, algorithms, variables, and metrics [14]. Other studies in the area of PAP address the balance and outliers in the samples. Research on predicting academic achievement and dropout focuses on varying algorithms, which cover different frames of the educational domain [15]. Likewise, research in predicting academic performance and dropout concentrates on the variability of the algorithms and addresses the scarcity and imbalance in the samples, using multiple classifiers and the majority voting scheme [16].

Predicting student multilevel academic risk is not extensively studied, and literature focusing on other academic outcome risks, including underachievement, is even more scarce. Most studies focus on predicting academic outcome risks, student under- and overachievement, failures, and student dropout. The reasons for student underachievement may vary at different levels and may include the student’s academic, behavioral, and social contextual conditions. Most of the works address the variability of student outcome risks by focusing on student achievement, behavior, underachievement, and dropout. This is done by exploiting the differentiability and plasticity of ML algorithms, and typically there is little to no concern for the interpretability of the models.

This study aims to build upon previous research by creating a more nuanced academic risk classification framework. The framework will consist of three classes: safe, at risk, and high risk. The study will gather and analyze data based on traditional academic indicators as well as behavioral and contextual data. The study will use standard machine learning (ML) classifiers after adjusting and selecting the most relevant features. The study aims to gain a comprehensive understanding of each classifier by using cross-validation and alternative assessments of model behavior. To extract the features most relevant to risk classification, the study will assess and evaluate the features in relation to risk classification. Overall, the goal of this study is to comprehensively assess and analyze risk in the academic context as opposed to predicting or explaining a single outcome (i.e., dropout).

This study presents a machine learning model for labeling students as At-Risk, High-Risk, or Safe. The model development process consisted of several steps, including data preparation, transformation, model building, and testing. The complete process is illustrated in Figure 1.

The experimental analysis is focused on the evaluation of the student academic risk classification framework developed during the course of this research. The evaluation is based on the results of the models across various validation folds. The features selected, which encompassed the student’s academic and behavioral performance as well as contextual features of the learning environment, were also considered. Classification differences between the considered learning approaches were analyzed. Various classification and regression techniques were employed to address the research questions. Classification performance of the developed models was analyzed using confusion matrices and ROC curves. The training and test generalization was also analyzed. The final element of the analysis was to explore the reasons for the academic risk classification. These analyses provided a foundation for the interpretation of the framework and helped identify the blind spots in classification.

The goal of this study was to develop a machine learning framework to classify students as At-Risk, High-Risk, or Safe based on various behavioral and psychological factors, as well as academic data. Eight classifiers were trained and tested using multiple metrics within the same parameter limits. Out of all the models assessed, Support Vector Classifier (SVC) showed the strongest performance, with an accuracy of 93.63%, and corresponding values of precision, recall, and F1 score of 93.64, 93.63, and 93.63, respectively. The assessed models showed variations in their predictive behaviors and in the ways they generalized to unseen data. According to variable importance analyses, predictors of academic risk included, but were not limited to, attendance, study hours, completion of assignments, past grades, and past academic failures. In summary, results revealed that the use of ML techniques to classify academic risk is viable and supports the case for using data to find learners who may benefit from proactive support and intervening before it may be too late. For this research, we will be able to assess the generalizability of our ML framework for academic risk by using extended, real-world, and longitudinal student data from numerous institutions. To further enhance the ML academic risk framework and allow for its broader use, we will integrate Explainable AI and ensure ML models used embody standards of fairness and equality.