• <tr id="yyy80"></tr>
  • <sup id="yyy80"></sup>
  • <tfoot id="yyy80"><noscript id="yyy80"></noscript></tfoot>
  • 99热精品在线国产_美女午夜性视频免费_国产精品国产高清国产av_av欧美777_自拍偷自拍亚洲精品老妇_亚洲熟女精品中文字幕_www日本黄色视频网_国产精品野战在线观看 ?

    Interpretable and Adaptable Early Warning Learning Analytics Model

    2022-08-24 03:29:12ShaleezaSohailAtifAlviandAasiaKhanum
    Computers Materials&Continua 2022年5期

    Shaleeza Sohail,Atif Alviand Aasia Khanum

    1The University of Newcastle,Sydney,2000,Australia

    2University of Management and Technology,Lahore,54770,Pakistan

    3Forman Christian College,Lahore,54600,Pakistan

    Abstract: Major issues currently restricting the use of learning analytics are the lack of interpretability and adaptability of the machine learning models used in this domain.Interpretability makes it easy for the stakeholders to understand the working of these models and adaptability makes it easy to use the same model for multiple cohorts and courses in educational institutions.Recently,some models in learning analytics are constructed with the consideration of interpretability but their interpretability is not quantified.However,adaptability is not specifically considered in this domain.This paper presents a new framework based on hybrid statistical fuzzy theory to overcome these limitations.It also provides explainability in the form of rules describing the reasoning behind a particular output.The paper also discusses the system evaluation on a benchmark dataset showing promising results.The measure of explainability,fuzzy index,shows that the model is highly interpretable.This system achieves more than 82%recall in both the classification and the context adaptation stages.

    Keywords: Learning analytics;interpretable machine learning;fuzzy systems;early warning;interpretability;explainable artificial intelligence

    1 Introduction

    Learning analytics(LA)has been defined as“the measurement,collection,analysis and reporting of data about learners and their contexts,for purposes of understanding and optimizing learning and the environments in which it occurs”[1], with the two most frequently used tasks being predicting students’success and providing feedback[2].LA is an active area of research and several studies have been done over the years to enhance its utility for various stakeholders including but not limited to teachers,students,and institutions.Current methods in LA have shown a lot of promise and are being applied in various setups with appreciable returns.One common outcome of these systems is to find at-risk students in early weeks of the course to provide extra support to facilitate them in successfully completing the course.The results of assessments and students’learning behavior are used as indicators of their success in completing programming courses [3].Similarly, a large amount of literature has shown that with correct and timely analysis of data from learners, learning processes, and learning environments the success rate in higher education can improve significantly[4].

    There are two basic limitations that are preventing across-the-board adoption of these methods with full confidence.First,these methods lack adaptability,and second there is poor understandability of the results of these models.Each course offering and the cohort has its own nuances like class size,students’demographic background,the year in which the student registers the course etc.Traditional Machine learning(ML)models are strongly dependent on the dataset used for training.Due to this reason, most LA systems are beneficial for only a specific type of context and cannot be adapted to different contexts involving other courses or other institutions.Poor understandability is a very common issue in ML approaches.Especially in this domain of learning analytics,such understanding is an important factor that limits the adaptability of these systems by different stakeholders.In recent years,interpretable and explainable ML techniques have gained a lot of popularity to design intelligent learning systems providing explanations of the predictions that are easily comprehended by common users [5].Even though there is no agreement on the definition of interpretation in the ML research domain,one simple definition can be“the ability to explain or to present in understandable terms to a human”[6–8].In the next section,recent literature on the use of interpretable ML for LA is reviewed,and as none of the research contributions measures the interpretability of student prediction systems,any comparison and evaluation is difficult to conduct.

    In this paper,the hybrid framework proposed for facilitating the portability of models for learning analytics [9] has been extended by adding a new module, interpretability optimiser, that focuses on measuring and enhancing the interpretability of the ML approach.Earlier,a two-step hybrid approach for implementing the early warning system was used in our previous work.The first step is a datacentric algorithm that processes the data collected from the Learning management system(LMS)to learn a decision tree(CART and REPTree)giving the statistical relationship of various student-related parameters to the final course outcome.In the second step, fuzzy rules were constructed from the decision tree and inference was applied to predict a given student’s likelihood of passing a course on the basis of his/her scores in the assessments so far.In the present paper,the framework has been enhanced by incorporating feature selection,context adaptation,and interpretability analysis.Feature selection gives us the best features to predict students at risk of failing the course during the early phase of the course.Context adaptation helps us to apply the learnt relationships between student data and course results across varying courses without having to be relearnt and remodeled.A comprehensive testing of the proposed approach has been performed using publicly available educational datasets.The transformative function shows the portability of the model across different courses.In addition,the interpretability of the ML model is analysed using fuzzy index[10].The main contributions of the paper are:

    ·A statistical fuzzy framework enabling early intervention to help weak students.

    ·Adaptability of the model to new courses without retraining.

    ·Production of a set of linguistic fuzzy rules that are highly interpretable.

    ·Quantification of the interpretability of the fuzzy rules.

    ·High recall of both the classification and adaptation modules.

    ·Responsive and easy to use system.

    Remainder of this paper is organized as follows.Section 2 discusses various research reports on LA, particularly addressing the interpretability and adaptability aspects.Section 3 describes the materials and methods used in the present study.Section 4 discusses the results.Lastly, Section 5 concludes the paper with recommendations for future research.

    2 Literature Review

    The amount of research in the domain of learning analytics involving ML approaches is tremendous.Surprisingly, very few research efforts consider explainability and interpretability of the ML models in this context.A few interpretable techniques are proposed for predicting students’performance based on data collected from learning management systems, assignment marks and other enrolment systems and provide interpretability of the results.However, none of the approaches measures the interpretability of student prediction systems,and hence any comparison and evaluation is difficult to conduct.When it comes to measuring interpretability of ML models,a common approach is to use qualitative methods using human subjects to analyse the perceived interpretability of these models [11].However, due to the difference of application domains and the nature of predictions,these methods may not be feasible and suitable.Hence,we use a quantitative method to evaluate the interpretability in the proposed system as it provides a way to objectively compare different ML models without involving human subjects.A method proposed for quantitatively measuring interpretability is discussed in Section 3.

    Moreover,the literature on quantification or measurement of explainability of AI models used for learning analytics is even more limited.In this section,a comprehensive overview of literature in the domain of learning analytics is provided with a focus on interpretability.

    Socio-demographic data in addition to academic and LMS activity data is used to predict student performance in order to provide early support for at-risk students [12].The three datasets are partitioned into sub-datasets to form student subgroups at first level on the basis of gender,age,attendance type and attendance mode.On the second level the gender based sub-datasets are further partitioned into 6 sub-datasets according to student age,attendance type and attendance mode.The student performance is predicted using four classification methods using the sub-datasets:two black box methods are na?ve-Bayes and SMO and the two white box methods are J48 and JRip.The black box techniques produced models that were not interpretable and hence cannot be effectively used for intervention purposes.The white box techniques produced interpretable models by identifying features that impacted performance predictions for different sub-groups.Due to the comprehensibility of the predictions in the rule and tree form,these models can be used for early intervention purposes[12].

    Most early warning systems predict students’performance using large student datasets that ignore the idiosyncrasies of underrepresented students and consider general student population data only[13].Moreover, the lack of decision making reasoning makes these systems difficult to adopt by institutions.Educators cannot use these systems for providing effective student support due the inability of these systems to provide motivation for particular predictions.An interpretable rule based genetic programming classifier by using student data from multiple sources can predict at-risk students, especially the ones belonging to underrepresented minorities.These students are generally faced with more challenges in their educational journey and hence,this interpretable prediction can assist educators to provide early intervention effectively.The performance of multi view programming approach is consistently better when compared with other white box rule and tree based traditional approaches.Moreover, the rules provide comprehensibility of predictions for at-risk and not at-risk classes for easy adaptation and understanding[13].

    The interpretation of rules is provided by CN2 rule inducer and multivariate projection for the student performance prediction system that uses video learning analytics and data mining techniques.Student academic data,student activity data and student video interaction data are used to predict student performance by multiple algorithms.In addition,the effect of feature selection and transformation is also compared.The best prediction accuracy was achieved by the Random Forest algorithm with equal width transformation method and information gain ratio selection technique.The CN2 rule inducer algorithm also performed well but it provided easy rule induction with probability for non-expert viewers like educators that require this interpretation for providing support to students[14].

    High drop-out rate and poor academic performance are two main issues affecting the reputation of educational institutes[15].Association rule algorithm with Classification Based on Algorithm(CBA)rule generation algorithm can predict student performance in advance.This provides interpretable information to educators to adjust their teaching strategies and to provide additional support for struggling students.The experiments showed medium accuracy in predicting student performance,but it provides insight into the factor affecting the performance.

    The student performance prediction systems that use black box techniques can be converted to interpretable systems by employing some design practices [16].A black box prediction model,C-parameterized margin, SVM classifier (C-SVC) was chosen due to its ability of providing high prediction accuracy,and the interpretability of the system was enhanced by leveraging a rich catalogue of data visualization methods.Three features were considered in the design of the classifier to improve impressibility:multiclass, probabilistic and progressive.Hence, a progressive multiclass probabilistic prediction model not only provided high prediction accuracy but was interpretable so the educators can use it to support their students.A set of representation tools was discussed to provide expressive graphical output.

    A knowledge gap has been identified between the model creation for student performance prediction and the interpretation of that prediction for actionable decision-making process.For this pedagogical change,a model based on recursive partitioning and automatic selection of features for robust classification was developed with high interpretability.The strength of the model was the transparent characterization of student subgroups based on relevant features for easy translation into actionable processes[17].

    For student performance prediction systems, an important aspect that must be considered is prediction uncertainty or confidence in addition to prediction accuracy for developing a reliable early warning system for at-risk students.Two Bayesian deep learning models and Long Short-Term Memory(LSTM)models were used for predicting students’performance in future courses based on their performance in courses already completed by the students.Prediction uncertainty associated with at-risk student prediction was considered before reaching out to provide additional support for effective and targeted utilization of resources.Also, the explainable results of the models provided information about the previous courses whose results influenced the prediction,which can be used for guiding students[18].

    A combination of black box and white box prediction approaches was proposed for high prediction accuracy and interpretability.High prediction accuracy was achieved by using the SVM model and for interpretability the Decision Tree(DT)and Random Forest(RF)models were employed for extracting symbolic rules.In addition,an attribute dictionary was built from students’comments which was converted to attribute vectors for predicting students’grades.The combination of these techniques showed accurate prediction of students’performance based on students’comments after each lesson and the interpretable results showed the characteristics of attribute patterns for each grade[19].

    The above mentioned literature provides interpretable student performance prediction models but the quantification and measurement of the interpretability is not considered in any of these[20].For quantification of explainability models, generally user studies and qualitative approaches are used[21].However,the interpretability of models can be iteratively enhanced by employing relevant quantification methods which also provides means to compare these approaches.Interpretability indices have been proposed for white box approaches like rule-based systems which can effectively be used to quantify the explainability of these models for comparison and evaluation[22].

    3 Proposed Approach

    The main components of the proposed framework are discussed in this section and Fig.1 provides an overview of the framework.

    Figure 1:System overview

    3.1 Decision Trees

    Decision Trees are an established and efficient tool in Machine Learning that are increasingly being adopted to support explainability of algorithmic decisions of classification and regression.Various algorithms have been developed for constructing optimized decision trees under various conditions.Some examples include CART,ID3,REPTree,C4.5 etc.

    Tree algorithms apply a top-down,divide-and-conquer approach to the data to construct a tree or a set of rules.In a decision tree, the inner nodes represent value sub-ranges of the input variable and the leaf nodes represent the output values.The tree is constructed by recursively splitting the dataset by applying statistical measures to the variables and selecting a split variable based on the results.Some examples of statistical measures are Entropy,Information Gain,and Gini Index.Once the tree has been constructed, various input-output mapping rules can be traced out by traversing the tree from the root to a leaf.Moreover, there are algorithms for optimizing the tree in terms of its complexity.Pruning algorithms are the most popular in this category and many tree construction algorithms incorporate pruning in their operation based on various criteria,e.g.,reduced error pruning(REP)is used by a REPTree.In the present work,the REPTree algorithm is used,which is an efficient decision tree algorithm capable of learning both classification and regression problems.

    3.2 Construction of Fuzzy Inference System

    Fuzzy inference systems (FIS) have been used effectively in various domains including pattern recognition,healthcare,robotics,and control engineering etc.Based around Fuzzy sets(FS),F(xiàn)IS are ideally suited to domains with a large number of complex factors and non-linear relationships which cannot be expressed by clear mathematical equations.FIS’s also have the unique advantage in terms of their adaptability.Unlike other modeling approaches like neural networks,regression,etc.,which need to be redeveloped and retrained for every new context,F(xiàn)IS can be easily adapted to the new context without having to be retrained.

    A fuzzy set(FS)is an extension of the classical set where members of a set have different degrees of membership in the set.FS offer an ideal representation tool to represent imprecise and approximate concepts.FS are used to define linguistic variables to partition a Domain of discourse (DD).For instance, class size may be partitioned into three linguistic variables {low, medium, high}, with each linguistic variable expressed as a FS μA(x)→[0,1]where A is the linguistic variable,x is any variable from the DD,and μ represents the degree of membership in the FS.Membership functions(MF)can take various forms, e.g., Gaussian, trapezoidal, triangular, sigmoidal etc.In the context of current research,two types of MF’s are important.Gaussian membership functions of the form Eq.(1)below are used by fuzzy sets inside the domain:

    wheremand σ are the mean and standard of the domain respectively.Sigmoidal MF’s of the form Eq.(2)below are used by fuzzy sets on domain boundaries:

    whereaandcare the slope and crossover point respectively.For most of the applications,normal fuzzy sets are preferred which require that at least one memberx∈Xsuch that μA(x) = 1where X is the DD.The set of all members having a membership degree of 1 in the FS constitute the core of the FS.Correspondingly,the support of a FS is a crisp set containing all members of DD having a non-zero membership in the FS.

    An FIS is a rule-based decision system making use of fuzzy sets.Various fuzzy operators are applied to aggregate the rule activations in the RB.The main components of an FIS are the rule base(RB)and the knowledge base(KB).The rule base comprises various rules of the form:

    whereare the input features,F(xiàn)ikare the input linguistic variables,piare the output variables,andPiare the output linguistic variables.The definitions of fuzzy sets corresponding to the input and output linguistic variables are stored in the knowledge base.Input is provided in parallel to all the rules which are fired according to the degree of activation of the antecedent fuzzy sets.An Inference Engine maps the rule activations to output fuzzy sets which are then aggregated and optionally defuzzied to produce the output.For extracting the fuzzy rules from the decision tree, two steps are performed.First,appropriate fuzzy sets are defined to represent the feature sub-intervals represented by the tree nodes.In the second step, various pas are traced out from the root to the leaves in the tree; each of these paths is converted to a fuzzy rule by mapping the input-output variables to the corresponding fuzzy sets.

    3.3 Interpretability Analysis

    Once the fuzzy rules and fuzzy sets are created the interpretability of the FIS is measured.A number of approaches have been proposed in literature for measuring interpretability of rule-based systems and tree based systems[23].The most common indices[22]used for fuzzy logic systems are fuzzy index and Nauck index.In this paper, the interpretability of the proposed system is evaluated using the fuzzy index.

    Fuzzy index is proposed as an interpretability measure for fuzzy systems and is inspired by the Nauck index[22].The fuzzy index is assessed using a hierarchical fuzzy system with six input variables and one output variable for the four linked knowledge bases that are part of the complete systems.Four different knowledge bases calculate:

    ·Rule Base Dimension of the system:considering the number of rules and premises.

    ·Rule Base Complexity of the system:considering the number of rules with one,two and three or more variables.

    ·Rule Base Interpretability of the system:considering Rule Base Complexity and Rule Base Dimension of the system.

    ·Fuzzy index of the system(final output):considering Rule Base Interpretability and average number of labels defined by input variables.

    The labels of the six inputs are identified and the four rule bases in the hierarchical fuzzy system work together to find the fuzzy index of the fuzzy system.The inputs and output of the system that calculates the interpretability of the fuzzy system are shown in Tab.1.The readers are encouraged to read[22]for further details on the calculation of the fuzzy index.

    Table 1:Input and output variables for the fuzzy index[11]

    Table 1:Continued

    3.4 Context Adaptation Module

    As mentioned earlier,F(xiàn)S can be adapted to new contexts without needing to be retrained.Context is extremely important in correctly applying an LA model to predict future performance of the students.For instance,a class of size 30 might be considered large in case of an elective course but would be considered medium for a core course.However,research on context-adaptation for LA applications is scanty so far.

    It is generally agreed that, among the two main components of an FIS, the RB is universal and context-independent whereas the KB is context-aware.Accordingly, an FIS can be adapted by transforming the KB according to the context without affecting the logical structure of the RB.The method given in[24](for the construction domain)is used for applying the transformation functions to the KB.As an example,the procedure involved in adapting a fuzzy set A from a base context B to its adapted version ?Ain a new context N is described by the following steps:

    Initially, the KB is populated with the definition of A according to the base context B defined over the DD[bi,bh].Since Gaussian MF’s are being used,the parameters of A will be(x,m,σ),wheremand σ are the mean and standard deviation respectively.When context adaptation is required,first of all determine the range of the new context N as[ni,nh],wherenl=min(x1,x2,...,xn)andnh=max(x1,x2,...,xn)wherexiare the feature values over the new domain.A context adaptation factor is then calculated as:

    The context adapted version of A is calculated by replacing the base parameters with the adapted values as follows:

    wherex∈[ni,nh] and τ(x)∈[bl,bh].The reader is encouraged to read for further details of these equations.

    4 Model Implementation and Evaluation

    In this section the implementation and evaluation of the proposed model is discussed.

    4.1 Dataset

    The validation of the proposed framework has been carried out using the Open University Learning Analytics Dataset (OULAD) [25], which is offered by the Open University, UK, one of the largest distance learning universities in the world.OULAD comprises anonymized student demographic and performance data for various course modules offered by the Open University in the years 2013 and 2014.The data captures registrations,demographics(gender,age band,region,index of multiple deprivation),assessments, and virtual learning environment interactions of the students.All data is tabular in CSV(comma separated values)format and the tables can be interlinked using key columns.The intent was to predict the passing or failing of a student early in a course by using machine learning and fuzzy logic on the student’s data.

    4.2 Preprocessing

    To extract the tree, student data for 2 offerings of the same course in 2013 and 2014 has been used,represented in OULAD as course BBB with offerings 2013B and 2014B respectively.There were 8590 relevant records of 2531 distinct students,considering that only those assessments were chosen that were marked by the tutor(and not by the computer),as they carried the majority of the marks,and were due at most by the 120th day of the course, since an early intervention based on these is desired.There were 4 such assessments for each of the chosen course modules.All the relevant data was combined into one file and was then further processed by taking the average of the scores obtained in the aforementioned assessments for each student.As it was required to have a binary classification of the final result of a student as Pass or Fail,the instances of a final result of Distinction were renamed as Pass,and Withdrawn as Fail,without a loss of accuracy.Context adaptation was later applied to another cohort comprising 2 courses represented as AAA in OULAD with offerings 2013 J and 2014 J respectively.The data was preprocessed as for the pre-classification step and resulted in 678 relevant records that were then input to the context adaptation module(see Section 4.7).

    4.3 Classification Using Decision Tree

    Decision tree learning was preferred as the predictive technique because it uses a white-box model and is thus easy to explain and interpret.Moreover,it is computationally less intensive and requires less data preprocessing.The Scikit-learn library for CART(Classification and Regression Trees)algorithm and REPTree(Reduced error pruning)algorithm]from WEKA(Waikato environment for knowledge analysis)library]were used.

    Both algorithms gave similar results.CART’s binary tree was built by splitting nodes on the basis of Gini impurity index.Pandas and Numpy were used for data manipulation.WEKA’s REPTree algorithm builds a decision or regression tree using information gain/variance reduction and prunes it using reduced-error pruning.Optimized for speed,it only sorts values for numeric attributes once.It deals with missing values by splitting instances into pieces,as C4.5 does.You can set the minimum number of instances per leaf,maximum tree depth(useful when boosting trees),minimum proportion of training set variance for a split(numeric classes only),and number of folds for pruning.

    The features found to be most important for classifying a student as likely to Pass or Fail are:i)the student’s average score in the assessments,ii)highest education level attained by the student previously,and iii)the index of multiple deprivation,which is essentially a poverty measure.A training-test split of 80-20 was used to build the REPTree model.Fig.2 shows the REPTree for the training data.

    Figure 2:REPTree model of training data

    4.4 Construction of Fuzzy Inference System

    After obtaining the tree,each path from the root to a leaf is traversed and the feature sub-intervals expressed in each of the nodes are extracted.After examining the collected feature sub-intervals,fuzzy linguistic variables were defined to express these sub-intervals.Following input features were fuzzified with the help of linguistic variables:F= {Score,Education,IMDBand}The output variable Outcome is also fuzzified.All linguistic variables employ normalized Gaussian membership functions defined by(1)and(2),as explained in the previous section.These MF’s are incorporated into fuzzy IF-THEN rules of the form(3).An example MF is shown in Fig.3.An FIS is constructed around these rules,with these properties:FIS Type:MAMDANI,Inference:MIN-MAX and defuzzification:Centroid.The FIS was further tuned to optimize the rule structure and composition.The FIS with three input features and one output feature is shown in Fig.4.The system is capable of reflecting the complex and nonlinear relationships between various variables.Fig.5 shows the rule surface underscoring the nonlinear relationship between the inputs and the output.

    Figure 3:Fuzzy sets for score

    Figure 4:FIS structure for grade prediction

    Figure 5:Non-linear relationships between the FIS variables

    4.5 Interpretability Analysis

    The parameters used for calculation of the fuzzy index of the FIS are shown in Tab.2:

    Table 2:Input and output variables for the FIS to calculate the fuzzy index

    Using the values given in Tab.2, the outputs of the four knowledge bases designed to find the fuzzy index[10]for this system are given below:

    ·Rule Base Dimension of the system:Low.

    ·Rule Base Complexity of the system:Low.

    ·Rule Base Interpretability of the system:Very high.

    ·Fuzzy index of the system(final output):Very high.

    The fuzzy index of the designed FIS is evaluated as very high which makes the system highly interpretable and hence,easy to adapt by professional and academic staff in educational institutions.

    4.6 Context Adaptation Module

    Context adaptation was also experimented with by using the existing data of previous cohorts to predict the performance of a new cohort(see Section 4.3) during the early phase of a semester.The parametersmand σ were adapted for the new cohort using the previous cohort as the base domain.It was observed that only the parametermwas enough to give sufficiently acceptable performance in our domain, so only μ has been used for the reported results.For each of the input features, the corresponding linguistic variables in the KB were adapted using the method stated in Section 3.4 above.FIS performance was tested on the adapted KB on the new domain,without making any changes to the RB.

    4.7 Results and Discussion

    Tabs.3 and 4 depict the performance of the modules.For the particular scenario of enabling early intervention to help students likely to fail a course, recall is the most important performance metric since it is imperative not to miss any potentially failing student in the results.Hence,having less false negatives(high recall)at the expense of having more false positives(low precision)is acceptable.The proposed system achieves more than 82% recall in both the REPTree classifier and the context adaptation module (Tabs.3 and 4 respectively).An important aspect of the model is its ability to predict students’performance by considering only a subset of the assessments undertaken in the first few weeks of the semester.This provides an opportunity for early intervention by providing academic support to these students.This inherent requirement of using a subset of data to predict students’performance lowers the prediction accuracy but provides crucial data required for successful early intervention to support academically weak students.Moreover,the fuzzy index of the proposed system is Very High (calculated in Section 4.6), which makes the predictions easy to understand and use for early intervention purposes for all the stakeholders.Due to a tradeoff between system accuracy and interpretability, more interpretable systems tend to be less accurate.Despite this limitation, the proposed system shows acceptable accuracy and high interpretability.

    Table 3:REPTree detailed accuracy by class

    Table 4:Accuracy details for adapted FIS

    Considering the class imbalance present in the dataset, weighted averages of the performance measures are also reported,using the class size as a weight.It is clear from Tab.3 that the weighted average is well above 50%for both precision and recall.

    5 Conclusion and Future Work

    This paper presented a comprehensive approach to provide an early answer to the million-dollar question in learning analytics:which students are likely to fail this course?Early intervention can put such students back on the path of success.This hybrid statistical fuzzy system identifies such students using their performance in the initial assessments of the course and a few other features through a learning decision tree and then generates a set of fuzzy rules.These rules are easy to understand,and this is quantified by measuring their interpretability using the fuzzy index.Finally,the FIS is context adapted and used to predict the likelihood of student success and failure in other courses,without any retraining being required.The performance of the system is high in terms of recall,the main parameter of success in this scenario.

    Some of the limitations of the proposed system are due to the fact that the system is designed to identify students that are at risk of failing the course considering only the subset of assessments undertaken in the early weeks of the semester.Hence,the accuracy of the system is not very high when compared to other models based on data from all assessments of the course.The other limitation of the system arises from the requirement of high interpretability which necessitates keeping the decision tree simple.

    The proposed system is an end-to-end solution to a key learning analytics problem and this work shall be extended by using the framework on several datasets to discover more features that contribute to student success and failure.Other learning algorithms will also be used, both supervised and unsupervised,in the classification stage of the framework and their performance and interpretability will be measured.Another research thread to be pursued in future is the human in the loop paradigm to make this framework even more accurate,interpretable,and sentient.

    Funding Statement:The authors received no specific funding for this study.

    Conflicts of Interest:The authors declare that they have no conflicts of interest to report regarding the present study.

    极品人妻少妇av视频| 精品一区在线观看国产| 一区二区三区乱码不卡18| 丝袜在线中文字幕| 人妻制服诱惑在线中文字幕| 91国产中文字幕| 男女边吃奶边做爰视频| 国产成人aa在线观看| 又黄又爽又刺激的免费视频.| 亚洲成色77777| 日本色播在线视频| 亚洲,一卡二卡三卡| 欧美丝袜亚洲另类| 国产成人一区二区在线| 欧美日韩视频精品一区| 国产av国产精品国产| 妹子高潮喷水视频| 久久鲁丝午夜福利片| 久久99蜜桃精品久久| 精品一区在线观看国产| 亚洲精品成人av观看孕妇| 少妇精品久久久久久久| 大话2 男鬼变身卡| 免费久久久久久久精品成人欧美视频 | 精品国产一区二区三区久久久樱花| 亚洲天堂av无毛| 国产精品欧美亚洲77777| 热re99久久国产66热| 久久午夜福利片| 亚洲中文av在线| 国产精品一区二区三区四区免费观看| www.av在线官网国产| 欧美日韩综合久久久久久| 人成视频在线观看免费观看| 日产精品乱码卡一卡2卡三| 91aial.com中文字幕在线观看| 亚洲熟女精品中文字幕| 久久久久久伊人网av| √禁漫天堂资源中文www| 91在线精品国自产拍蜜月| 欧美 亚洲 国产 日韩一| 国产片特级美女逼逼视频| 日韩免费高清中文字幕av| 欧美 亚洲 国产 日韩一| 少妇熟女欧美另类| 三上悠亚av全集在线观看| 一级爰片在线观看| 欧美激情 高清一区二区三区| 亚洲经典国产精华液单| 三级国产精品欧美在线观看| 久久99蜜桃精品久久| 亚洲精品乱久久久久久| 国产日韩一区二区三区精品不卡 | 国产亚洲精品久久久com| 99热这里只有是精品在线观看| av线在线观看网站| 日日摸夜夜添夜夜添av毛片| 人人妻人人澡人人爽人人夜夜| 国产成人精品久久久久久| 一本—道久久a久久精品蜜桃钙片| 亚洲国产最新在线播放| 精品久久久噜噜| 夫妻性生交免费视频一级片| 18在线观看网站| 国产欧美日韩一区二区三区在线 | 青春草国产在线视频| 韩国av在线不卡| av女优亚洲男人天堂| 久久久久网色| 久久精品国产亚洲网站| 亚洲国产成人一精品久久久| 建设人人有责人人尽责人人享有的| 欧美丝袜亚洲另类| 涩涩av久久男人的天堂| 国产伦理片在线播放av一区| 夫妻性生交免费视频一级片| 亚洲精品久久久久久婷婷小说| 国产精品国产三级国产av玫瑰| 丰满乱子伦码专区| 日韩成人av中文字幕在线观看| 亚洲精品国产av成人精品| 国产成人a∨麻豆精品| 婷婷色综合www| 狂野欧美白嫩少妇大欣赏| 中文字幕亚洲精品专区| 亚洲精品久久成人aⅴ小说 | 免费人妻精品一区二区三区视频| 少妇人妻久久综合中文| 国产亚洲最大av| 插阴视频在线观看视频| 久久精品夜色国产| 亚洲国产精品国产精品| 国产成人午夜福利电影在线观看| 国产白丝娇喘喷水9色精品| 久久热精品热| 欧美bdsm另类| 日日摸夜夜添夜夜爱| 三上悠亚av全集在线观看| 久久精品人人爽人人爽视色| 九色成人免费人妻av| 国产片特级美女逼逼视频| av视频免费观看在线观看| 久久久午夜欧美精品| 久久影院123| 美女视频免费永久观看网站| 中文字幕人妻丝袜制服| 中文字幕最新亚洲高清| 成年av动漫网址| 亚洲人成77777在线视频| 各种免费的搞黄视频| 色婷婷久久久亚洲欧美| 一个人看视频在线观看www免费| 最黄视频免费看| 嫩草影院入口| 欧美亚洲日本最大视频资源| 久久久久久久大尺度免费视频| 免费av不卡在线播放| 中文字幕人妻熟人妻熟丝袜美| 热99久久久久精品小说推荐| 看免费成人av毛片| 免费高清在线观看视频在线观看| 少妇人妻 视频| 男女边吃奶边做爰视频| 免费av中文字幕在线| 日韩免费高清中文字幕av| 亚洲国产欧美日韩在线播放| 赤兔流量卡办理| 少妇高潮的动态图| 国产成人精品久久久久久| 亚洲av国产av综合av卡| 亚洲怡红院男人天堂| 9色porny在线观看| 婷婷色综合www| 香蕉精品网在线| av视频免费观看在线观看| 亚洲欧美精品自产自拍| 最黄视频免费看| 制服人妻中文乱码| av不卡在线播放| 精品人妻一区二区三区麻豆| 国产在线视频一区二区| 亚洲一区二区三区欧美精品| 青青草视频在线视频观看| 国产爽快片一区二区三区| 大陆偷拍与自拍| 精品久久蜜臀av无| 欧美亚洲日本最大视频资源| 永久网站在线| 啦啦啦中文免费视频观看日本| 亚洲图色成人| 国产男女超爽视频在线观看| 有码 亚洲区| 韩国av在线不卡| 国产精品一区二区三区四区免费观看| 久久久久久伊人网av| 国产精品一区www在线观看| 国产精品不卡视频一区二区| 七月丁香在线播放| 久久久精品区二区三区| 免费观看av网站的网址| 国产片内射在线| 中文字幕制服av| 一级毛片黄色毛片免费观看视频| 一级片'在线观看视频| 国产免费一区二区三区四区乱码| 久久人妻熟女aⅴ| 亚洲成人av在线免费| 边亲边吃奶的免费视频| 亚洲欧美成人综合另类久久久| 自拍欧美九色日韩亚洲蝌蚪91| 啦啦啦中文免费视频观看日本| 麻豆乱淫一区二区| 国产午夜精品一二区理论片| 91久久精品电影网| 2018国产大陆天天弄谢| 亚洲精品国产av成人精品| 久久久久久久久久久丰满| 岛国毛片在线播放| 免费人妻精品一区二区三区视频| 国产视频首页在线观看| 国产成人aa在线观看| 亚洲精品久久成人aⅴ小说 | 中文欧美无线码| 99九九线精品视频在线观看视频| 狂野欧美白嫩少妇大欣赏| 久热这里只有精品99| 观看av在线不卡| 国产精品久久久久成人av| 久久亚洲国产成人精品v| 菩萨蛮人人尽说江南好唐韦庄| 水蜜桃什么品种好| 国产男女超爽视频在线观看| 黄片播放在线免费| 中文字幕久久专区| 日本wwww免费看| 国产 精品1| 国产精品.久久久| 一级a做视频免费观看| 飞空精品影院首页| 亚洲av欧美aⅴ国产| 最近中文字幕高清免费大全6| 免费少妇av软件| 亚洲精品,欧美精品| videossex国产| 少妇被粗大的猛进出69影院 | 中文精品一卡2卡3卡4更新| 国产精品久久久久久精品古装| 亚洲第一区二区三区不卡| 免费大片18禁| 在线观看免费视频网站a站| av福利片在线| 国产乱来视频区| 99国产综合亚洲精品| 亚洲国产精品一区三区| 在线观看免费视频网站a站| 最新中文字幕久久久久| 国产日韩欧美视频二区| 成人毛片60女人毛片免费| 国精品久久久久久国模美| 日韩电影二区| 亚洲国产精品专区欧美| 晚上一个人看的免费电影| 两个人的视频大全免费| 午夜福利影视在线免费观看| 久久鲁丝午夜福利片| 又粗又硬又长又爽又黄的视频| 国产精品蜜桃在线观看| 汤姆久久久久久久影院中文字幕| 国产高清有码在线观看视频| 在线亚洲精品国产二区图片欧美 | 91国产中文字幕| 国产精品人妻久久久影院| 丰满少妇做爰视频| 99热这里只有是精品在线观看| 国产av一区二区精品久久| 精品熟女少妇av免费看| 成人国语在线视频| 国内精品宾馆在线| 一级毛片aaaaaa免费看小| 午夜福利网站1000一区二区三区| 一本—道久久a久久精品蜜桃钙片| 久久99热这里只频精品6学生| 99久国产av精品国产电影| 精品亚洲成国产av| 亚洲av日韩在线播放| 18禁裸乳无遮挡动漫免费视频| 夜夜骑夜夜射夜夜干| 一级爰片在线观看| 亚洲av欧美aⅴ国产| 亚洲欧洲精品一区二区精品久久久 | 一区在线观看完整版| 久久 成人 亚洲| 十八禁高潮呻吟视频| 欧美日韩综合久久久久久| 国产精品女同一区二区软件| 考比视频在线观看| 自拍欧美九色日韩亚洲蝌蚪91| 精品99又大又爽又粗少妇毛片| 国产黄色免费在线视频| 免费观看的影片在线观看| 亚洲精品美女久久av网站| √禁漫天堂资源中文www| av黄色大香蕉| 国产伦精品一区二区三区视频9| 建设人人有责人人尽责人人享有的| 肉色欧美久久久久久久蜜桃| 亚洲国产最新在线播放| 人人妻人人澡人人爽人人夜夜| 成人漫画全彩无遮挡| 国产色婷婷99| 中国三级夫妇交换| 久久精品国产a三级三级三级| 免费人妻精品一区二区三区视频| 精品人妻熟女av久视频| 两个人的视频大全免费| 免费日韩欧美在线观看| 中文字幕人妻丝袜制服| 人妻人人澡人人爽人人| 久久久久久久久久人人人人人人| av卡一久久| 伦精品一区二区三区| 国产男女内射视频| 极品少妇高潮喷水抽搐| 成人国产麻豆网| 丰满饥渴人妻一区二区三| 男女免费视频国产| 国产精品熟女久久久久浪| 国产精品.久久久| 人人澡人人妻人| 日韩精品免费视频一区二区三区 | 男女高潮啪啪啪动态图| 免费人成在线观看视频色| 日韩制服骚丝袜av| 黑人巨大精品欧美一区二区蜜桃 | 久久97久久精品| 亚洲欧美成人精品一区二区| 男女高潮啪啪啪动态图| 国产av一区二区精品久久| 精品午夜福利在线看| 国产乱人偷精品视频| 最黄视频免费看| 丰满饥渴人妻一区二区三| 久久久久久久久久久久大奶| 男人爽女人下面视频在线观看| 久久国产精品大桥未久av| 最近中文字幕高清免费大全6| 久久毛片免费看一区二区三区| 自拍欧美九色日韩亚洲蝌蚪91| 极品少妇高潮喷水抽搐| 亚洲av.av天堂| 如何舔出高潮| 91aial.com中文字幕在线观看| 狂野欧美激情性bbbbbb| 亚洲精品亚洲一区二区| 少妇人妻精品综合一区二区| 亚洲精品视频女| 国产片特级美女逼逼视频| 插逼视频在线观看| 18禁在线无遮挡免费观看视频| 一个人看视频在线观看www免费| 中文字幕最新亚洲高清| 91精品一卡2卡3卡4卡| 多毛熟女@视频| 国产欧美日韩一区二区三区在线 | 一区二区日韩欧美中文字幕 | av在线老鸭窝| 日韩av在线免费看完整版不卡| 国产成人免费无遮挡视频| 美女中出高潮动态图| 91aial.com中文字幕在线观看| 夜夜骑夜夜射夜夜干| 熟女av电影| 亚洲一级一片aⅴ在线观看| 天堂俺去俺来也www色官网| 99视频精品全部免费 在线| 久久影院123| 国产精品麻豆人妻色哟哟久久| 亚洲色图 男人天堂 中文字幕 | 日韩欧美精品免费久久| 蜜桃在线观看..| 草草在线视频免费看| 久久精品夜色国产| 久久av网站| 色吧在线观看| 美女视频免费永久观看网站| av.在线天堂| 久久青草综合色| h视频一区二区三区| 人人妻人人澡人人看| 韩国av在线不卡| 永久网站在线| 国产精品一区二区在线不卡| 亚洲少妇的诱惑av| 国产精品一国产av| 最近的中文字幕免费完整| 久久久国产精品麻豆| 美女国产视频在线观看| 亚洲四区av| 久久鲁丝午夜福利片| 色哟哟·www| 久久精品夜色国产| www.av在线官网国产| 国产精品秋霞免费鲁丝片| 简卡轻食公司| 国产成人精品福利久久| 日韩av免费高清视频| 国产精品久久久久久久久免| 99视频精品全部免费 在线| 久久综合国产亚洲精品| av在线老鸭窝| 男人操女人黄网站| 国产亚洲精品久久久com| 高清视频免费观看一区二区| 大码成人一级视频| 亚洲国产欧美日韩在线播放| 国产成人精品久久久久久| 中国国产av一级| 亚洲国产毛片av蜜桃av| 看非洲黑人一级黄片| 国产无遮挡羞羞视频在线观看| 伦理电影大哥的女人| 国产国拍精品亚洲av在线观看| 丰满饥渴人妻一区二区三| 99精国产麻豆久久婷婷| 一本一本综合久久| a级毛片在线看网站| 在线免费观看不下载黄p国产| 这个男人来自地球电影免费观看 | 草草在线视频免费看| 嘟嘟电影网在线观看| 男的添女的下面高潮视频| 女人久久www免费人成看片| 超色免费av| 免费观看性生交大片5| 日本91视频免费播放| 欧美日韩成人在线一区二区| 亚洲婷婷狠狠爱综合网| 日韩视频在线欧美| 国产国语露脸激情在线看| 亚洲国产色片| 国产免费现黄频在线看| 久久久精品94久久精品| 乱人伦中国视频| 中文欧美无线码| 欧美精品亚洲一区二区| .国产精品久久| 视频中文字幕在线观看| 日韩一区二区视频免费看| 少妇人妻久久综合中文| 亚洲av综合色区一区| 午夜福利网站1000一区二区三区| 国产精品久久久久久av不卡| 蜜桃在线观看..| 在线观看三级黄色| 内地一区二区视频在线| 国产午夜精品久久久久久一区二区三区| 高清欧美精品videossex| 精品熟女少妇av免费看| 少妇 在线观看| 街头女战士在线观看网站| 欧美日韩亚洲高清精品| 99热网站在线观看| 成人综合一区亚洲| 亚洲国产精品一区三区| 国产精品.久久久| 日韩欧美精品免费久久| 欧美精品人与动牲交sv欧美| 妹子高潮喷水视频| 亚洲国产av影院在线观看| av在线老鸭窝| 国产片内射在线| 99re6热这里在线精品视频| 卡戴珊不雅视频在线播放| 国产av码专区亚洲av| 女性被躁到高潮视频| 久久热精品热| 亚洲国产欧美日韩在线播放| av免费在线看不卡| 亚洲精品久久成人aⅴ小说 | 亚洲人成网站在线播| 亚洲国产av影院在线观看| 亚洲国产毛片av蜜桃av| 亚洲精品国产色婷婷电影| 99国产精品免费福利视频| 国产精品久久久久久av不卡| 亚洲av日韩在线播放| 女人精品久久久久毛片| 乱人伦中国视频| 777米奇影视久久| 热99久久久久精品小说推荐| 中文字幕av电影在线播放| 亚洲成人av在线免费| 国产老妇伦熟女老妇高清| 久久久亚洲精品成人影院| 亚洲av综合色区一区| 亚洲av成人精品一二三区| 丝袜喷水一区| 狂野欧美激情性xxxx在线观看| 国产在线视频一区二区| 亚洲精品国产av成人精品| 精品人妻偷拍中文字幕| 国产一级毛片在线| 一级毛片 在线播放| 国产淫语在线视频| 精品少妇久久久久久888优播| 免费av不卡在线播放| 中文乱码字字幕精品一区二区三区| 一级,二级,三级黄色视频| 久久精品久久久久久久性| 日韩不卡一区二区三区视频在线| 亚洲三级黄色毛片| 人人妻人人添人人爽欧美一区卜| 插逼视频在线观看| 久久人人爽人人片av| 99热国产这里只有精品6| 欧美精品高潮呻吟av久久| 国产亚洲最大av| 少妇高潮的动态图| 色婷婷久久久亚洲欧美| 欧美日韩在线观看h| 国产深夜福利视频在线观看| 2018国产大陆天天弄谢| 国产在线视频一区二区| av不卡在线播放| 蜜臀久久99精品久久宅男| 在线亚洲精品国产二区图片欧美 | 熟女av电影| 丝袜美足系列| 日韩一本色道免费dvd| 婷婷色av中文字幕| 日本午夜av视频| 狠狠精品人妻久久久久久综合| 久久精品国产自在天天线| 最近中文字幕高清免费大全6| 欧美精品亚洲一区二区| 国产有黄有色有爽视频| 丝袜美足系列| 国产一区二区三区综合在线观看 | av又黄又爽大尺度在线免费看| 日韩电影二区| 成人影院久久| 99国产精品免费福利视频| av在线播放精品| 性高湖久久久久久久久免费观看| 亚洲精品国产色婷婷电影| 亚洲精品久久成人aⅴ小说 | 国产片特级美女逼逼视频| 三级国产精品欧美在线观看| av电影中文网址| 中文字幕人妻丝袜制服| 一本大道久久a久久精品| av.在线天堂| 国产亚洲精品久久久com| 亚洲欧美精品自产自拍| 国产精品嫩草影院av在线观看| 制服丝袜香蕉在线| 国产老妇伦熟女老妇高清| 国产免费一级a男人的天堂| 国产老妇伦熟女老妇高清| 交换朋友夫妻互换小说| 51国产日韩欧美| 亚洲av中文av极速乱| 免费黄色在线免费观看| 欧美亚洲 丝袜 人妻 在线| 丰满饥渴人妻一区二区三| 91aial.com中文字幕在线观看| 国产乱人偷精品视频| 伦理电影免费视频| 亚洲av二区三区四区| 丝袜脚勾引网站| 国产极品粉嫩免费观看在线 | 免费不卡的大黄色大毛片视频在线观看| 少妇被粗大的猛进出69影院 | 久久精品夜色国产| 日韩制服骚丝袜av| 国产精品不卡视频一区二区| 最近手机中文字幕大全| 精品人妻偷拍中文字幕| 建设人人有责人人尽责人人享有的| 街头女战士在线观看网站| 97精品久久久久久久久久精品| 久久久亚洲精品成人影院| 久久久精品区二区三区| 日韩不卡一区二区三区视频在线| 亚洲欧美色中文字幕在线| 久久ye,这里只有精品| 免费看光身美女| 精品久久久精品久久久| 国产成人精品久久久久久| 国产在视频线精品| 久久国产精品大桥未久av| 国产精品久久久久久av不卡| 欧美激情极品国产一区二区三区 | 日日摸夜夜添夜夜爱| 日产精品乱码卡一卡2卡三| 婷婷色av中文字幕| 日韩不卡一区二区三区视频在线| 97在线人人人人妻| 大码成人一级视频| 国产av一区二区精品久久| 中国国产av一级| 亚洲国产日韩一区二区| 王馨瑶露胸无遮挡在线观看| 亚洲第一区二区三区不卡| 亚洲国产精品成人久久小说| 18在线观看网站| 国产69精品久久久久777片| 看非洲黑人一级黄片| 高清毛片免费看| 日韩中文字幕视频在线看片| 午夜免费观看性视频| 搡老乐熟女国产| 久久久久久久久久成人| 男男h啪啪无遮挡| 久久人妻熟女aⅴ| 亚洲欧美一区二区三区国产| 如日韩欧美国产精品一区二区三区 | 亚洲五月色婷婷综合| 欧美日韩在线观看h| 欧美成人精品欧美一级黄| 最黄视频免费看| 蜜桃久久精品国产亚洲av| 成人国语在线视频| 亚洲av成人精品一区久久| 亚洲av福利一区| 一区二区日韩欧美中文字幕 | 制服诱惑二区| 国产一区亚洲一区在线观看| 午夜精品国产一区二区电影| 国产成人免费观看mmmm| 少妇被粗大的猛进出69影院 | 黄色一级大片看看| 久久韩国三级中文字幕| 全区人妻精品视频| 高清欧美精品videossex| 高清黄色对白视频在线免费看| 亚洲精品成人av观看孕妇| 精品久久久久久久久亚洲| 精品亚洲成国产av| 国国产精品蜜臀av免费| 十八禁网站网址无遮挡| 国模一区二区三区四区视频| 男的添女的下面高潮视频| 国产亚洲一区二区精品| 搡女人真爽免费视频火全软件| 精品亚洲成a人片在线观看| 国产精品国产三级国产av玫瑰| 成年女人在线观看亚洲视频| 9色porny在线观看| 美女脱内裤让男人舔精品视频| 99热全是精品| 日本vs欧美在线观看视频| a级片在线免费高清观看视频| 青青草视频在线视频观看| 夜夜看夜夜爽夜夜摸|