Rasch Model Assessment for Bloom Digital Taxonomy Applications

Mohd Effendi Ewan Mohd Matore

Faculty of Education,Universiti Kebangsaan Malaysia,Bangi,43600,Malaysia

Abstract:Assessment using Bloom’s taxonomy levels has evolved in a variety of contexts and uses.In the era of the COVID-19 pandemic, which necessitates use of online assessment, the need for teachers to use digital-based taxonomyskills or Bloom’s DigitalTaxonomy(BDT)has increased even more.However, the existing studies on validity and reliability of BDT items are limited.To overcome this limitation, this study aims to test whether BDT has good psychometric characteristics as a teacher’s self-assessment tool using the Rasch model analysis and to investigate the pattern of BDT usage in teaching and learning.By using a quantitative online survey design,this study involves six levels of BDT,namely,Remembering,Understanding,Applying,Analyzing, Evaluating, and Creating.The questionnaire was developed and validated by two experts prior to administration.A stratified random sampling technique was conducted on 774 secondary teachers from five geographical zones in Malaysia,and the Rasch model was analyzed using WINSTEPS 3.71 software.The performances of items improved by Rasch psychometric assessment including the application of BDT among teachers.The hierarchy level was also assessed through graphical analysis,including the Wright map and bubble chart, to demonstrate the powerful performance of the Rasch model analysis in investigating item quality and reliability.Overall,these empirically validated items using the Rasch model could advance the academic knowledge of BDT for future assessment and promote the Rasch calibration in an educational setting.

Keywords: Bloom’s digital taxonomy; assessment; dichotomous score; Rasch

1 Introduction

The term “taxonomy” originates from the Greek words “taxis” and “nomos,” which refer to “order” and “method,” respectively.This term may be referred to as an arrangement or a law in a specific order that is borrowed from biology, which allows certain classifications of the order.In the development of effective methods to perform mental operations, the notion of ordering is essential to classify these operations and skills and to determine the formation sequences in order to grow and solve certain problems [1].Bloom’s taxonomy was introduced in 1965 by Benjamin Bloom [2] based on his idea of pedagogical taxonomy that refers to the development of a clear pedagogical goal system that focuses on cognitively, affectivity and psychomotor development.There are six cognitive activity levels surrounding the cognitive domain of Benjamin Bloom’s taxonomy, which range from the easiest to the hardest levels, including Knowledge, Comprehension, Application, Analysis, Synthesis, and Evaluation.The taxonomy was prudent to verbalize the objectives of education based on each cognitive activity level.In 2001,Revised Bloom’s Taxonomy (RBT) was introduced by Lorin Anderson and David Krathwohl by revising and developing Benjamin Bloom’s concepts into four dimensions:factual, conceptual,procedural, and metacognitive [2].The number of cognitive domains remains six with changes from noun to the verb form; namely, the domain names have been changed to:Remembering,Understanding, Applying, Analyzing, Evaluating, and Creating [2].Bloom’s Taxonomy has been recommended to be incorporated in both teaching and learning processes, as well as assessment practices [3].

In 2008, the era of Bloom’s taxonomy named the Bloom’s Digital Taxonomy (BDT) was introduced by Andrew Churches [4].The BDT consists of six key terms that involve six levels based on difficulty, from the easiest level to the hardest, starting from Remembering and followed by Understanding, Applying, Analyzing, Evaluating, and Creating.Digital taxonomy proposes learning objectives to be in a technology-enhanced teaching context and necessary instructions on how to practice a certain skill to realize its full potential.Each level of taxonomy is supported by an extensive list of thinking skills adapted to a technology-enhanced teaching and learning environment.These activities can make the learning process fun and interesting [5].The BDT has its own advantages in Bloom’s development.The main advantage of the BDT is that educators can get some ideas on how to use digital tools for learning and teaching.Reference [6] emphasized that educators need to be competent in terms of knowledge, skills, and attitudes in learning.One of the main competencies relates to pedagogic competencies.In this work, the BDT is one of the pedagogic components that need to be acquired.However, there is a question of whether teachers use all of BDT activities in their teaching and learning.These findings are crucial for providing information to related parties about current shortcomings in teaching that can be improved to enhance both teaching skills and teachers’confidence in applying digital-based activities in class.

2 Related Work

In recent years, there has been an increasing amount of literature on BDT, which has been published.Some studies have been interested in examining the BDT application to the teaching process.The related studies have shown that the role of teachers in BDT is of great significance when it is used in the classroom.In fact, previous studies have tended to criticize the original Bloom’s taxonomy because of its changes toward the digital.

In [7], the knowledge and application of digital verbs and tools by both teachers and students were investigated in order to understand the environment of virtual and conventional learning conceptually.This study has explained that students attending online (distance) learning are better in the application of digital tools, and they can understand them well in addition to their involvement in higher-order thinking tasks, such as publishing and podcasting.Meanwhile,in [1], Benjamin Bloom’s psychological and pedagogical model was developed and modified for the systems of adult learning.Based on the analysis, the techniques stemmed from Bloom’s taxonomy, which was modified for adult training, enabled the development of students’skills and abilities in analyzing problems thoroughly and comprehensively as well as producing effective and creative solutions.

In [8], the authors described the teaching experience involving a course that introduced educational technologies to teachers in Macau, which was designed based on connectivism, which represented learning theory in the digital era that highlighted the interaction and engagement with digital media and sharing of digital artifacts.The learning outcomes constructively coincided with the activities and assessments of learning and teaching relative to the students’learning needs and disparity in competencies and technological skills, contributing to the discussion regarding how the teachers could learn to teach using digital technologies.In a study conducted in Bulgaria [9],the authors discussed the changes in modern pupils’characteristics, regarding them as a “digital generation,”particularly in the area of computer sciences.The results [8,9] showed that introduced changes directly affected the learning-objective taxonomy from classic to revised and ultimately to digital.The dynamics of the learning-objective taxonomy were further explored to clarify the concepts of e-learning, blended learning, electronic learning object, and m-learning based on the digital generation’s characteristics.This is vital in order to evolve the learning-objective taxonomy, elucidate the interactivity levels achieved upon the development of e-learning objects to be implemented in the blended learning.It can provide digital instruments as well as authoring tools required by students in the creation of electronic learning objects involving the high cognitive levels of Bloom’s digital taxonomy, i.e., evaluation and creation.Generally, the mentioned studies have outlined the teachers’critical role in the implementation of BDT in the teaching process.

In [10], the authors examined the application of Bloom’s taxonomy to describe a psychotherapeutic game relative to cognitive processing and knowledge level.The RBT was introduced and applied to five psychotherapeutic games:Personal Investigator, Treasure Hunt, Ricky and the Spider, Moodbot, and SuperBetter.Based on the results, the revised Bloom’s Taxonomy was not suitable for comparing the game content.Also, RBT should not be applied to objectively classify psychotherapeutic game content since the results yielded a very low intercoder reliability value.

The adaptation of the revised taxonomy to a new generation of students and a general summary of how Bloom’s Original Taxonomy could evolve to Bloom’s Revised Taxonomy and initiate Bloom’s Digital Taxonomy were presented in [11].As concluded by the authors, the current restrictions on technology usage in the classroom limited establishment of an association between the classroom and in real life.In [12], it was highlighted that the latest ICT technologies could enhance teaching and learning.The SAMR model and Bloom’s taxonomy of educational objectives were implemented at the secondary grammar school for General English and the higher institution for the English for Specific Purposes subject.The two abovementioned studies analyzed the adaptation of Bloom’s Original Taxonomy to Bloom’s Digital Taxonomy.In [13], thirty compilations posted on websites were analyzed and evaluated on the extent to which these verbs were in alignment with Bloom’s taxonomy categories.As explained by the author, Bloom’s taxonomy value was heuristic for writing student learning outcomes, and these learning outcomes should be considered by other faculties to describe the expertise level of students who had obtained an associate’s, bachelor’s, or graduate degree.Reference [14] analyzed the original and revised taxonomies and presented the major criticisms on Bloom’s original taxonomy, as well as several criticisms on the revised taxonomy.

Recent taxonomies of objectives and learning-objective strategies can be categorized in terms of the content types (e.g., facts, principles, procedures, concepts, and processes) and performance level (e.g., using and remembering).In [15], a pilot study on the BDT application in an online art project aimed at identifying challenges and affordances in helping amateur artists build their art portfolio through social media sites and other Internet resources was conducted.Due to the high demand of communication technology and computers in art education, the learning possibilities in the online environment have been required to be extended.

In [16], it has been explained that Bloom’s taxonomy can be used by information professionals who train or instruct others how to write learning objectives that describe skills and abilities that learners should master and demonstrate.Since Bloom’s taxonomy distinguishes levels of cognitive skill, the Bloom’s practice requires learning objectives with a high cognitive skill level and leads to in-depth learning as well as the knowledge and skill transfer to various contexts and tasks.In [17],the responses given by a total of 1,245 science students and 47 science teachers from 14 Catholic high schools in Sydney, Australia, were analyzed.The students and teachers analyzed types of activities using laptops as self-reported, and the BDT was used to differentiate the activities from lower to a higher order.Although the use of pen and paper gradually shifted to using laptops,the students’modal practice entailed the lower-order paradigm of note-taking, as well as working from textbooks electronically using Word processing and electronic textbooks in addition to online searching.In addition, it was observed that students had benefited from higher-order activities,such as blogging and video editing, while teachers were not inclined to engage in these activities.

In Malaysia, only a few studies have discussed Bloom’s taxonomy from the digital-based aspect.Previous studies have reported that Bloom’s taxonomy has the potential to be applied to different fields, including vocational taxonomic proposals [18], discussion on Bloom improvement in Islamic perspectives [19], promoting creative and critical thinking through English syllabus with augmented taxonomy [20], and understanding more about children’s skills in the process of designing digital storytelling games using a tablet [21].The mentioned studies have highlighted the need for Bloom’s taxonomy and BDT to be used in a wider context.However, to the best of the author’s knowledge, there have been very limited studies on assessing the psychometric characteristics of BDT measurement items.Quality items can lead to a better measurement of BDT in the local context based on BDT levels.In [18,19], Bloom’s Taxonomy was discussed and criticized from the Malaysian context, laying a foundation for further studies on BDT.

In [18], the weaknesses of Bloom’s taxonomy in classifying vocational domains were discussed,and a new taxonomy was suggested.Using Delphi techniques, six major domains of vocational taxonomy have identified and verified, namely, knowledge, gross motor skills, fine motor skills,visualization, problem-solving, and inventive skills.Meanwhile, in [19], Western criticism and Islamic views on Bloom’s taxonomy were discussed, and it was found that there had been criticisms and improvements from the past studies focusing on Bloom’s taxonomy to a new taxonomy regarding four topics, namely, hierarchical arrangements, structural classification, uses,and needs.The new taxonomy refers to the 21st century learning, especially to the field of Islamic measurement context in Malaysia.In contrast, in [20], the relevance of Bloom’s taxonomy that included digital elements was discussed from the aspect of augmented reality.A literature review was conducted to examine the extent to which Bloom’s Taxonomy of Educational Objectives could be relevant for teaching creative and critical thinking among Malaysian students, identifying the missing aspects in Bloom’s taxonomy in the indigenous context as well as highlighting the importance of promoting creative and critical thinking among Malaysian students while reporting the issues surrounding English Literature to be taught as a subject.Finally, the English syllabus,in addition to augmented taxonomy, was suggested based on the outcomes of holistic learning comprising three sets of ability-rationale thinking, purposeful thinking, and context effective relation.However, in this work, one of the objectives is to overcome the critiques by assessing the quality of items for the BDT using the screening list of the existing items without modification,particularly using the new modern measurement theory.

As presented in this section, there have been many studies discussing BDT application to the teaching activities, and some of these studies considered the BDT usage in Malaysia.However, in order to measure the BDT, a very high quality of the measurement items is required.The related literature has emphasized the lack of psychometric assessment of items to measure BDT.This limitation can be overcome using modern assessment theory, such as the Rasch model, to ensure the reliability and validity of the measurement.Hence, the BDT practice application in the classrooms by teachers should be further studied.These findings could provide a useful reference in identifying the teachers’ability to use each level of BDT maximally.In view of all mentioned,one may suggest an approach to examine the psychometric characteristics of BDT measurement items in more detail besides creating a new bloom.Hence, BDT discussion with efforts to provide empirical evidence for new psychometrics items or constructs will help researchers strengthen their future studies more meaningfully.

2.1 Conceptualization and Operationalization

Conceptualization and operationalization used in this study entail BDT based on the definition presented by [4] as shown in Tabs.2 to 7.The BDT is based on the six key terms, involving six levels based on difficulty, starting from the easiest level that is Remembering, followed by Understanding, Applying, Analyzing, Evaluating, and finally Creating.The BDT definitions are presented in Tab.1.However, there is a limitation of the use of digital verbs in academia.First,digital tools are grouped based on their appropriate level, which is sometimes difficult to conduct because as they may be used for multiple purposes.In this context, this study examines the extent to which teachers use BDT key terms in the teaching and learning process.

Table 1:Bloom’s digital taxonomy definitions

Table 2:Indicators for remembering based on Bloom’s digital taxonomy

Table 3:Indicators for understanding based on Bloom’s digital taxonomy

2.2 Research Motivation and Aims

In recent times, COVID-19 has been a major public health problem worldwide, including Malaysia, and it has been recording a large number of new cases exceeding a thousand cases daily.The break out of COVID-19 has affected many life aspects, including education.The recent increase in the number of COVID-positive cases has highlighted the need for transforming the teaching and education process from face-to-face methods to online education.The primary concern of this transformation is how to conduct lectures online since this is compulsory for all institutions in order to avoid the risk of further spreading of the COVID-19 virus.The need for online teaching and learning is high since all schools and learning institutions have been closed.However, there are many problems related to online education, such as problems of unstable Internet access, low student focus, incomplete equipment, and many others.Online learning also makes it difficult for some teachers to assess and test students’achievements and knowledge.Namely, for cognitive assessment, teachers need to be adept at applying BDT since the learning process is conducted online.Thus, teachers should master and use BDT well in their teaching process.To measure the extent to which the BDT aspects are used among teachers, the evaluation of psychometric characteristics on the measurement items is necessary, and it is very important to ensure that the measurements are accurate, especially those involving the use of modern measurement theories.

Table 4:Indicators for applying based on Bloom’s digital taxonomy

Hence, this study aims to test whether BDT measurement items have good psychometric characteristics based on the teacher’s self-assessment using the Rasch model analysis, which represents modern measurement theory.The modification of assessment by Rasch model will be able to ensure that BDT items difficulty are match with the individual abilities.This study also examines the pattern of the BDT application in teaching from the teachers’perspective.

Table 5:Indicators for analyzing based on Bloom’s digital taxonomy

Table 6:Indicators for evaluating based on Bloom’s digital taxonomy

Table 7:Indicators for creating based on Bloom’s digital taxonomy

3 Methodology

3.1 Research Design and Sampling

This study adopts a quantitative approach using an online survey research design.The quantitative approach is used because it is suitable for a large number of respondents [22];this study includes a total of 774 respondents.This approach also helps obtaining more credible findings because it is efficient [22].Besides, an online survey is very suitable for this study due to the limitations imposed by the COVID-19 pandemic, which has caused difficulty in obtaining research findings in a face-to-face manner.Hence, an online instrument via Google Forms was used.Further, as stated in [23], online surveys have a few advantages such as fast delivery,easy to administer, and inexpensive.Besides, respondents can also answer at their convenience,similar to the mail questionnaire.A stratified sampling technique was conducted on 774 teachers involving five geographical zones in Malaysia, such as North, West, East, South, and Borneo(stratum).Convenient sampling was also used due to the nature of samples that separated the target population into different strata groups.The advantages of such sampling are that it ensures representativeness of samples and estimates the target population with less error and higher precision [22].Initially, the 200 instruments were given per zone with total of 1000 instruments.However, only 774 were successfully obtained, achieving a return rate of 77.4%.This return rate exceeded the targeted return rate for online collection, which is 60% [24].

3.2 Instrumentation

The measurement originally included 30 measurement items that involved 6 levels of BDT,including 6 items for Remembering (A1 to AF), 5 items for Understanding (2A to 2E), 5 items for Applying (3A to 3E), 4 items for Analyzing (4A to 4D), 6 items for Evaluating (5A to 5F), and 4 items for Creating (6A to 6D).These items were adapted according to the BDT definitions [4].All of these items were undergone facial validity assessment and content validity by three experts who conducted the Content Validity Ratio (CVR) analysis, as suggested in [25].The experts were professionals and practitioners, as suggested by [26].The experts fully agreed to verity item testing and screening, which involved the structure of sentences and words only.

3.3 Data Analysis and Procedure

The data of this study entailed a dichotomous scale that was used to elicit a Yes or No answer [23].The respondents were required to choose either Agree or Disagree for each item,which indicated the level of agreement, as recommended by [27] as a measurement options.The research included 30 items that were used to identify whether teachers had performed all the key terms for the six BDT levels in their teaching and learning processes.The data were processed using the Rasch model analysis to provide information on:(a) item fit and unidimensionality,(b) Wright map and a bubble chart, (c) mean measure for each BDT level, and (d) reliability and separation index.The WINSTEPS 3.71 was used to perform the Rasch analysis.

The Rasch model assumes that each item comprises only a difficulty parameter and that all items have the same discriminatory index.This ensured that low-capable students could not obtain the correct answer to the items that they did not know by guessing [28].In short, the probability of success depends on a difference between an individual’s ability and the difficulty level of an item.The Rasch model [29] adopts an algorithm that expresses the expectation of the probability of an item asiand the individual’s ability asnin the form of a mathematical equation as follows:

4 Results and Discussion

The results considered several key parameters:(a) item fit and unidimensionality, (b) Wright map and a bubble chart, (c) mean measure by each level of BDT, and (d) reliability and separation index.The obtained results not only showed the quality characteristics of the psychometric items but also indicated the pattern of BDT usage in teaching and supervision from the teachers’perspective.As explained in Section 2.3, there were 30 items assigned with 6 levels of BDT.

4.1 Item Fit

As shown in Tab.8, 27 out of 30 measurement items fulfilled the fit characteristics in the Rasch model.In Tab.8, Infit refers to inlier-pattern-sensitive fit statistic and the Outfit refers to outlier-sensitive fit statistic [30].

Table 8:Fit statistics of measurement items

The highest measure value was that of item 6A (1.61 logits) and the lowest values corresponded to 5C and IB (?1.44 logits).Overall, activity 6A denoted “programming,” and 5C denoted “moderating.”The standard value of errors was in a range of 0.09–0.10 and complied with the recommended value [31].Meanwhile, the maximum value of MNSQinfitwas 1.13, and the minimum value of MNSQinfitwas 0.82.In addition, the maximum value of MNSQoutfitwas 1.21,whereas the minimum value of MNSQoutfitwas 0.79.Additionally, PTMEA Corr.had a maximum value of 0.64 and a minimum value of 0.46.The range of MNSQinfitwas 0.31, and that of the MNSQoutfitrangewas 0.42, while the range of PTMEA Corr.was 0.18.

Based on the MNSQ fit settings, the used range was from 0.77 to 1.30 [31].The values that exceeded 1.30 were considered as misfitting, and those less than 0.70 were regarded as overfitting [29].A total of three measurement items were dropped due to the non-fulfillment of the fit range value, which were:IF (MNSQoutfit= 1.51), 5B (MNSQoutfit= 1.41), and 3E(MNSQoutfit=1.35).Item IF denoted “searching” or “Googling” by which the students were able to simply enter a phrase or a keyword into the basic entry field of search engines; 5B referred to“posting” by which the students were able to comment on discussion boards, blogs, and threaded discussions; 3E represented “editing” by which the students were able to make editing in most media (procedure or process employed by the editor).The expected score ICC pattern and some unsuitable response patterns (misfits) of items IF, 5B, and 3E are presented by the dotted circle lines in Figs.1–3, respectively.However, these items were removed because they did not meet the fit requirements.

Figure 1:Expected score ICC of 1F

Figure 2:Expected score ICC of 5B

Figure 3:Expected score ICC of 3E

4.2 Items Unidimensionality

The unidimensionality of items indicated that items did not have equality characteristics in the matter to be measured.Dimensionality can be defined as determining an instrument in one direction and one dimension or the force given to one dimension or attribute at once [29] to ensure the instrument’s content and construct validity [32].The raw variance value explained by measures was recorded at 36.3% for overall and each level, which was above the specified value of 20% [33].The eigenvalue of the entire BDT was 3.5These values complied with the specified value of less than five [34].Meanwhile, the overall noise value was recorded at 12.9.These noise values for each L1 to L6 were below 15% [35].The noise for each construct indicated an underachieved value such as L1 (23.3%), L2 (21.8%), L3 (28.6%), L4 (26.4%), L5 (25.4%), and L6 (29.7%).

4.3 Wright Map and Bubble Chart

Wright Map or item-person map in this analysis denotes a figure that represents items by the item number and the performance of each person to effectively observe the ability of the measured scale items to match the respondents.The distribution of the measurement items according to BDT levels from the aspect of their usage by teachers is presented in Fig.4.

Figure 4:Wright item-person map

A total of 55.5% of the total items were above the average difficulty value, while 45% of the total items were below the average difficulty value.This distribution proved that the respondents found it difficult to perform item 6A (Programming), by which the students were able to create programs suitable to their needs and goals (applications, macros, multimedia applications, or games in systematic environments).Meanwhile, the most easily performed activities by the teachers were item IB (Highlighting), by which the students were encouraged to select and highlight phrases and keywords as a recalling technique, and item 5C (Moderating), by which the students were able to assess comments or postings from various viewpoints in terms of their value, worth, and suitability.The results indicated that itemmaxwas +1.61, and itemminwas ?1.44.Meanwhile, in the item–person relationship, personmaxwas 3.59, and personminwas ?4.89.The range values for item and person were 3.05 and 8.48, respectively.The value of μitemwas zero, while the value of μpersonwas 0.05.The mean of individual abilities was slightly higher than the mean of item difficulty, which suggested that BDT measurement items, overall, were easy to perform for the respondents and that, on average, the teachers’performance was higher than the difficulty level of BDT items.

The bubble chart that graphically illustrates the measurement value and item compatibility [36] is presented in Fig.5.The bubble shape between the overfit and underfit was classified as accepted, which was within thet-value range of ±2.00.The bubble chart also shows the bubble positions for all 27 items after the screening was conducted.This screening involved MNSQ because, if MNSQ was considered,Zstdcould be ignored [34].The expectation was that difficult items would be answered by more able persons, and easy items would be answered by all.A total of seven items were in the erratic or unpredictable area of two items withZstdvalue of more than 2.00, which were 4A (Zstdinfit=3.0,Zstdoutfit=2.0) and 4C (Zstdinfit=3.2,Zstdoutfit=3.0).Meanwhile,Zstdvalue of less than 2.00 had five items:1C (Zstdinfit=?4.6,Zstdoutfit=?2.9),2D (Zstdinfit=?2.6,Zstdoutfit=?2.0), 2E (Zstdinfit=?4.6,Zstdoutfit=?2.9), 3C (Zstdinfit=?3.7,Zstdoutfit=?2.7), and 5E (Zstdinfit=?2.6,Zstdoutfit=?2.0).In this study, erratic or unpredictable referred to items that hadZstdvalue within theZacceptance range of ±2.0, and they were regarded as a misfit.

Figure 5:Bubble chart of item fitness

The L1 analysis (Remembering) showed that item IE was the hardest item with 1.20 logits.This result showed that Social Bookmarking was the least applied activity by teachers in the teaching and learning process at the L1 level.Meanwhile, activity 1B (Highlighting) included the most performed items by teachers with ?1.44 logits.The L2 analysis (Understanding) showed that item 2C was the hardest item with 0.96 logits.This result indicated that Categorizing and Tagging was the least applied activity by teachers in the teaching and learning process.In contrast,activity 2D (Commenting and Annotating) included the most performed items by teachers with 0.23 logits.The L3 analysis (Applying) showed that item 3A was the hardest item with 0.52 logits,indicating that Running and Operating was the least applied activity by teachers in the teaching and learning process.Meanwhile, activity 3B (Playing) included the most performed items by teachers with ?0.95 logits.The L4 analysis (Analyzing) showed that item 4D was the hardest item with ?0.16 logits; thus, Cracking was the least applied activity by teachers in the teaching and learning process at the L1 level.However, activity 4B (Linking) included the most performed items by teachers with ?1.08 logits.The L5 analysis (Evaluating) showed that item 5D was the hardest item with 1.05 logits; Collaborating and Networking was the least applied activity by teachers in the teaching and learning process.Meanwhile, activity 5C (Moderating) included the most performed items by teachers with ?1.44 logits.The L6 analysis (Creating) showed that item 6A was the hardest item with 1.61 logits, and Programming was the least applied activity by teachers in the teaching and learning process.Last, activity 6D (Publishing) included the most performed items by teachers with ?1.44 logits 0.62 logits.

4.4 Mean Measure of Each BDT Level

Based on the mean logit value results, the hardest level of BDT to be conducted by teachers in their teaching and learning was Level 6, which was Creating (+1.08), followed by Level 2(+0.53), Level 3 (?0.24), Level 1 (?0.33), and Level 5 (?0.48), while the easiest level was Level 4,i.e., Analyzing.Based on the level of BDT, Level 6 (Creating) is the highest level of BDT.This level represents the hardest level from the aspect of activity implementation.Thus, the results presented in this study are logical because the logit value of 1.08 has indicated level L6 as the most difficult activity for teachers to implement.However, interestingly, the results showed that the easiest level to be implemented by the teachers was not L1 (remembering) as it was expected but L4 (analyzing).According to [4], L4 can be defined as separating concepts or materials into parts to determine the relation or interrelation between the parts relative to their overall purpose or structure.Level L4 also includes mental actions, which comprise the ability to differentiate,organize, attribute, and distinguish between components.

This result could be caused by the elements of Higher Order Thinking Skill (HOTS) instilled by teachers to the students.The Malaysian Education Development Plan 2013–2025 explains that national examinations and school-based assessments (PBS) have been revamped to gradually increase the percentage of questions that define high-level thinking skills.By 2016, high-level thinking questions included at least 40% of the questions inUjian Penilaian Sekolah Rendah(UPSR), and at least 50% of the questions inSijil Pelajaran Malaysia(SPM).More group-based projects and assignments were also done to improve students’high-level thinking skills and their ability to work individually and in groups.They were given more community-based projects and cross-school activities to foster interaction between individuals from all backgrounds.In addition,Wave 1 (2013–2015) played an important role in changing the education system by supporting teachers focusing on key skills and redesigning exam questions to put a higher focus on highlevel thinking skills questions [37].The high-level thinking should be starting at Analyzing level;therefore, the findings indirectly had proven that the implementation of the BDT activities by teachers were at a higher level.

4.5 Reliability and Separation Index

The rating scale instrument quality criteria used in this study are based on the setting in [31].Person reliability entails the consistency of person ordering to be accepted under conditions that the equivalent set of items that measures the same construct is given to this respondents [38].The overall person measurement reliability value was 0.87.For all levels of persons ability, which could be considered as good.While the item measurement reliability was 0.99, which could be regarded as excellent; the person separation index was 2.60, which could be considered as satisfactory.Item separation denotes the ability of all participants to answer all the items’difficulty levels.This means that the respondents can be distinguished by the constructs being tested [38].The item separation index was 9.74, which could be considered as excellent.This means that the quality of BDT measurement items in this instrument is excellent, but the consistency of answers from teachers is only fair.The grouping of persons and items can be obtained using the following formula:H=[(4?Separation)+1]/3, whereHrepresents the separation value, which can be taken from reliability and separation index produced by the Winsteps software.

The overall separation value of persons wasHperson=(4 ?2.6+1)/3=3.8, which could be rounded to 4.This means that there were four groups of teachers according to the ability levels.These findings hold for all BDT levels, from L1 to L6.Meanwhile, the overall separation value of items wasHitem=(4 ?9.74+1)/3=13.32, which could be rounded to 13.This means that there were 13 clusters of items according to the difficulty levels.TheHvalues of different levels were as follows:Hitem(L1)=(4?11.15+1)/3=15.2;Hitem(L2)=(4?2.63+1)/3=3.84;Hitem(L3)=(4?6.30+1)/3=8.73;Hitem(L4)=(4?3.83+1)/3=5.44;Hitem(L5)=(4?10.45+1)/3=14.27; andHitem(L6)=(4 ?3.60+1)/3=5.13.These results indicated that the items of level L1 could be grouped into 15 difficulty levels, items of level L2 could be grouped into four difficulty levels, items of level L3 could be grouped into nine difficulty levels; items of level L4 could be grouped into five difficulty levels; items of level L5 could be grouped into 14 difficulty levels, and finally, items of level L6 could be grouped into five difficulty levels.In comparison to [29], the separation values with more than two levels are sufficient.In summary, these results help to improve the item quality provided to teachers during self-assessment.The findings of this study will directly benefit teachers with lacking of BDT in regaining the digital element for teaching.The Rasch psychometrics evidence may be help the researcher to measure BDT accurately.This will enable teachers that have struggle to implement the BDT to be more dynamics and creative in teaching pedagogically.

5 Conclusion and Future Works

The current study aims to improve the measurement for BDT items through teachers’selfassessment in teaching and learning, and the Rasch measurement model is proposed for the assessment of psychometric properties.The results show that a total of 27 measurement items can be used as an alternative for BDT measurement using the Rasch model.The results show that the Rasch model can more clearly demonstrate various item properties compared to the classical test theory.Moreover, this study indirectly shows to which extent teachers tend to apply each level of BDT in their teaching and learning practice and examines which BDT activities are the hardest and easiest to apply.However, certain limitations need to be considered in future works.First, the results presented in this study are applicable only to the Malaysian population,so the study should be expanded regarding both contexts and countries.Namely, it would be interesting to explore and compare more characteristics of item response for various levels of respondents’ability through systematic comparisons.Second, this investigation has been limited to the teachers’perspective, so future research is highly encouraged to introduce scale analysis to develop specific questionnaires from the perspective of students’understanding of the BDT levels.In fact, this measurement construct can be tested for its validity using multivariate analysis,such as factor analysis or principal component analysis, to provide empirical evidence for future reference.Third, this study is limited to general definitions in each level, starting from L1 to L6,so further investigations can be performed to each specific activity for every level of BDT.The information can be useful for customizing digital teaching activities that suit both teacher and students’abilities relative to the implementation of teaching and learning in class.

Acknowledgement:The author would like to express the utmost gratitude to the anonymous reviewers for their insightful comments and suggestions, which helped to improve the content,quality, and presentation of this research paper.The author would also like to honor and give high appreciation to the Faculty of Education, University Kebangsaan Malaysia (UKM).

Funding Statement:This study was funded by the Ministry of Higher Education (MOHE),Malaysia under the Fundamental Research Grant Scheme (FRGS) (FRGS/1/2018/SSI09/UKM/02/1), and the Faculty of Education, University Kebangsaan Malaysia (UKM), Geran Galakan Penyelidik Muda (GGPM-2017-088).

Conficts of Interest:The author declares that he has no conflicts of interest to report regarding the present study.