Effectiveness of the fl ipped classroom on the development of self-directed learning in nursing education: a meta-analysis

Y-Qin Liu, Yu-Feng Li, Meng-Jie Lei, Peng-Xi Liu, Julie Theold, Li-N Meng,Ting-Ting Liu, Chun-Mei Zhng, Chng-De Jin,*

aDepartment of Graduate, Tianjin University of Traditional Chinese Medicine, Tianjin 301617, China

b Department of Nursing, Tianjin University of Traditional Chinese Medicine, Tianjin 301617, China

Abstract: Objectives: To examine the best practice evidence of the effectiveness of the fl ipped classroom (FC) as a burgeoning teaching model on the development of self-directed learning in nursing education.

Keywords: fl ipped classroom · blended learning · reverse teaching · self-directed learning · self-learning ability · self-management · nursing education · meta-analysis

1. Introduction

In a complex health care environment where social,technological, and medical changes present them with huge challenges at a rapid pace, nurses are expected to strengthen their self-directed learning ability to ensure that they can adapt and respond to these challenges.1Self-directed learning is related to meta-cognition, motivation, and behavior with a positive attitude.2It includes self-management, self-control, and desire for learning.3Self-directed learning can be beneficial in providing nurses with the skills to seek, analyze, and apply theory to practice effectively.4In consideration of its numerous benefits, self-directed learning is highly emphasized in nursing education and clinical circumstance and also deemed as an indispensable skill for learning and working in the 21st century.5Thus, self-directed learning is critical to enhance the clinical capacity of clinical nurses.

Researchers had confirmed that the fl ipped classroom (FC) had a positive effectiveness on self-directed learning and discussion skills.6However, no systematic review has yet demonstrated the evidence for using the FC method to improve nursing students' self-directed learning. Many findings suggest that FCs could be used as an effective teaching method in promoting selfdirected learning.7-9In the study of Gagnon et al, blending learning with internet-based tutorials compared with traditional teaching methods introduced nursing research to nursing undergraduates.7The results indicated that the blended-teaching method could much fit for nursing undergraduates, depending on their level of motivation and degree of readiness to engage in selfdirected learning. A Chinese study9compared the “FC”teaching model design with traditional teaching models in the application of basic nursing scenarios through a non-random concurrent controlled trial. The result suggested that the “FC” teaching model could help nursing undergraduates promote self-learning ability. However,apparent inconsistency still remains. An experimental and comparative study conducted by Jahromi et al.showed insignificant differences between intervention and control groups.8Tian also showed no significant increase in self-learning ability in the intervention group,but after controlling for age, the adjusted mean scores of the FC group were higher than those of the traditional group, the lecture group.9

During the nursing education process, the impact of the FC model on self-directed learning should be determined. The purpose of this meta-analysis was to systematically review and statistically summarize evidence of the effect of the FC teaching method on selfdirected learning and to provide clearer insights into the application of the FC teaching method in nursing education. Thus, this meta-analysis answers the following questions: what are the primary resources and research development of studies are in which the FC teaching method was used to improve self-directed learning in nursing education; how the quality of the included studies is; what is the effect of the FC teaching method is on development of self-directed learning in nursing education, and whether this evidence could be applied to guide nursing education.

Self-directed learning (SDL) is a teaching method that can be defined in terms of the quantity of responsibility accepted by learner's own learning and performance.10In the views of nursing educators, self-directed learning has been defined in different implications,11-13while the most common definition presented by Knowles is that self-directed learning is a process in which learners take action, with or without the assistance of others, in judging their learning requirements, discovering learning goals, identifying available learning resources to learn, choosing appropriate learning strategies and evaluating learning outcomes.14Overall, self-directed learning in nursing has been identified with as having several stages, including assessing (readiness for self-direction, learning needs, material and human resources, and the learning environment), planning(interpreting self-directed learning), implementing, and evaluating.15

Many reviews of the literature revealed that a number of tools were used to measure self-directed learning ability, such as the Self-Directed Learning Readiness Scale(SDLRS),16Self-Directed Learning Readiness Scale for Nursing Education (SDLRSNE),10Self-Regulated Learning scale (SRL),17Autonomous Learning Competencies scale (ALC),18and the Competencies of Autonomous Learning of Nursing Students (CALNS).19The SDLRS is a scale that was used to assess the degree to which people considered themselves as having the skills and attitude essential for self-directed learning.Many studies report that it has good reliability and validity,8,16,20but several others cast on the cost and criticize its intent.16,21,22This scale is a 41-item questionnaire in five parts. It is based upon a Likert scale and consists of three sections: self-management, learning engagement,and self-control. The SDLRSNE was developed as an alternative to Gugliemino's (1997) SDLRS. The internal consistency of SDLRSNE and its subscales has been reported in several studies.7,23,24This scale consists of three components: self-management (12 items; Cronbach's α = 0.8), self-control (15 items; Cronbach's α =0.74), and desire for learning (13 items; Cronbach's α= 0.8). Each item was assessed using a 5-point Likert scale, ranging from strongly disagree (1 point) to strongly agree (5 points).23Furthermore, SRL, as developed by Zhang and Li,17is a 30-item scale with four subscales:learning motivation (eight items), self-learning ability(five items), self-management skills (eleven items), and information literacy (six items). The scale demonstrates favorable inter-item consistency and split-half reliability.17Meanwhile, the internal consistency reliability of the ALC18and the CALNS19using the Delphi Report's consensus definition of self-directed learning ability was satisfactory.

As is well-known, traditional teaching models that provide a teacher-centered approach mean that students are not actively engaged in processing information, developing understanding, or applying knowledge into practice.6However, the FC model, which provides a studentcentered approach, presents nursing students with the opportunity to enhance their self-directed learning, analytical skills, problem-solving abilities, and lifelong learning.25Moreover, this model integrates cooperation, team learning and active learning, which are crucial strategies for students to provide the essential skills and abilities to work in a cross-disciplinary practice environment.26,27Through the FC process, nursing students receive the same content as traditional classroom in the course setting, but the instructors' techniques are different as black and white: face to face lectures; homework before class;using case studies, laboratories, and games; discussing in groups, and simulation-based learning.28Thus, nursing students were responsible for their own education and strengthened their autonomy of their studies.

The sudden increase in FCs conforms the studentcentered requirements of higher education.29Although the FC model was taken up in other disciplines, there is a lack of evidence about the effect in the nursing curricula. Therefore, this meta-analysis review aims to fill this gap in examining the best evidence of the effectiveness of the FC as a burgeoning teaching model on the development of self-directed learning in nursing education.

2. Methods

2.1. Search strategy

A systematic search was conducted of literature in electronic databases without geographical restriction.The primary sources included PubMed, Embase, Web of Science, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Cochrane Central Register of Controlled Trials (CENTRAL), Wanfang Data,China National Knowledge Infrastructure (CNKI), and Chinese Science and Technology Periodical Database(VIP). The search was restricted from inception to June 2017 with the papers published in English and Chinese.The search terms were finally identified according to the relevant literature and reviews in Cochrane Library and Joanna Briggs Institute (JBI) Library, by researching and summarizing the titles, abstracts, and keywords. The MeSH terms and keywords of searching were used separately or in assembly. The search strategy for PubMed is shown in Table 1.

All results were downloaded as duplicates into Endnote X7 for checking and removing into different lines in accordance with the titles, abstracts, and keywords.The potential papers in the references were checked to identify appropriate studies satisfying the inclusion criteria that may have been excluded by the search strategy. Additionally, full texts of any indefinite papers were downloaded for review.

2.2. Inclusion and exclusion criteria

Two reviewers screened the titles and abstracts of all articles in the initial broad searching. Articles were selected for inclusion if: (1) a randomized controlled trial (RCT) or non-RCT comparative study; (2) subjects included higher vocational students, undergraduates,and postgraduates; (3) they used fl ipped teaching as an intervention; (4) include the outcome indicators for self-directed learning; and (5) reported the sample size of the subjects, the mean difference with 95% CI of selfdirected learning scores. Articles were excluded if: (1)the same group of subjects also received other teaching methods; (2) without complete data; and (3) duplicate articles.

2.3. Data extraction

Two of the authors extracted and summarized independently the data from the included studies. Any disagreement was resolved by the adjudicative senior authors.The data were extracted in a pre-designed coding manual: the first authors, publication years, countries,study designs, subjects, sample size (intervention group and control group), curriculum, teaching model inintervention and control groups, duration of intervention,outcome, and outcome measures.

Table 1. Search strategies.

2.4. Quality appraisal

The methodological quality of the studies was assessed as three levels including low bias, uncertain, and high bias by the authority of the Cochrane Handbook for Systematic Reviews of Intervention.30The list of sevenitems checked on the Cochrane Handbook is as follows:adequacy of the generation of the allocation sequence,concealment of allocation, blinding participants and personnel, blinding outcome assessment, incomplete outcome data, selective outcome reporting, and other sources of bias. The assessment was independently conducted by two reviewers, and all disparities between them were resolved by consensus.

2.5. Statistical analysis

A meta-analysis of RCTs and quasi-experimental studies using Review Manager (RevMan) 5.3 software was conducted to explore differences between the intervention group and control group. The main strategy was to abstract and analyze the self-directing learning scores from different trials. Since the continuous outcomes which were expressed as a mean value and standard deviation measured by the scales, the standardized mean difference (SMD) of continuous outcomes was calculated from different scales and weighted mean differences (WMDs) was analyzed from the same scales.Chi-squared test andI2value were applied to estimate the inf l uence of heterogeneity on the results of this metaanalysis. According to the Cochrane review guidelines,the random-effects models would be selected if severe heterogeneity was shown atI2＞50%; otherwise, the fixed-effects models would be chosen. Moreover, sensitivity analysis was performed by deleting each study individually to appraise the consistency and quality of the results. Subgroup analysis was performed according to the scale dimension.

3. Results

3.1. The search results

The fl ow diagram about the process of including and excluding studies is shown in Fig. 1. The search yielded 724 relevant articles, and 462 duplicated records were excluded. A further 413 studies were excluded after screening by reading the titles and abstracts. The full texts of 49 studies were assessed, while 29 studies were excluded due to following reasons: 23 articles contained outcome variables different from the interest of this review, two articles were reported in Korean language, and four articles were self-controlled studies. Of the 20 studies eligible for qualitative analysis, 12 were included in this meta-analysis

3.2. Description of included studies

A description of the included studies is listed in Table 2.Twelve articles were included in the meta-analysis, and their total sample size was 1440 persons (755 in the intervention group and 685 in the control group). Ten of the included studies were completed in China and written in Chinese.9,31-39One study was carried out in Canada7and one in Iran,8both were written in English.The subjects of the included studies were nursing students. Among them, the participants of six studies were nursing undergraduates and those remaining were higher vocational nursing students. The sample size of included individual studies ranged from 30 to 193 participants, and the pooled sample size was 1440 (intervention group = 755 and control group = 685). Four studies were performed in nursing practice courses, and the others were medical theoretical lessons. The duration of intervention varied greatly, ranging from two periods to two semesters.

The scales for measuring outcomes of the included studies were as follows: SRL,9,31,38-40ALC,32,34SDLRS,8SDLRSNE,7and some specially designed scales.33,36,41The specially designed scales created by Chinese authors were not universally applied due to regional restrictions.

3.3. The quality of included studies

Fig. 2 and Fig. 3 show the differences in the risk of bias among the studies. Each included study was evaluated as a bias of high risk, low risk or unclear by using the assessment standards mentioned earlier. Of 12 included studies, the quality rating of one was A and of the others was B. Of nine RCTs and three quasi-experimental studies, four reported the random sequence generation in details, while three articles32,36,37were concealed by drawing lots from a sealed box and one paper7was a computerized random number generator. The allocation sequence of three studies8,33,35was assigned by convenience sampling or the preference of the researchers, in which students were divided into experimental or control groups. Thus, ten studies did not report allocation concealment, which may cause selection bias. And for the remaining no details were mentioned. Considering the feature of the intervention, blinding of nursing students and teachers to the intervention is impracticable.However, ten studies reported the same teacher, course duration, and teaching schedule in experimental or control groups. This was not likely to be inf l uenced by lack of blinding procedures and awarded adequate for blinding. In addition, all studies described the results to prevent reporting bias. Beyond that, all studies reported that there were no significant difference between experimental and control groups about baseline data, which included age, sexuality, and academic records. At last, all included studies were free from “other bias” as defined in the Cochrane Handbook.

Figure 1. Flow diagram of included and excluded studies.

3.4. The result of meta-analysis

3.4.1. Overall self-directed learning scores

Fig. 4 shows the overall self-directed learning scores of different scales. Twelve studies in meta-analysis involving 1440 nursing students (intervention group =755, control group = 685) were provided data for pooling to show the effect of FC on self-directed learning.One7reported three times of outcome data on three different SDLR levels at baseline (low, medium, and high), and therefore, 14 studies of outcome data were reported in the meta-analysis. The heterogeneity test showed obviously significant heterogeneity between the included studies using different scales (I2= 95%)and indicated that the FC had a significant effect on self-directed learning ability (SMD = 1.18, 95% CI[0.71, 1.66],P＜ 0.00001). Considering that the duration of intervention for each included study was different, subgroup analysis was used to explore the source of heterogeneity source.

The results of the subgroup analysis conducted according to the duration of interventions were as follows: Jahromi et al.8and Tian et al.37showed no statistically significant difference in nursing students'overall self-directed learning scores between the intervention and control groups, while others showed statistically significant differences. Each subgroup analysis indicated a significant effect on the different

duration of intervention in two to four periods (SMD =0.87, 95% CI [0.20,1.55],P= 0.01), half semester (SMD= 3.20, 95% CI [2.86, 3.54],P＜ 0.00001), one semester(SMD = 0.68, 95% CI [0.45, 0.91],P＜ 0.00001), and two semesters (SMD = 0.40, 95% CI [0.23, 0.58],P＜0.00001).

Table 2. Characteristics of included studies.

Table 2. (Continued).

Figure 2. Risk of bias graph.

3.4.2. Self-directed learning measured by SRL

Five studies including 618 subjects, which were measured by the SRL scale, provided data for pooling to show the effect of the FC on self-directed learning ability. Heterogeneity analysis showed a significant heterogeneity among the included studies (I2= 65%) (Fig. 5).Thus, meta-analysis based on the random-effects model and subgroup analysis was conducted according to the four dimensions of the scale. The results showed a more significant effect in the intervention group than that in the control group on the domains of learning motivation (SMD = 2.05, 95% CI [0.96, 3.13],P= 0.0002),self-learning ability (SMD = 1.89, 95% CI [1.34, 2.45],P＜ 0.00001), self-management skills (SMD = 2.69, 95%CI [1.68, 3.70],P＜ 0.00001), and information literacy(SMD = 0.93, 95% CI [0.16, 1.70],P= 0.02).

Figure 3. Summary of risk of bias assessment.

3.4.3. Self-directed learning measured by other scales

Of the five studies included, three used different scales and one was measured twice by the ALC scale and once before and after once the intervention. However,the combined results had a significant heterogeneity,but no obvious heterogeneity source was found. Therefore, only descriptive analysis was conducted. Findings of FC effects in other scales are summarized in Table 3.

In the study of Deng32, self-directed learning ability overall score of ALC scale was assessed. The pooled effect sizes for it favored intervention group on selfmanagement ability (SMD = 0.79, 95% CI [0.43, 1.16],P＜ 0.00001), self-learning ability (SMD = 0.65, 95% CI[0.28, 1.01],P= 0.0005), and information literacy (SMD= 0.82, 95% CI [0.44, 1.20],P＜ 0.0001).

Inversely, Jahromi et al.8reported the subscale scores of the SDLRS. The pooled effect size for selfengagement (SMD = 0.01, 95% CI [-0.43, 0.46],P= 0.95) and self-management (SMD = 0.11, 95% CI[-0.34, 0.55],P= 0.64) favored the control group, and this indicated that the scores were higher in the control group than those in the experimental group. However,self-control (SMD = -0.79, 95% CI [-1.25, -0.33],P= 0.0009) showed a significant difference between the two groups.

Hui33reported the scores of three levels of the CALNS on self-directed learning ability. The pooled effect sizes favored the intervention group on self-learning ability (SMD = 2.19, 95% CI [1.08, 3.30],P= 0.0001) and information literacy (SMD = 3.33, 95% CI [1.66, 5.00],P＜ 0.0001), while the scores of self-management ability(SMD = 1.23, 95% CI [-0.28, 2.74],P= 0.11) showed no significant difference between the two groups.

4. Discussion

4.1. Finding summaries

In accordance with the result of searching eight databases, this is the first meta-analysis about the impact of the FC method on self-directed learning of nursing students. This systematic review and meta-analysis examined how the FC improved the self-directed learning of nursing graduates and provided evidence supporting the effectiveness of this emerging teaching model on self-directed learning in nursing education.

There are several problems concerning methodological quality of included studies in the assessment process. Most comparative studies were limited by the random allocation of the intervention and control groups descripted on the standard of Cochrane Collaboration for Systematic Reviews of Interventions.Many studies were lacking adequacy of the random sequence generation and allocation concealment,which caused greater selection bias. Owing to no description of whether nursing students in experimental and control groups used the same teaching resources, such as teaching platforms and teachers,curriculum content, and duration, some studies did not record adequate details about the blinding of participants and personnel. However, data collection by the teachers who were directly in teaching may be not feasible. Questionnaires were released together by the teachers at the end of the course and recovered after students have anonymously filled out.

Figure 4. Forest plot of overall self-directed learning scores after using FC compared with traditional lectures.

The FC teaching model intervention of the included studies was conducted over differing lengths of time,from two periods to two semesters. Thus, the results of subgroup analysis according to the duration of intervention showed that all students in the experimental group from two periods to two semesters had higher overall self-directed learning scores. Heterogeneity of the half semester and two semester subgroups was zero percent, which meant homogeneity and the result of the included studies were reliable. Considering the few studies measuring the outcomes for two to four periods of intervention, more researches are needed to confirm the impact of the duration of FC teaching on selfdirected learning development.

Among multifarious scales used to measure the effect of the FC teaching method on self-directed learning in this review, the SRL scale was the most frequently applied. Subgroup analysis showed that students in the FC had more active self-directed learning performance and motivation, self-learning ability, self-management skills, and information literacy compared with those in the traditional teaching. The FC teaching model was beneficial to learning motivation, because it was an internal power that could directly encourage nursing students to learn and an inward requirement that could strongly stimulate them to learn. The FC teaching model was of benefit to self-learning ability, because it helped students to adjust their psychological activity process of self-awareness, self-evaluation, self-development, selfeducation, and self-control during the course of learning. Higher scores in self-management skills ref l ected that learners in the FC could communicate with other classmates more positively and actively, cooperate with each other more harmoniously, and more willingly seek for help in the face of difficulties and obstacles. The reason might be that students must work at their own pace to master per-determined material with little or no help from an instructor.42The FC teaching model was also beneficial to information literacy, because it improved the ability of nursing students to seek relevant information for themselves and evaluate how to utilize learning resources better.6The pooled effect size for self-directed learning assessed by the SRL scale revealed that the FC was superior to the control group in the training of self-directed learning skills. According to the sensitivity analyses, the result is extremely stable.

Figure 5. Forest plot of subscale scores measured by SRL.

Through the implementation of FC teaching, nursing students could acquire skills on how to apply information and resource and build capabilities and competencies by themselves rather than simply gain knowledge from books.43In addition to this, the chop and change fantasticality characteristic of nursing work demanded current nurses be equipped with problem-solving, critical thinking, and information literacy off their own bat. More specifically, the cultivation plan of FC teaching better satisfied the above requirements by way of various learning activities serving as a datum line for the FC teaching design, including mini-lectures, self-learning resources,and debates/discussion.6Conversely, individual students in traditional lecture-based classrooms were less motivated to study by autonomous learning. Acting as empty vessels absorbing information passively, they did not energetically participate in processing learning information, developing further understanding, or translating monotonous knowledge into practice.44Therefore, the traditional classroom was not beneficial to explore initiative and self-directed learning.

4.2. Strengths and limitations

In this systematic review and meta-analysis, there were several aspects worth strengthening as follows: a more rigorous and comprehensive search strategy was developed by the authors, who received training on professional evidence-based courses, which broadly included the MeSH and keywords of FC, self-directed learning,and nursing education; the research was conducted on eight databases, and as many relevant articles were retrieved as possible; the quality of the included studies was moderate, and the overall sample size was adequate; most of the included studies well reported low risk on attrition bias and reporting bias, so the result could ref l ect the true effect of the FC model.

Table 3. Description of self-directed learning measured by other scales.

Meanwhile, findings of this paper should be interpreted in line with the following limitations: first, some potential studies have been possibly missed although the retrieval process was extensive and inclusive. Second, this review was limited by the small number of non-RCT studies that met the inclusion and quality criteria.The selection and extraction stages should be assessed independently by four reviewers to mitigate limitations.Lastly, due to the limitations of the RCT study type,many included studies ascertained self-directed learning by a diversity of measuring tools. The result was that statistical data analysis could not be reliably performed.

4.3. Implications for practice and future research

Clinical nursing practice in a dynamic health care environment requires more than knowledge just gained from books. Clinical nurses must also demonstrate essential self-directed learning skills such as self-learning ability,self-management, and self-engagement. Developing self-directed learning skills was inf l uenced by personality responsibility and individual characteristics.45The evaluation of self-directed learning should focus on not only objective variables but also responsibility and characteristics. Whether the responsibility and characteristics of students were framed positively or negatively, it was often in keeping with the individuals'learning experience in the context of the FC. Improving self-directed learning ability is a lengthy process; it permeates into the periods of classroom teaching, internship, and clinical work.46Furthermore, a comprehensive assessment system should be exploited to assess selfdirected learning.

5. Conclusions

This systematic review and meta-analysis provided specific evidence on adopting the FC teaching method to improve self-directed learning in nursing education. By using the SRL, ALC, SDLRS, and CALNS as surveying instruments for assessing self-directed learning, the result of the meta-analysis proved that FC teaching could enhance the self-directed learning, including learning motivation, self-learning ability, self-management skills,and information literacy. However, due to a bewildering diversity of measuring tools, future studies with uniform measurement tools and more RCTs are needed to draw conclusions from the current study.

Conflicts of interest

All contributing authors declare no conflicts of interest.