Best Practice for Teaching and Learning Strategies to Facilitate Student Reflection in Pre-Registration Health Professional Education: An Integrative Review

Abstract

There is a growing expectation from registering authorities and the public for health professionals to enter their respective professions armed with the necessary skills, attitudes and behaviours to be reflective practitioners. However, there is limited and inconsistent evidence of the effectiveness of pedagogic strategies used for teaching reflective practice in health practitioner courses. We therefore conducted an integrative literature review of recent original studies (2004-2014) to gain an understanding of current initiatives for teaching reflective practice. A key finding from the review is the shift from the reliance on written reflective activities to more dialogic and social based reflection. Studies that use reflective dialogue suggest that facilitated dialogue with peers and clinical educators is the most effective approach to foster deeper reflection, critical thinking and clinical reasoning. Also the consensus evident from the review is that for reflection to be meaningful it needs to be intentionally connected to clinical programs. Although we find compelling evidence of the usefulness of embedding reflective learning strategies in health curricula, repeated reference to a lack of adequate training to prepare health educators to teach reflective practice suggests that there is an urgent need for research into how this can be achieved.

Share and Cite:

McLeod, G. , Barr, J. and Welch, A. (2015) Best Practice for Teaching and Learning Strategies to Facilitate Student Reflection in Pre-Registration Health Professional Education: An Integrative Review. Creative Education, 6, 440-454. doi: 10.4236/ce.2015.64044.

1. Introduction

Reflective practice (RP) is considered as a core competency of contemporary health practitioners ( Smith & Trede, 2013 ). Registration authorities require practitioners to evaluate their practice through self-reflection and self- assessment ( Australian Physiotherapy Council, 2006 ; Medical Board of Australia, 2014 ; Medical Radiation Practice Board of Australia, 2014 ; Nursing and Midwifery Board of Australia, 2010 ). Reflective practice is the process of paying attention to what is occurring in practice, and thoughtfully considering the impact of personal, cultural and social assumptions on experiences. This evaluation requires the capacity to think critically and gain knowledge and understanding to improve practice ( Delany & Watkin, 2009 ). The demands placed on health practitioners in the 21st century require a capacity to be flexible, independent and reflective.

The theory of reflection and reflective thinking can be traced back to the early writings of the educational reformer John Dewey. He described reflection as the “active, persistent and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it” ( Dewey, 1933: p. 113 ). His early work has been progressed by others, such as Schön (1983) , Boud, Keogh, & Walker (1985) , Brookfield (1995) and Mezirow (1990) .

Within health professional education, the introduction of learning strategies aimed to promote RP is relatively recent, with nursing paving the way in the 1990s ( Johns, 1995 ; Nicholl & Higgins, 2004: p. 580 ). Medicine and the allied health professions have been slower to incorporate reflective learning strategies and much of the literature dealing with RP from these professions remains predominantly theoretical.

Although educational strategies that promote RP have been reported in the literature, the effectiveness of these has been disputed, with the findings often being inconclusive or inconsistent ( Delany & Watkin, 2009 ; Mann, Gordon, & MacLeod, 2009 ; Wessel & Larin, 2006 ). Additionally, limited guidelines for facilitating student reflection are available ( Canniford & Fox-Young, 2014 ). Boud & Walker (1998) examined the common practices going “under the banner of reflection” and found that at times what was being taught was not effective in promoting reflection due to a possible misunderstanding of the ideas ( Boud, 2010: p. 27 ). In part, Boud attributed this to the technical orientation of educators who saw reflective learning in the same light as instrumental learning akin to the concept of adherence to rigid protocols.

However, a number of review papers published over the past decade provide excellent coverage of a number of pertinent views related to reflection and reflective practice. Examples include a systematic review by Mann, Gordon, & MacLeod (2009) in which they question the nature of student reflection can be developed or assessed and what contextual factors influence reflection. Alternatively, Norrie et al. (2012) focus their review of the literature on teaching “reflective practice across several different professions” ( Norrie et al., 2012: p. 566 ). They present an informative paper exploring the different epistemological underpinnings to teaching reflection across the health professions, and the implications for inter-professional practice. Dyment & O’Connell (2011) review the literature assessing reflection in student journals. Their review identifies methods of assessment and concludes that one consistent approach may not be appropriate due to the differing samples, approaches to reflection and aims of the activity. They do, however, suggest that a single assessment technique may provide an unambiguous signal that reflective learning is a recognized, honored and valued approach ( Dyment & O’Connell, 2011: p. 92 ).

Each of these reviews informs and extends the theory and knowledge of teaching and assessing reflective practice. Our integrative literature review complements them by identifying and synthesizing strategies that have been reported for teaching reflective practice in pre-registration health professional courses. We consider this as an important starting point for educators wishing to prepare their students for clinical practice.

Aim

The aim of this paper is to present an overview of the best practice for reflective learning strategies used in pre- registration courses to prepare students to become reflective health practitioners.

2. Method

An integrative review of the literature was undertaken to determine what pedagogical strategies to prepare pre- registration health students to be reflective practitioners have been reported in the recent literature. The integrative review method was chosen because it provides a structured means to “identify, analyze and synthesize the

findings from numerous independent peer-reviewed studies to determine the current knowledge (what is known and not known) in a particular area” ( Burns, Grove, & Gray, 2011: p. 540 ). First, inclusion and exclusion criteria were determined, second a critical appraisal tool, the Critical Appraisal Skills Programme (CASP) tool was selected to evaluate the rigour of the research reported, and finally, selected studies were read and coded by independent reviewers using a directed content analysis approach. The conclusion of the analysis will provide the best available evidence to inform educators about facilitating reflective practice in pre-registration health courses. Before describing the review strategy in detail, the theoretical framework that guided this review is considered.

2.1. Conceptual Framework: Transformational Learning Theory

Transformational Learning Theory (TLT) ( Mezirow, 1991; 1995 ) provides the conceptual framework underpinning this review. Mezirow (1997) views transformative learning as a rational process of metacognition driven by critical reflection on assumptions. That is, critical reflection on assumptions can transform a “frame of reference―a mind-set, perspective or worldview”, leading to new knowledge and transformed action ( Dirkx & Mezirow, 2006: p. 124 ). The underlying premise that transformative learning results in transformed perspective and improved practice guided the selection of papers to include in this review―that is, we searched for studies that contain strategies that provide students with the opportunity and support to critically reflect on their practice and underlying assumptions. TLT also informed the data analysis process by drawing our attention to the characteristics of the learning process and outcomes presented in each paper. Of particular importance was the process of critical analysis of experience through reflection, and consideration of other perspectives ( Mezirow, 1997 ).

2.2. Search Strategy and Selection Process

A comprehensive online search of databases was conducted to locate research that describes frameworks and strategies for teaching and learning RP in pre-registration health professional courses. Online databases―Scopus, PubMed and the Cumulative Index to Nursing and Allied Health Literature (CINAHL)―were searched using combinations of key terms: reflective practice, frameworks, reflection, higher education, and health professional education. The search was restricted to primary research studies published in English between 2004 and 2014. A total of 87 studies were identified. In order to limit this review to relevant material, we developed the inclusion and exclusion criteria shown in Table 1 based on the review aim and knowledge of the RP literature. The literature shows evidence of quantifying certain attributes of RP using statistical measurement scales. Examples of RP attributes that have been evaluated in this way include personality traits ( Rosenthal et al., 2011 ), clinical judgment ( Nielsen, Stragnell, & Jester, 2007 ) and diagnostic thinking ( Sobral, 2005 ).

What is less understood is the application of reflection. Whilst there are some reliable and valid quantifying tools to measure these attributes, to understand the nuances of context and the experience of participants necessitates an interpretive and constructivist approach we therefore decided that only qualitative data would be used.

If a study used mixed methods, only the qualitative component was considered in this review and no quantitative component or study included. The decision to exclude quantitative research was made following discussion

Table 1. Inclusion criteria for selection of published studies.

and review of scholarly works about meta-analysis and synthesising literature. Mann et al. (2009) cautioned that quantifying and measuring RP efficacy can be challenging due to multiple interaction effects of such complex phenomena as RP and critical reflection. Additionally, aggregation of quantitative and qualitative data is not recommended due to the differing paradigms that provide direction of purpose to the inquiry ( Sandelowski, Barroso, & Voils, 2007 ). Therefore, in this case, a description of best practice based on the analysis of qualitative evidence was seen as useful information. Alternatively, knowledge about the interpretive paradigm achieves rich description to gain a deeper understanding.

We had originally intended to include papers published over the past twenty years (1994-2014). A preliminary search revealed that many early papers on RP offered theoretical expositions, some suggesting models and frameworks for RP pedagogy, but few discussed the implementation or evaluation of RP programs. In contrast, the RP literature since the mid-2000s has reported the findings from evaluation of RP programs in greater detail. Although theoretical discussions surrounding reflection is valuable, our aim here was to examine the empirical evidence for implementing reflection into curricula. We therefore decided to reduce the search timeframe to the last ten years (2004-2014).

Of the 87 studies identified through our initial search, 36 met the inclusion criteria. Examination of reference lists of the key literature located a further two relevant studies. Following critical appraisal 26 were deemed unsuitable. The final count was twelve studies for inclusion in this review. Figure 1 summarizes the search and selection process used for selection of the studies.

Critical Appraisal

To ensure that only papers of high quality were included in the review, a systematic and rigorous assessment of the quality of the studies that met the inclusion criteria was subsequently performed using the Critical Appraisal Skills Programme (CASP) tool for qualitative research ( Critical Appraisal Skills Programme (CASP), 2014 ). This appraisal tool was developed to assist researchers to make informed decisions about the quality of research evidence by identifying methodological flaws that may be present in published research.

The CASP tool uses ten questions to evaluate rigor and reliability. Each study was screened against the criti-

Figure 1. Summary of search and selection strategy.

cal appraisal questions to ensure the included studies used a thorough and appropriate research design and that the findings are relevant to the aim of this review. The first two questions of the ten provide a quick screening; if a study meets these two criteria, it is worth proceeding with the remaining questions. Twelve studies met the initial screening and were consequently critically appraised. Table 2 shows the results of the critical appraisal results of these twelve studies.

Of the twelve studies, ten attained a positive match to all appraisal questions, indicating they were of a high methodological quality and relevant to the aims of this review. A major strength of these studies was found in the description and evaluation of reflective learning strategies used by health educators to facilitate student reflective capacity. Two studies had limited representation for at least one of the criteria. One study ( Moriarty & McKinlay, 2008 ), in addition to data analysis by the researcher, there is reference to the use of a sociolinguist to assist the analysis, but the authors failed to sufficiently describe the significance of this analysis to the findings. Canniford & Fox-Young (2014) did not seek ethical clearance to use student data, however the authors were transparent in their reasoning, that the activities were part of the coursework for the module, and hence not required. Although these limitations may have reduced the rigor, close examination of the methodological design and importance of the findings to our review, resulted in the decision that both studies were sufficiently robust to be included. The entry for Donaghy & Morss represents two papers that, for the purposes of this review, we con- sider as one: a framework ( Donaghy & Morss, 2000 ) and a subsequent paper evaluating that framework ( Donaghy & Morss, 2007 ).

A short overview of each of the twelve studies is given in Table 3.

3. Data Analysis

The twelve studies were read and independently coded by the three review authors. Analysis used a directed content analysis approach, which identifies key concepts related to the phenomenon of interest―in this case, teaching and learning reflective practice. The main strength of this approach is the opportunity to focus on the phenomenon of interest with the goal of extending and supporting existing theory ( Polit & Beck, 2008 ).

The authors of this integrative review all practice and teach reflective practice; it was therefore important to consider the pre-assumptions of the authors and views gained from the existing literature prior to further data analysis in the integrative literature review. Current understandings held by the authors included encouraging an examination of the level of student engagement with RP, obtaining students’ views of engaging in the RP activities, and the importance of context and content of student reflections. We were, however, keen to be opened to new understandings about the application of RP for health care students and therefore adopted a questioning approach to reviewing the combined data found within the studies, knowing that new insights may be possible when considering all findings ( Polit & Beck, 2008 ).

Table 2. Summary of critical appraisal.

Table 3. Summary of studies included in review.

To direct our analysis and synthesis of the selected studies we developed three a priori themes:

・ Learning strategies to facilitate RP;

・ Models of reflection, context and implementation of learning strategies;

・ Recommendations proposed to enhance the effectiveness of learning strategies for RP.

4. Findings

The health professions represented in the twelve studies were medicine (4), nursing & midwifery including mental health nursing (4), physiotherapy (3), and radiation therapy (1). The findings are presented under the three a priori themes.

4.1. Theme 1: Learning Strategies to Facilitate Reflective Practice

A variety of strategies for teaching RP were identified in the reviewed studies and are listed in Table 4. Strategies were used both in isolation and in combination. The practice of critical reflection was central to all strategies.

4.1.1. Reflective Writing

Of the written reflective formats, journaling was the strategy most frequently reported. Written reflections were referred to as incident reports in two studies ( Delany & Watkin, 2009 ; Nicholl & Higgins, 2004 ) and as learning contracts in one ( Ramli et al., 2013 ). Although the process for reflection was similar in all studies, the reflective output in some was part of specific learning activities, such as patient care reports and learning contracts. Similarly, two studies from the medical profession used a portfolio format that contained written reflections as a component ( Howe et al., 2009 ; Pitkälä & Mäntyranta, 2004 ). Of the eleven studies that used journaling, two employed a web-based platform ( Canniford & Fox-Young, 2014 ; Cooper et al., 2005 ).

1) Journaling

Most studies in our review supported the use of reflective journaling as an active learning strategy to capture and foster transformative learning and to develop reflective practice. In particular, student transformation occurred in a number of areas, including, decision making, awareness of patient concerns and circumstances, and socio-political decision making.

Moriarty & McKinlay (2008) , for example, analysed reflective journals of seventy-nine medical students. The students, using a predesigned journal template, reflected on an experience stimulated by a community health placement. The authors found evidence in the reflections of transformed thinking and transformed perspectives ( Mezirow, 1997 ) related to health professional ethos, acknowledgment of patient perspectives on health and health care, and the socio-political environment of health care delivery.

From the radiation therapy literature, Abrahams (2012) evaluated a reflective learning program involving pre- registration graduates during supervised practice. Analysis of reflective journals as well as feedback from a questionnaire indicated that students valued reflective writing as an opportunity to analyse past events, review how they were handled and why, and to consider how decisions would be made in similar circumstances (Abra- hams, 2012: p. 44).

Donaghy & Morss (2000) used a reflective patient case format to frame the reflection, thus creating a targeted micro-context that focused the reflection to just the process of data analysis and problem identification. They postulated that reflection on an entire patient consultation would impede student engagement with reflection due to the overwhelming quantity and complexity of the entire clinical process ( Donaghy & Morss, 2000: p. 10 ). Additionally, Ramli et al. (2013) employed a reflective report as part of a learning contract to guide students to choose a specific learning task on which to focus during their clinical placement.

2) Personal Portfolios

Only two studies were included in the review that used portfolios. In both studies, medical students produced portfolios in which written reflections formed part of an ongoing record of their experience of learning to be a professional. Howe et al. (2009) analysed the students’ in-depth reflective component, written each year over the five years of the course. The authors found that the students used their reflections to re-examine their experiences, from which they then strived to expand their involvement with professional standards and principles.

Table 4. Strategies identified in reviewed literature for teaching reflective practice.

They also identified a number of students who struggled to understand complex personal and social issues such as racism and patient safety ( Howe et al., 2009: p. 949 ).

The second study that utilised reflection as a component of a portfolio ( Pitkälä & Mäntyranta, 2004 ) involved twenty-two Finnish medical students participating in an optional portfolio course offered during the first clinical year. Detailed prompts to encourage reflection on practice focused students to explore their personal values, feelings and emotions resulting from patient encounters, and their experience in the role of student-physician. Analysis of the student reflections revealed three main areas of student development: exploration of emotional experiences such as feelings related to their own competence; confusion of role―and learning to deal with the experience of helplessness, especially in dealing with death and dying.

The above are examples of what Schön (1983) referred to as reflection-on-practice. Boud et al. (2001) suggest that reflective writing is an effective means of learning from complex and unruly experience. In parallel with the work of these two key authors, Mezirow’s transformative learning theory supports examining experiences to become critically reflective of one’s own assumptions and to revise and transform these frames of reference to inform future action ( Mezirow, 1991 ).

These findings support previous literature in which reflective journaling has been reported as one of the most frequently used strategies for enhancing reflective learning in health professional education ( Blake, 2005 ; Boud, 2001 ; Kok & Chabeli, 2002 ; Moon, 2003 ; Williams, Wesse, Gemus, & Foster-Seargean, 2002 ). Although most studies report that the writing of reflective journals, blogs and reports improved student thinking and clinical decision making, not all students felt writing down their reflections was necessary as they already “reflected in-action” ( Abrahams, 2012 ).

4.1.2. Reflective Writing Consolidated through Dialogue

Eight of the studies linked the reflective journal exercises with activities involving reflective dialogue. Dialogue in this context refers to constructive verbal interaction to explore a topic or issue from as many perspectives as is practical. Examples of this were group discussions in which students presented clinical scenarios from their journal writing for deconstruction and problem solving, and reflective dialogue in a one-on-one context with either a peer or clinical supervisor.

Abrahams (2012) evaluated a year-long reflective tutorial program involving five radiology students on pre- registration placement. In addition to keeping a reflective journal, these students subsequently shared their reflections with one of the clinical supervisors, who provided feedback. Findings from this study revealed that discussion of their personal reflections with the supervisor expanded the students’ perspective and better equipped them to evaluate their own practice. The authors support other literature that argues that deeper levels of reflection can be facilitated with searching questions from the educator that probe the experience rather than just reviewing it superficially ( Abrahams, 2012: p. 44 ). In a similar pedagogical combination of journaling with dialogue, Lutz et al. (2013) found that medical students felt supported and were opened up to new perspectives through reflective dialogue with peers and challenging questioning by supervisors.

Using a written critical incident report that was consolidated by group dialogue, physiotherapy students from a study by Delaney & Watkins (2009) reflected on issues pertaining to ethical practice, professional relationships with patients and the impact of implicit power relations in clinical scenarios.

The authors concluded from analysis of group dialogue sessions that student discussion increased recognition and awareness of their reactions, responses and ways of thinking about people and this in turn contributed to their learning on clinical placement ( Delany & Watkin, 2009: p. 423 ).

This emphasis on the use of reflective dialogue as a means of deepening learning is supported by the principles of transformative learning theory ( Mezirow, 1991 ). For a shift in perspective, new information needs to be incorporated by the learner into their existing frames of reference, which is not always possible in the protected solitary experience of journal writing. Mezirow argues that to facilitate transformative learning, educators must help learners become aware and critical of their own and others’ assumptions ( Mezirow, 1997: p. 10 ). Critical discourse with others provides the opportunity for the learner to consider other perspectives, question their own convictions and redefine their concepts of reality.

A novel method to extend the depth of student reflection, described in the study by Donaghy & Morss (2007) , was stimulated recall (p. 8), using recorded reflective dialogue between physiotherapy students and their tutors. As part of a reflective framework (described in more detail later), students on clinical placement engaged with their clinical tutor in a recorded dialogue related to their patient case. Each student was given a copy of the recorded discussion, which they then used to “reflect on their reflection” by listening to and thinking about their own responses ( Donaghy & Morss, 2007: p. 90 ). The authors suggest that the process prompts critical appraisal of previous thought processes and enables “knowledge that is normally tacit to become more explicit” ( Donaghy & Morss, 2000: p. 8 ).

4.2. Theme 2: Models of Reflection, Context and Implementation of Strategies

4.2.1. Models of Reflection

In each of the reviewed studies, instruction for reflective writing asked students to reflect on a clinical learning experience they considered significant. There were a number of models of reflection and reflective practice used to direct the research and the students’ reflective process, based on the work of key reflective practice authors. These included the “reflection-on-practice and reflection-in-practice” model from the seminal work of Donald Schön, focusing on the reflective practitioner ( Schön, 1983 ); the model of “reflection for learning from experience” of Boud, Keogh and Walker ( Boud, 2001 ; Boud et al., 1985 ); Tanner’s clinical judgement model of noticing, interpreting and responding with reflection in and on practice ( Nielsen et al., 2007 ; Tanner, 2006 ); and The Balint Group model, based on the work of psychoanalysts Michael and Enid Balint, for improving the quality and therapeutic nature of doctor-patient relationships through self-reflection and exploration of meaning ( The Balint Society, 2012 ). A common feature of all of these models is that they provide a structured approach to reflection with the aim of directing the learner through a series of phases in the reflective process: a description of the situation or incident, a deeper examination including underlying assumptions and the perspective of others, and planning how to approach similar situations in the future.

This review found the most cited model of reflection to be that of Boud et al. ( Boud, 2001 ; Boud et al., 1985 ). The extensive descriptions for implementation render this model relevant and accessible to the busy educator and others looking for practical application of the theories supporting reflective practice. This model is consistent with transformative learning theory ( Mezirow & Associates, 1990 ), where learning is underpinned by the assumptions that learning is always grounded in prior experience and learning from experience necessitates active engagement with thoughts and emotions ( Boud, 2001: p. 2 ). The three main tenets of the model are returning to experience―providing a description of the issue or event; attending to (or connecting with) feelings―this involves acknowledging both positive and negative feelings with the intent to make constructive use of them; and evaluating experience―this may include examining the assumptions brought to the experience as well as ways of integrating new knowledge ( Boud et al., 1985: p. 26 ).

4.2.2. Contextual Implications for Reflection on Practice

Each of the reviewed papers implemented reflective practice as a learning tool linked to the context of patient contact during clinical practice. The reflective activities described aimed to develop higher order cognitive pro- cesses associated with reflective practice ( Donaghy & Morss, 2007 ). These cognitive processes include skills of critical inquiry ( Delany & Watkin, 2009 ), problem solving ( Donaghy & Morss, 2007 ) and clinical reasoning that provided students with the necessary knowledge and rationale for reflection and its function in reflective practice ( Abrahams, 2012 ; Delany & Watkin, 2009 ; Nicholl & Higgins, 2004 ). In a number of the studies, student learning materials such as handbooks providing theoretical concepts and guidelines for reflecting were also made available to students in hard copy ( Delany & Watkin, 2009 ) or in a variety of electronic formats online ( Canniford & Fox-Young, 2014 ; Cooper et al., 2005 ).

As described in the previous section, reflective writing formed the central method for reflection, often consolidated and extended by reflective dialogue with peers and educators. There is a growing consensus in the literature that reflection must be a social endeavour, in addition to requiring experiential learning opportunities ( Boud, 2010 ; Smith & Trede, 2013 ). Of particular importance is reflective discourse. The success of these reflective dialogue sessions requires skilled facilitation by experienced educators who can pose challenging and probing questions at the same time as managing group dynamics ( Nicholl & Higgins, 2004 ; Stockhausen, 2005 ).

In two studies, students posted their reflections online. Cooper, Taft, & Thelen (2005) evaluated a study using the Web CT online course management system, in which thirty-two senior nursing students posted weekly reflections about their clinical experience over a six week period. Online asynchronous conferencing was used to provide support and constructive direction to the students, especially those experiencing clinical challenges. Findings from analysis of the student reflections indicated that students focused on confronting ethical issues, self-evaluation of their learning needs and recognition of several factors that limit their current nursing practice ( Cooper et al., 2005: p. 295 ).

In another study using an online format, Canniford & Fox-Young (2014) introduced an interactive online learning package into an undergraduate nursing program, which included information and instructional videos to supplement online reflective activities. Students were supplied with reflective writing guidelines based on a modified version of Tanner’s Clinical Judgement Model. This model provided headings for students to organise their reflection: noticing, interpreting, responding, reflection-in-action and reflection-on-action/clinical learning. Lecturers provided feedback (also online) to the reflections, to which students were required to respond with reflection on the feedback. Evaluation of the initiative, using open and closed questions on a Likert scale, revealed mixed reactions from the students with 53% reporting they did not find the program stimulating. The open-ended questions revealed ambivalence towards the program, with the most positive part of the program being the online feedback from lecturers.

4.2.3. Stand-Alone Reflective Modules

Of particular interest were four studies that implemented an optional “stand-alone” reflective module or framework, where the critical reflection program was offered as a separate program that focused exclusively on reflective learning activities during clinical placement.

The first of these studies involved 26 physiotherapy students in a six week, three hour per week critical reflection program, conducted in the hospital setting and led by one of the researchers who were also a trained reflection facilitator ( Delany & Watkin, 2009 ). Each session followed a staged process of group critical reflection based on the work of Boud et al. (2006) . Students shared a critical incident from their week and the group engaged in a guided deconstruction of the underlying knowledge, values and assumptions that underpinned the incident. The authors found that the program broadened student thinking and helped them gain confidence to deal with the demands of clinical performance. Highlighted in the findings was the benefit of learning from others―from listening to other students deconstruct their stories and describe how they dealt with their situation. Some students negatively commented that the separate nature of the program took away from time they could have spent in clinical practice.

From a community-based context, Moriarty & McKinlay (2008) describe an innovative reflective program in New Zealand where second year medical students (in pairs) undertook a placement visiting chronically ill patients in their homes. With the aim of fostering transformative learning and addressing the demands of a stressful profession, the students attended open group therapy and self-help sessions in addition to keeping a reflective journal. Qualitative analysis of student journals identified three areas of benefit: changes in student perspective and the development of a new frame of reference; interpreting meaning from emotionally charged experiences; and the use of new tools to understand observed apparent anomalies in practice.

With a similar objective to guide medical students through the stresses of their training, Lutz et al. (2013) introduced an innovative voluntary program known as Critical Reflection Training (CRT). Framed by the concern that professional capabilities such as empathy and patient-centeredness decline during medical education, the authors offered the CRT to medical students completing their final rotation at a German teaching hospital. The 90-minute CRT group sessions were run every two weeks for a total of five sessions. Based on the Balint group method ( The Balint Society, 2012 ), the sessions were facilitated by a trained psychosomatic medicine specialist. Once introduced to the topic of reflective practice and guided in the practice of observing for professional dilemmas on the ward, 18 participating students brought observations and questions back to the group for reflection around psychosocial, moral, personal and interpersonal issues. The authors found that the CRT reduced student stress, enhanced the quality of patient care, and led to personal professional development.

The fourth example of these stand-alone reflective modules, a three-phase reflective framework that was integrated into a semester long clinical placement for Year 3 physiotherapy students ( Donaghy & Morss, 2000 ). The framework consisted of a half-day workshop (to outline the process of reflection, reflective dialogue and examination of written examples of reflection), written reflections on patient reports, a recorded meeting to discuss reflections with the clinical educator and a final write-up of the reflective process to extend and further the reflection. The authors repeated the framework with three separate cohorts over three years. Evaluation of focus group data involving 43 students from two of the cohorts found that participation in the framework increased student insight and awareness into their practice including assumptions about their patients and identification about their strengths and weaknesses. The most helpful part of the framework from the students’ reports was

reflective dialogue with their clinical tutors. These findings confirm previous findings of such integrated reflective learning where students can engage with their clinical educators to more deeply explore the issues raised in their reflections.

Although each of these stand-alone reflective programs reported positive learning outcomes for students, Lutz et al. (2013) questioned the generalizability of such programs due to selection bias. They suggest that students who agree to take part may be more motivated and reflective than the norm. Of equal concern is how to engage those students who do not participate.

4.3. Theme 3: Recommendations to Enhance Effectiveness of Learning Strategies

From the reviewed studies, we identified a number of recommendations for successful inclusion of reflective learning strategies into healthcare curricula.

4.3.1. Reflective Activities Need to Be Contextually Set Within Clinical Practice

The evidence in the studies suggests that for reflection to be meaningful it must be embedded within real life practice. It is the practice of reflection in context that educators must emphasise, where engagement with activity and current problems from practice may lead the learner to a shift in perspective ( Moriarty & McKinlay, 2008 ). In the study by Abrahams (2012) , radiation therapy students undertook a reflective practice program that was integrated into supervised clinical practice, allowing the students to reflect-on-action as they experimented with practice techniques in a safe environment (Abrahams, 2012). Lutz et al. (2013) evaluated a study where medical students participated in reflective discussions with supervisors and other medical staff during ward rotations. Through these shared reflections the students engaged with others to work through clinical problems resulting in increased professional development as well as reduced stress and increased satisfaction ( Lutz et al., 2013: p. 344 ).

4.3.2. Train Staff to Facilitate and Teach Reflection

Central to the success of reflective programs is skilled mentoring from appropriately-trained educators. In the reviewed studies we found repeated reference to the need for staff training to prepare teachers to perform roles associated with facilitating critical reflection ( Cooper et al., 2005 ; Delany & Watkin, 2009 ; Donaghy & Morss, 2007 ; Nicholl & Higgins, 2004 ). Teachers in a study by Nicholl found teaching reflection to be “emotionally exhausting”. They suggested the skills required to teach reflection included “group facilitation skills”, “skills in handling sensitive topics”, “an ability to respond to distressed students in a group context”, and skills in “teaching students with different life experiences and levels of emotional maturity” ( Nicholl & Higgins, 2004: p. 582 ).

4.3.3. Incorporate Reflective Dialogue

Several studies suggest that reflective dialogue for exchange of ideas and differing perspectives in combination with written reflections is more effective than written reports alone ( Donaghy & Morss, 2007 ; Lutz et al., 2013 ). While on placement, structured reflective dialogue with an academic has the potential to stimulate further reflection on a clinical experience, driving analysis of actions, reactions, thoughts and decisions ( Donaghy & Morss, 2007 ) and clarifying learning objectives ( Ramli et al., 2013 ). Verbalisation of thoughts and exposure to the ideas of others appears to expose one’s thinking to a new level of scrutiny that will highlight biases or blind spots by deconstructing and examining values ( Delany & Watkin, 2009 ; Mezirow & Associates, 2000 ; Ramli et al., 2013 ). Differences of opinion coupled with constructive critique may encourage self-assessment and perspective transformation with the possibility of improved inter-professional relationships and patient care.

4.3.4. Repeat Encounters with Reflection throughout the Curriculum

As with any cognitive skill, reflection requires repeated practice throughout the curriculum for the development of deeper and more meaningful reflection ( Donaghy & Morss, 2000 ; Moon, 2004). Donaghy & Morss argue that by placing structured reflective activity across the curriculum and into each stage of clinical practicum, students may be stimulated to reflect each time they engage in clinical practice ( Donaghy & Morss, 2000: p. 10 ). There is also evidence from the medical profession that scaffolding reflection throughout the curriculum may enhance self-awareness and encourage student self-care in the face of overwhelming emotional strain from the stressful and complex clinical situations with which they are faced ( Lutz et al., 2013 ; Pitkälä & Mäntyranta, 2004 ).

4.3.5. Provide Timely Personalised Feedback

Students welcome timely feedback from educators on their reflective work. Such feedback provides direction for the student to understand the expectations of the learning activity, conveys to students the sense of being supported ( Abrahams, 2012 ; Cooper et al., 2005 ), increases the level and depth of subsequent reflection ( Fisher, 2003 ), and provides context for their own self-assessment and planning for learning needs. From evaluation of an online reflective program, it was found that the immediacy of timely online-feedback assisted the development of their online reflective blogs ( Abrahams, 2012: p. 3 ).

These recommendations, whilst challenging―especially for the educator new to reflective teaching―have been reported in the reviewed studies to increase the efficacy of these reflective programs.

5. Discussion

The findings from this review indicate that health educators involved in pre-registration healthcare courses utilise a variety of pedagogical strategies to encourage the development of reflective practice skills. Learning to critically reflect is the core learning activity underpinning each of the educational strategies identified in the reviewed papers. Written reflection is the most commonly used method, in the form of journals ( Abrahams, 2012 ; Moriarty & McKinlay, 2008 ), online blogs ( Canniford & Fox-Young, 2014 ; Cooper et al., 2005 ), learning contracts ( Ramli et al., 2013 ), patient case reports ( Donaghy & Morss, 2000, 2007 ) and portfolios ( Howe et al., 2009 ; Pitkälä & Mäntyranta, 2004 ).

There is a consensus from all authors in the reviewed studies that reflection during the clinical phase of courses is crucial for students to integrate theory with practice. They found evidence that this contextualised re- flection helped students develop a sense of the patient’s experience ( Stockhausen, 2005 ), critically examine their own practical and professional development ( Howe et al., 2009 ) and strengthen their inter- and intra-personal attitudes and skills in dealing with stressful and complex clinical situations ( Lutz et al., 2013 ; Moriarty & McKinlay, 2008 ).

An important key finding to emerge from the reviewed studies is that reflection is particularly beneficial when incorporated into more interactive and social processes. The studies suggest that supportive reflective dialogue with peers and academics increased student awareness of the possibilities of alternative ways of looking at their practice and the practice of others ( Abrahams, 2012 ; Delany & Watkin, 2009 ; Donaghy & Morss, 2007 ; Lutz et al., 2013 ; Nicholl & Higgins, 2004 ; Pitkälä & Mäntyranta, 2004 ; Stockhausen, 2005 ). Students reported that sharing reflections including their feelings, thoughts and perceptions helped them to consider other points of view ( Delany & Watkin, 2009 ) and draw conclusions from their clinical experiences ( Stockhausen, 2005 ). This view is supported by a previous review by Mann, Gordon, & MacLeod (2009) and by TLT, in which it is argued that dialogue with others can assist the learner to move personal awareness from introspection to a broader perspective, by exposure to alternative thinking and reflective analysis ( Mezirow & Associates, 2000 ).

Brockbank & Mills (2007) make the distinction between ordinary talk and reflective dialogue, in that the former involves the speaker reporting their experience without necessarily sharing any personal content whereas the latter “requires careful listening: it implies a mutually shared agreement”―together, deeper experiences and reflections regarding feelings, ideas and perceptions will be exchanged and explored ( Brockbank & McGill, 2007: p. 67 ).

A number of issues, related to educators using reflection as a learning strategy, were identified from the review. First, teaching staff raised the concern that they lacked suitable training both on how to reflect and on how to assist others in the process of critical reflection ( Delany & Watkin, 2009 ; Lutz et al., 2013 ; Nicholl & Higgins, 2004 ). Analysis of a questionnaire from twenty nurse educators found that running such sessions can be challenging for educators lacking training and experience in reflective teaching ( Nicholl & Higgins, 2004 ). Second, there needs to be a shift in curriculum planning to allow time within clinical training for explicit reflective learn- ing activities ( Delany & Watkin, 2009 ). And finally, teaching staff, especially those new to reflective learning, require support networks for their own personal debriefing but also for students who experience distress ( Nicholl & Higgins, 2004 ).

5.1. Limitations of This Study

Although we used peer-reviewed literature to learn what strategies have been implemented, we also recognised that the format of a journal paper is perhaps not adequate to fully describe a RP learning program in detail. Some authors have incorporated curriculum guidelines and teaching materials as data or appendixes in an attempt to present a comprehensive overview, although with varying success ( Delany & Watkin, 2009 ; Norrie et al., 2012 ). Others have confined their reporting to a summary of a strategy, or of a single moment during practitioner education or continuing professional development. What we have attempted to present here is a summary and synthesis of best practice for teaching reflection as presented in the reviewed papers. For more detail of the programs the reader is referred to the individual papers where contact details of the authors can be found.

A limitation of any study assessing the effectiveness of RP learning strategies is the validity and relevance of the findings. Data for studies in this review were based on written student work, student self-reports, or both. Criticism directed at the problems of using such data can readily be found in the literature―respondents telling the researcher what they think the researcher wants to hear, and respondents misreporting skill or technique use ( Owings Swan & Hoffer, 2008 ). This does not mean that student work or self-reports are not useful forms of data, as evaluation of the effectiveness of learning programs should include appraisal from its participants’ perspectives, and the use of data from multiple studies provides triangulation of the findings ( Leech & Onwueg- buzie, 2007 ).

Evidence of reflection is difficult to evaluate, not only because of limitations in factors such as pre-existing personality traits and a lack of measures that can assess reflective attributes and changes in these attributes, but also due to the elusive nature of reflective thinking ( Hubbs & Brand, 2010 ). Ethical issues related to the use of participants’ personal reflections as data, and randomisation processes including selection of control groups, compound the difficulty. Furthermore, differing philosophical perspectives between the health professions leads to divergence as to what constitutes reflective capacity ( Norrie et al., 2012 ).

5.2. Concluding Remarks

This integrated literature review has provided a synthesis of findings from twelve original research studies evaluating reflective learning strategies in pre-registration healthcare education curricula. The findings revealed a variety of educational approaches to teaching reflective practice. Although written reflection is the most common format utilized, there is evidence that it may be not sufficient on its own. Reflective dialogue with peers and clinical educators appears to be the most effective approach to foster deeper reflection. Several innovative stand-alone reflective courses report very positive outcomes for student learning.

The reviewed studies support the continued use of reflective learning activities in pre-registration healthcare courses, but with increased emphasis on the development of strategies that involve dialogue and learning activities that are related to relevant “real-world” practice. It is also important that students should be provided with meaningful instruction and constructive, timely feedback from experienced mentors and educators.

While there is literature that explicates the methods for teaching RP, there is still a need for greater transparency of such programs. Further research is needed to examine more carefully the factors involved and the effects of different educational formats, in particular the differences between written reflection and reflective dialogue, and to understand why and how different reflective learning strategies work. We also identified a need for research into practical methods of assisting those responsible for training health care students to reflect and supporting them in that endeavour.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1] Abrahams, K. (2012). Evaluation of a Reflective Learning Programme for Radiation Therapy Graduates. Radiographer: The Official Journal of the Australian Institute of Radiography, 59, 40-45.
[2] Australian Physiotherapy Council (2006). Australian Standards for Physiotherapy. In Safe and Effective Physiotherapy (pp. 1-82). Canberra: Australian.
http://www.physiocouncil.com.au/files/the-australian
-standards-for-physiotherapy
[3] Blake, T. (2005). Journaling: An Active Learning Technique. International Journal of Nursing Education Scholarship, 2.
http://dx.doi.org/10.2202/1548-923X.1116
[4] Boud, D. (2001). Using Journal Writing to Enhance Reflective Practice. New Directions for Adult and Continuing Education, 2001, 9-18.
http://dx.doi.org/10.1002/ace.16
[5] Boud, D. (2010). Relocating Reflection in the Context of Practice. In H. Bradbury, N. Frost, & S. Kilminster (Eds.), Beyond Reflective Practice (pp. 25-36). New York: Routledge.
[6] Boud, D., & Walker, D. (1998). Promoting Reflection in Professional Courses: The Challenge of Context. Studies in Higher Education, 23, 191-206.
http://dx.doi.org/10.1080/03075079812331380384
[7] Boud, D., Cressey, P., & Docherty, P. (2006). Productive Reflection at Work. Learning for Changing Organisations. New York: Routledge.
[8] Boud, D., Keogh, R., & Walker, D. (1985). Promoting Reflection in Learning: A Model. Reflection: Turning Reflection into Learning. London: Routledge.
[9] Brockbank, A., & McGill, I. (2007). Facilitating Reflective Learning in Higher Education (2nd ed.). Maidenhead: Open University Press McGraw-Hill.
[10] Brookfield, S. (1995). Becoming a Critically Reflective Teacher. San Francisco: Jossey-Bass.
[11] Burns, N., Grove, S., & Gray, J. (2011). Understanding Nursing Research: Building an Evidence-Based Practice (5th ed.). St Louis, MO: Elsevier Saunders.
[12] Canniford, L. J., & Fox-Young, S. (2014). Learning and Assessing Competence in Reflective Practice: Student Evaluation of the Relative Value of Aspects of an Integrated, Interactive Reflective Practice Syllabus. Collegian, in Press.
http://dx.doi.org/10.1016/j.colegn.2014.04.003
[13] Cooper, C., Taft, L., & Thelen, M. (2005). Preparing for Practice: Students’ Reflections on Their Final Clinical Experience. Journal of Professional Nursing, 21, 293-302.
http://dx.doi.org/10.1016/j.profnurs.2005.07.002
[14] Critical Appraisal Skills Programme (CASP) (2014). CASP Checklists.
http://www.casp-uk.net/#!casp-tools-checklists/c18f8
[15] Delany, C., & Watkin, D. (2009). A Study of Critical Reflection in Health Professional Education: “Learning Where Others Are Coming from”. Advances in Health Sciences Education, 14, 411-429.
http://dx.doi.org/10.1007/s10459-008-9128-0
[16] Dewey, J. (1933). How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process. Boston, MA: D.C. Heath & Co Publishers.
[17] Dirkx, J., & Mezirow, J. (2006). Musings and Reflections on the Meaning, Context, and Process of Transformative Learning: A Dialogue between John M. Dirkx and Jack Mezirow. Journal of Transformative Education, 4, 123-139.
http://dx.doi.org/10.1177/1541344606287503
[18] Donaghy, M., & Morss, K. (2000). Guided Reflection: A Framework to Facilitate and Assess Reflective Practice within the Discipline of Physiotherapy. Physiotherapy Theory and Practice, 16, 3-14.
http://dx.doi.org/10.1080/095939800307566
[19] Donaghy, M., & Morss, K. (2007). An Evaluation of a Framework for Facilitating and Assessing Physiotherapy Students’ Reflection on Practice. Physiotherapy Theory and Practice, 32, 83-94.
http://dx.doi.org/10.1080/09593980701211952
[20] Dyment, J. E., & O’Connell, T. S. (2011). Assessing the Quality of Reflection in Student Journals: A Review of the Research. Teaching in Higher Education, 16, 81-97.
http://dx.doi.org/10.1080/13562517.2010.507308
[21] Fisher, K. (2003). Demystifying Critical Reflection: Defining Criteria for Assessment. Higher Education Research & Development, 22, 313-325.
http://dx.doi.org/10.1080/0729436032000145167
[22] Howe, A., Barrett, A., & Leinster, S. (2009). How Medical Students Demonstrate Their Professionalism When Reflecting on Experience. Medical Education, 43, 942-951.
http://dx.doi.org/10.1111/j.1365-2923.2009.03456.x
[23] Hubbs, D., & Brand, C. F. (2010). Learning from the Inside Out: A Method for Analyzing Reflective Journals in the College Classroom. The Journal of Experiential Education, 33, 56-71.
http://dx.doi.org/10.5193/JEE33.1.56
[24] Johns, C. (1995). Framing Learning through Reflection within Carper’s Fundamental Ways of Knowing in Nursing. Journal of Advanced Nursing, 22, 226-234.
http://dx.doi.org/10.1046/j.1365-2648.1995.22020226.x
[25] Kok, J., & Chabeli, M. (2002). Reflective Journal Writing: How It Promotes Reflective Thinking in Clinical Nursing Education: A Students’ Perspective. Curationis, 25, 35-42.
[26] Leech, N., & Onwuegbuzie, A. (2007). An Array of Qualitative Analysis Tools: A Call for Data Analysis Triangulation. School Psychology Quarterly, 22, 557-584.
http://dx.doi.org/10.1037/1045-3830.22.4.557
[27] Lutz, G., Scheffer, C., Edelhaeuser, F., Tauschel, D., & Neumann, M. (2013). A Reflective Practice Intervention for Professional Development, Reduced Stress and Improved Patient Care—A Qualitative Developmental Evaluation. Patient Education and Counseling, 92, 337-345.
http://dx.doi.org/10.1016/j.pec.2013.03.020
[28] Mann, K., Gordon, J., & MacLeod, A. (2009). Reflection and Reflective Practice in Health Professions Education: A Systematic Review. Advances in Health Sciences Education, 14, 595-621.
http://dx.doi.org/10.1007/s10459-007-9090-2
[29] Medical Board of Australia (2014). Good Medical Practice.
http://www.medicalboard.gov.au/Codes-Guidelines-Policies.aspx
[30] Medical Radiation Practice Board of Australia (2014). Code of Conduct: Professional Values and Qualities.
[31] Mezirow, J. (1990). How Critical Reflection Foster Transformative Learning. In J. Mezirow (Ed.), Fostering Critical Reflection in Adulthood: A Guide to Transformative and Emancipatory Learning (pp. 1-20). San Francisco, CA: Jossy-Bass.
[32] Mezirow, J. (1991). Transformative Dimensions of Adult Learning. San Francisco, CA: Jossey-Bass Inc.
[33] Mezirow, J. (1995). Transformation Theory of Adult Learning. In M. Welton (Ed.), In Defense of the Lifeworld: Critical Perspectives on Adult Learning (pp. 37-90). New York: State University of New York Press.
[34] Mezirow, J. (1997). Transformative Learning: Theory to Practice. New Directions for Adult and Continuing Education, 1997, 5-12.
http://dx.doi.org/10.1002/ace.7401
[35] Mezirow, J., & Associates (1990). Fostering Critical Reflection in Adulthood: A Guide to Transformative and Emancipatory Learning. San Francisco, CA: Jossey-Bass.
[36] Mezirow, J., & Associates (2000). Learning as Transformation. San Francisco, CA: Jossey-Bass.
[37] Moon, J. (2003). Learning Journals and Logs, Reflective Diaries. Good Practice in Teaching and Learning, Dublin: University College Dublin.
[38] Moriarty, H., & McKinlay, E. (2008). Reflective Journals Reveal the Transformative Nature of Early Community-Based Experience. Focus on Health Professional Education: A Multi-Disciplinary Journal, 10, 59-70.
[39] Nicholl, H., & Higgins, A. (2004). Reflection in Preregistration Nursing Curricula. Journal of Advanced Nursing, 46, 578- 585.
http://dx.doi.org/10.1111/j.1365-2648.2004.03048.x
[40] Nielsen, A., Stragnell, S., & Jester, P. (2007). Guide for Reflection Using the Clinical Judgment Model. Journal of Nursing Education, 46, 513-516.
[41] Norrie, C., Hammond, J., D’Avray, L., Collington, V., & Fook, J. (2012). Doing It Differently? A Review of Literature on Teaching Reflective Practice across Health and Social Care Professions. Reflective Practice, 13, 565-578.
http://dx.doi.org/10.1080/14623943.2012.670628
[42] Nursing and Midwifery Board of Australia (2010). National Competency Standards for the Midwife. Melbourne: APHRA. www.nursingmidwiferyboard.gov.au/documents/default.aspx
[43] Owings Swan, K., & Hoffer, M. (2008). Technology and Social Studies: Self-Report. In L. Levstik, & C. Tyson (Eds.), Handbook of Research in Social Studies Education (pp. 307-328). Oxford: e-Library Taylor and Francis.
[44] Pitkälä, K., & Mäntyranta, T. (2004). Feelings Related to First Patient Experiences in Medical School: A Qualitative Study on Students’ Personal Portfolios. Patient Education and Counseling, 54, 171-177.
http://dx.doi.org/10.1016/S0738-3991(03)00209-X
[45] Polit, D., & Beck, C. (2008). Nursing Research: Generating and Assessing Evidence for Nursing Practice (9 ed.). Philadelphia, CA: Lippincott Williams and Wilkins.
[46] Ramli, A., Joseph, L., & Lee, S. (2013). Learning Pathways during Clinical Placement of Physiotherapy Students: A Malaysian Experience of Using Learning Contracts and Reflective Diaries. Journal of Educational Evaluation for Health Professions, 10, 6.
http://dx.doi.org/10.3352/jeehp.2013.10.6
[47] Rosenthal, S., Howard, B., Schlussel, Y. R., Herrigel, D., Smolarz, B. G., Gable, B. et al. (2011). Humanism at Heart: Preserving Empathy in Third Year Medical Students. Academic Medicine, 86, 350-358.
http://dx.doi.org/10.1097/ACM.0b013e318209897f
[48] Sandelowski, M., Barroso, J., & Voils, C. I. (2007). Using Qualitative Metasummary to Synthesize Qualitative and Quantitative Descriptive Findings. Research in Nursing & Health, 30, 99-111.
http://dx.doi.org/10.1002/nur.20176
[49] Sch?n, D. (1983). The Reflective Practitioner: How Professionals Think in Action. London: Temple Smith.
[50] Smith, M., & Trede, F. (2013). Reflective Practice in the Transition Phase from University Student to Novice Graduate: Implications for Teaching Reflective Practice. Higher Education Research & Development, 32, 632-645.
http://dx.doi.org/10.1080/07294360.2012.709226
[51] Sobral, D. T. (2005). Medical Students’ Mindset for Reflective Learning: A Revalidation Study of the Reflection-in-Learning Scale. Advances in Health Sciences Education, 10, 303-314.
http://dx.doi.org/10.1007/s10459-005-8239-0
[52] Stockhausen, L. (2005). Learning to Become a Nurse: Students’ Reflections on Their Clinical Experience. Australian Journal of Advanced Nursing, 22, 8-14.
[53] Tanner, C. A. (2006). Thinking Like a Nurse: A Research-Based Model of Clinical Judgment in Nursing. Journal of Nursing Education, 45, 204-211.
[54] The Balint Society (2012). The Balint Society.
http://balint.co.uk/
[55] Wessel, J., & Larin, H. (2006). Change in Reflections of Physiotherapy Students over Time in Clinical Placements. Learning in Health & Social Care, 5, 119-132.
http://dx.doi.org/10.1111/j.1473-6861.2006.00124.x
[56] Williams, R. M., Wesse, J., Gemus, M., & Foster-Seargean, E. (2002). Journal Writing to Promote Reflection by Physical Therapy Students during Clinical Placements. Physiotherapy Theory & Practice., 18, 5-15.
http://dx.doi.org/10.1080/095939802753570657

Copyright © 2024 by authors and Scientific Research Publishing Inc.

Creative Commons License

This work and the related PDF file are licensed under a Creative Commons Attribution 4.0 International License.