Reflection and Quality Assessment in Service-Learning Projects. When, With Whom, and Why

Introduction

Much has been written about the benefits of service-learning (SL) in classrooms at all educational levels, including university, without questioning at times whether it was really being applied according to the requirements that define this methodology. In many cases, proposals are similar to SL, but they are not actually SL, regardless of whether they may be appropriate pedagogical practices or not. It seems that the mere label of SL in a project may trigger an improvement for all involved and this is obviously far from the reality, especially when lacking, in many cases, a rigorous assessment design (Santos Rego et al., 2016).

In recent years, university campuses in Spain have witnessed a significant expansion of this methodology as part of the process of convergence and adaptation of the European Higher Education Area (Santos Rego et al., 2017). The implementation of SL projects in the university responds to a clear intention: the optimization of students' learning. Howard (1993, 2001) noted that this methodology could not be conceived as a means to improve students' academic learning exclusively, since the orientation toward both civic learning and academic education is its distinctive feature. The learning process is mediated, to a large extent, by the provision of a service to the community aimed at addressing community issues. SL has great potential to breathe life into the contents of the subjects in a curriculum, because community service is designed in connection with the curricular objectives (Lorenzo et al., 2019). However, like any educational practice, its impact is conditioned by a series of factors and variables that determine the effectiveness of the projects, as they become clear indicators of quality, and have a direct impact on the results to be achieved, especially on the students.

The definition of SL includes the basic principles that characterize this methodology:

“A pedagogical proposal that addresses the search for concrete formulas to engage the students in the daily life of the communities, neighborhoods, and nearby institutions. It is conceptualized within experience-based education and is characterized by: a) student protagonism; b) addressing a real need; c) connection to curricular objectives; d) execution of the service project and e) reflection” (Naval et al., 2011, p. 88).

This definition identifies the need for establishing a correct planning of all the elements, hence the importance of the design of the SL project. Imperial et al. (2007), after a thorough literature review, concluded that although there were repeated mentions of SL outcomes, little thought was given to how the quality of project design determined those outcomes. Conner and Erickson (2017) warned that when a project was poorly designed and implemented, unexpected results (possibly negative) may be obtained.

Puig et al. (2007) supported the relevance of differentiating the principles of quality from what would be basic requirements of a project. Furco and Norvell (2019) included both options by defining 11 essential elements of SL, grouped into three clusters, the last of which belonged to the critical components supporting learning and service, which comprised: the student's voice by selecting, designing, implementing, and assessing the project; diversity through its participants, practice, and outcomes; promoting communication, partnerships, and collaboration with the community; preparing students for the tasks they will develop (skills, understanding of tasks and roles); reflection; and using different methods to celebrate and validate students' service work.

In any case, quality results, in part, from optimal management of the basic requirements that define an initiative of this type. Therefore, we should not be surprised that one of the main critical SL-defining factors is the necessary link between service and learning, that is, the connection between the activities developed by students in the community and a subject curriculum (Imperial et al., 2007). Hatcher et al. (2004) found that students gave importance to the integration of academic content into the service experience as the most important variable in the quality of learning.

The study of the pedagogical components involved in the quality of the SL projects, understood mainly as the effectiveness of the results obtained by the students, is one of the questions that has raised most interest in research. Three decades ago, we came across what is arguably one of the first recognized classification of quality principles and good practice for combining academic learning and community service (Honnet and Poulsen, 1989):

- Engage students in responsible and challenging actions aimed at the common good.

- Provide structured opportunities for students to critically reflect on their experiences. Service alone does not guarantee learning, so one should find time to discuss and share their experience regarding relevant moral or theoretical issues. Howard (1993, 2001) pointed out that students should be prepared to learn from the community, through strategies such as observation and reflection.

- Both the students and the recipients of the service have to be aware, from the very beginning, of what they want to achieve and what they want to learn, so the objectives need to be defined through communication and contribute to increase the competences of all the parties involved.

- Allow the recipients of the service to define their needs. In this way, they must also participate in defining the activities to be carried out by the students, and how they will be developed.

- Clarify the responsibilities of all persons and entities involved. The different actors (students, teaching staff, and community) have to negotiate their role in the development of the project. It is especially important to make the entity responsible for student learning. In other words, the role of the teaching staff should be rethought, as a consequence of a more active role of students. It is no longer a question of transferring information, but of acting as a guide in students' learning process (Gargallo et al., 2018).

- Link students to service needs that are recognized as changing. There has to be a continuous feedback process, in which the changing nature of the service allows for the expansion of students' competences by having to adapt constantly to reality.

- Provide a genuine, active, and sustained organizational commitment. The quality of the projects will depend on the institutional commitment provided by the university, and also by the recipient entity.

- Include training, monitoring, recognition, and assessment to achieve service and learning objectives. It is a reciprocal responsibility of those providing the service and those receiving it, with particular emphasis on a formal and planned assessment that includes all participants. This process has to take into consideration the differences and diversity among subjects (Lorenzo and Belando-Montoro, 2019).

- Ensure that the time spent on service and learning is flexible, appropriate, and best suited to the interests of all involved.

- Promote the participation of diverse populations. A good SL project promotes access and removes barriers to participation. This diversity must be used to strengthen the objectives proposed in students' learning process and service.

Moreover, in an attempt to further clarify the variables to be taken into account before implementing an SL course, with the aim of achieving the desired learning outcomes, Howard (1993, 2001) added:

- The academic credit is not for the service or its quality, but for demonstrating students' academic and civic learning.

- Academic rigor should not be jeopardized, and the concept of Service-Learning as a “soft” learning resource should be rejected. Moreover, while in traditional initiatives students only have to meet academic learning objectives, in the SL they have to meet both academic and community service objectives.

- Establish learning objectives. The combination of learning and a service to the community turns it into a methodology that multiplies the possibilities of learning, so it must be correctly planned.

- When choosing the places where the service has to be carried out criteria should be narrowed, allowing the connection with the contents of the subject, and the relevance of the learning that students acquire.

- Academically proven strategies should be anticipated in order to assess learning in the community.

- The differences between students' roles in the community and in the classroom should be minimized. The role taken on in both environments should be similar, in an attempt of bringing the classroom (more passive) and community (more active) contexts closer together.

- The variations in students' learning outcomes should be prepared, as well as possible loss of control. Service-Learning implies heterogeneous academic results among students, even when they are exposed to the same situations, both in the community and in the classroom.

Batchelder and Root (1994) designed the Evaluation of Service-Learning (ESL), a guide to evaluate those aspects which, hypothetically, could affect service-learning outcomes. The factors they considered were the following:

- Students' autonomy and decision-making ability in service activities.

- Reflection in the classroom, so that all the work conducted on the subject could help to understand the experiences in the service setting and allows learning from them.

- The support provided by the teacher in charge who, as a guide, has to help the students in their process of adaptation to the experience and, obviously, to maximize its benefits.

- The clarity of the students' role in the project (as perceived by them).

- The relationship with the head of the community entity in which the service was developed, who must supplement the teacher's support.

- The students' perception of having contributed to really helping the recipients of the service.

- The students' perspective, regardless of how their experience has developed, on the potential of the project to help the recipients.

These classifications, although they might be seen as starting rules, lack empirical evidence supporting them. This research was aimed at identifying the variables that mediate the educational success of service-learning, in an attempt to examine which elements can be adopted as quality principles of this educational strategy. Thus, Mabry (1998) studied three dimensions which were associated with results achieved by students:

- The amount of time devoted to the service in the community mainly affected the academic dimension, stating that a reduced service did not provide enough content and material to be linked to the subject. A minimum of 15–19 h of service was recommended in order to enjoy contact with people and reality, so that some effectiveness of the project could be guaranteed.

- Students who were in constant contact with the recipients and interacted with them presented clear differences in terms of civic and social values, in addition to academic learning. This revealed that those students who did not have any contact with the recipients changed for worse, although these changes were insignificant. Conner and Erickson (2017) warned that implementing a service-learning project could actually be more negative than positive if students were involved in experiences that included casual contact with reality, i.e., short and superficial contact with the recipients.

- Reflection, both inside and outside the classroom, has positive effects when it is more continuous and regular, especially in terms of social and civic outcomes. By focusing on reflection in the classroom, a greater impact is achieved, especially on academic learning. In addition, the participation of all the actors is positive: students, teaching staff and people in charge of the collaborating entity.

Following the same line of research, Lambright and Lu (2009) highlighted the characteristics of the SL courses that were associated with self-reported student learning: the consistency of the mentoring provided by the teacher; the degree to which the project was linked to the contents of the subject; the amount of time devoted in class to reflection; the level at which students were able to influence the development of the project; the contact between students and recipients, i.e., the intensity and duration of the experience; and the presence of teamwork activities in the project.

Moely and Ilustre (2014) concluded that the variables on which the success of an SL project depended were related, on the one hand, to the quality itself and, on the other, to the orientation given to it (Morton, 1995; Ward and Wolf-Wendel, 2000; Moely et al., 2008). When they referred of quality as a dimension that were associated with the effects of an experience, they pointed out the following elements:

- The usefulness and value of the service for the students.

- The degree of importance attached to the SL project in the planning and development of the subject (so that students and partners could be prepared for the service, and so that it could be integrated into the subject and linked to the curriculum).

- Ensure opportunities for reflection.

Within this framework, an important variable on which the possibility of complying with quality criteria and factors will depend is teacher training and preparation. As proposed by Imperial et al. (2007), the commitment of the teaching staff is a determining factor in the success of SL, while institutional support, reflected in elements such as training or funding, is decisive in pushing teachers to implement this methodology and, especially, in determining how it is managed. Bringle and Hatcher (1995) noted that consciously planned teacher training was a predictor of future project quality. Thus, while not denying the benefits of teachers discovering SL on their own or accidentally, the authors suggested that planned, deliberate, and centralized professional development would lead to more and better results (Morton, 1996). In this regard, Lorenzo et al. (2019) conducted a study with 1903 teachers from six Spanish universities, analyzing the variables which explained why a university professor introduced this methodology in their teaching. They concluded that the professor's agreement with the social commitment of the university was the one that best predicted the use of SL, so that the probability of using this methodology was 3.52 times higher than not doing it.

The review of these classifications (Table 1) means that we have reflection as a focal point, not only in the design of a service-learning project, but also as the main quality criterion for this type of initiative, or at least the one about which there is the greatest consensus. Thus, the quantity and type of reflection determine the quality of the projects (Eyler, 2002).

Therefore, fostering effective reflection processes will be the key to strengthening and optimizing the potential of service-learning, i.e., “a service-learning project will be more useful and of higher quality as long as it inspires a real reflection process around it” (Santos Rego et al., 2015, p. 20). Solid reasons must be given during that process, as it is not just a superficial deliberation to make matters easy. This critical thinking must create strong links between learning and service.

Santos Rego (1992) admitted that there was no debate at the time of consenting to the opposite to the reflective action, that is, the automatic and almost dogmatic acceptance of principles, without a further analysis in this regard. Therefore, reflective action is structured as one of the elements on which the current educational change experienced at the university is based, leaving behind the passive and uncritical reception of information and pointing toward a pedagogical model focused on students' autonomous and active learning (Escofet and Rubio, 2017; Gargallo et al., 2018).

Saltmarsh (1996) stated that learning was generated through reflective thinking in problem solving. Without reflection as part of critical thinking, it would be difficult for students to move toward metacognitive empowerment. Dewey (1916, p. 169) defined reflection as “the discernment of the relation between what we try to do and what happens in consequence.” It is, therefore, the reflective thinking attached to the experiences that gives them meaning, by turning them into learning that will determine people's future activities and decisions. According to González-Geraldo et al. (2017, p. 69), “the educational side of the continuous reconstruction of experience is not in the experience itself, but in its cognitive re-elaboration.”

However, according to Maddux and Donnett (2015), Dewey's ideas for reflection had been imperfectly adapted to service-learning on many occasions. The authors argued that reflection implied advances and improvements in the way students understood their world and linked it to their own learning, and did not simply give satisfaction with the experience or sympathy with the people with whom they interacted. To this end, they proposed that the students' experience involved problematic situations, which would force them to discuss among themselves and with members of the community in order to explore possible solutions (Santos Rego, 1991, 1992). Understanding the consequences of such problems through reflection would therefore be the best way to enhance meaningful learning. According to Harkavy and Benson (1998), genuine learning occurs when the individual concentrates their efforts on solving problems and dilemmas by reflecting on their own experience, thus improving their ability to think and act in the future. The idea is that reflection should serve as an element around which to understand the consequences arising from one's own experiences (Dewey, 1916).

In this regard, in SL we should consider the critical reflection approach explicitly designed after years of research by Ash and Clayton (2009). This is the DEAL model that aims to get students to take responsibility for their own learning process through three sequential steps:

1. Describe the experiences in an objective and detailed manner. This process can occur before, during, and after the activity.

2. Examine these experiences in the light of specific learning, allowing students to make sense of their activity, identifying the connections between the learning objectives and their personal experience.

3. Articulate the learning process, including goals for future actions by improving their practice and learning. This enables students to capture their learning so that they can act on it. This is only possible if students can clearly articulate their learning process by answering questions such as “What did I learn?,” “How did I learn it?,” “Why does that matter?,” and “What will I do now?,” which will transform the students' experience into a substantive and applicable learning process.

Reflective thinking becomes one of the pillars of experiential educational practices, and in the case of SL, the element that makes it possible to establish a critical connection between service activities and the learning associated with them (Saltmarsh, 1996). Critical reflection is a crucial skill for students because it connects community service activities with academic learning (Deeley, 2015). The importance, in this sense, is justified by the supposition that community service does not produce learning by itself, it is reflection that establishes a link between that service and the contents of the subject (Bringle and Hatcher, 1999; Ash and Clayton, 2009). In other words, reflection lies in a set of activities that make possible a relationship of reciprocal influence between community service, on the one hand, and academic and civic learning, on the other (Howard, 2001). One could even state that this is an element that turns a service activity promoted from the academic field into service-learning and not simply an initiative parallel to the curriculum:

Reflection is the key element that connects service and learning. Without reflection, service is simply that: service. Reflection on service experiences allows students to make cognitive and affective connections to learning, create linkages to prior learning and experiences, and make new sense of the world (Bucco and Busch, 1996, p. 242).

In SL, reflection is proposed as a clear indicator of the quality of experiences, although it has to be constituted as a reflection organized in an intentional way by students and teachers, which differs from the timely and low profile reflection that usually is attached to all human actions (Páez and Puig, 2013). According to Billig (2007), it had to involve cognitively challenging activities for the students, which stimulated them to think in depth about a problem in order to deconstruct it and find possible solutions.

However, even when time and preparation are devoted to it, reflection can become superficial, merely collecting students' impressions and feelings, with little opportunity to build links between their experience and the subject or to challenge their own assumptions (Eyler, 2002). Billig (2007) also understood it this way, stating that many teachers who made use of SL did not introduce variations in the type of reflection activities they promoted, because they resorted mainly to written journals and summaries that simply reflected students' feelings while they were involved in community service.

It is advisable to pay attention to those elements that will determine whether a process of reflection is developed in an effective way. In this regard, Eyler et al. (1996) presented four principles that every reflective process had to follow to be considered of quality: continuous, connected, challenging, and contextualized:

- It is a continuous process, parallel to the entire educational process and service activity, so it has to be done before, during, and after the SL experience.

- It has to connect the service to the intellectual and academic goals of the students. The service proves and makes the theories real, while in class work students examine theoretical frameworks that explain the service.

- Challenging students to solve problems in a more critical way is a feature of effective reflection that can be more difficult for teaching staff.

- It has to fit every SL project. It has to be appropriate and complementary to the level of the other learning activities of the subject.

Bringle and Hatcher (1999) proposed that reflection activities should take into account the following five points: clearly link the service to the subject contents and learning objectives; be structured in terms of description, expectations, and criteria for assessment; be regular and continuous; provide feedback from the teacher so that students can improve their critical analysis; and include opportunities for students to explore, clarify, and alter their own values.

We believe that the pillars on which a quality reflection in SL projects should rest must be associated with the research supporting them. Hatcher et al. (2004) found in a study with 471 students, that the quality of learning was significantly associated with the integration of academic content into the service provided and by the nature of reflection, i.e., whether it is structured, regular and allowing for the clarification of learning values. This study confirmed much of what Bringle and Hatcher (1999) had proposed.

In a meta-analysis conducted by Conway et al. (2009), in which a sample of 103 different projects was included, found that the presence of structured reflection was associated with positive changes on personal and civic outcomes. Eyler (2002) presented the features of the reflection processes at these three moments:

Before the service or “pre-reflection”

Experiential education results in students being confronted with situations and information conflicting with their conception of the world, which is why appropriate preparation is needed beforehand. Before community service, time should be devoted to analyzing students' perceptions of the community, without hiding the problems and needs to be addressed, and in order to identify gaps in communication.

Another objective is to encourage awareness of their own learning process and to monitor it. Developing these metacognition skills can optimize procedures that favor problem analysis and decision making (Billig, 2007).

During the service

The aspect that determines an effective reflection during the service is its continuity, because the observations must be constant, challenging, and connected in terms of information (Eyler et al., 1996). In this sense, experiential learning takes place in a cyclical process of action and reflection on that action.

In addition, regular reflection throughout the service allows students to address the discrepancies arising between their previous assumptions and current frameworks. This idea is linked to Mezirow (1977) principles of transformational learning, as reflection helps transform the original schemes of perception, leading students to adopt new perspectives on problems. Thus, when students are immersed in a certain social situation, they realize that their old beliefs are not adequate to face such a situation. According to Deeley (2015), reflection can be an uncomfortable experience, as it confuses the way students understand and interpret reality, which can lead them to reject SL, taking them out of their traditional comfort zone.

At the end of the SL project

It is especially helpful if the students have been previously aware of their own learning process while they are engaged in ongoing reflection. This is a good time to consolidate learning, examine understanding of the subject, and identify problems that have not yet been solved.

Another element of great relevance, which also has an impact on the reflection quality, is the type of activities selected. In this sense, there is a great variety of reflection formats, which according to Bringle and Clayton (2012) can be written, oral, or both; individual, collaborative, or both; planned over time or informal and casual; they can involve feedback from numerous constituencies (teachers, peers, community entities); and can be conducted with a wide variety of tools and within numerous activities, such as journals, presentations to peers, or discussion sessions. However, Bringle and Hatcher (1999) saw writing as a special reflective procedure through which students can create new meanings and new understandings of problems, as well as new ways of organizing experiences. Writing allows for better management of the experience in its psychological components, encourages critical thinking and establishes links among previous, current and future experiences.

Eyler and Giles (1999), in their book Where's the learning in service-learning?, ended up answering the inquiry raised by stating that learning lies precisely in the questions that the situations of the service developed by the students inherently entail, and especially in the reflection which, guided by the teachers, is attached to these issues and allows for the connection between the knowledge that one already has and their new experiences.

In short, reflection, as critical reasoning on the whole of the experience and the establishment of links with the academic content of the subject, allows students to give an account of the way in which they have acquired different learning experiences, thus becoming an optimum mechanism for assessing those (Ash et al., 2005).

Contrary to what might be thought, research has shown that the effect of service-learning on students learning and development is not direct, it takes place through the transformations that can occur in certain intermediary variables: self-esteem, empowerment, prosocial behavior, motivation, and commitment (Furco, 2005). Rodríguez Gallego (2014) presented three general areas in which SL has been shown to have positive effects, and which are closely related to the variables presented by Furco (2005) as mediators in students' improvement:

- Curriculum dimension. It encourages greater mastery of the subject matter, as well as more positive attitudes toward learning and work.

- Personal development. It prepares students to take responsibility, increases the feeling of self-esteem and self-efficacy, increases teamwork skills and ability to overcome adversity, and leads to improvements in motivation, creativity, and communication skills.

- Social development. It encourages prosociality, a positive attitude toward diversity, and contexts of interaction.

Simonet (2008) also argued that this methodology indirectly affected the academic success of university students, through its influence on another set of variables: increased relationship with faculty, increased participation in campus and university activities, greater satisfaction with the campus environment and, therefore, an increase in active learning.

Out of these mediating variables, studies pointed out motivation and self-efficacy. First, in terms of motivation, Alonso et al. (2013) found in their study that improvements in this dimension were the result of a process in which students learned in an active and practical way, taking a leading role. As instructed by Zayas et al. (2019), SL tackles the lack of motivation of students in such a way that it allows them to overcome the distance between reality and university classrooms.

If we focus on the possibilities of this methodology in order to influence the feeling of self-efficacy and self-concept, it is fair to address the opportunities it offers so that students could feel that they can have a positive impact on their community, making a difference if they try hard enough (Morgan and Streb, 2001). One must understand, therefore, the development of self-efficacy as a result of meaningful work and interaction with peers and the community (Song et al., 2017).

The data obtained in the study conducted by Chiva-Bartoll et al. (2018) confirmed this line of research. They found that students who enrolled in service-learning initiatives demonstrated higher levels of development in problem-solving self-efficacy (effective decision-making and coping with challenges) and academic self-realization (motivation, expectation, and attribution of academic performance) than their peers who did not.

In addition, Brozmanová et al. (2016) studied the development of key competences in a group of 33 students who incorporated service-learning strategies at the University of Matej Bel (Slovakia), thereby confirming that these strategies were associated with the development of these competences (Sevin et al., 2016).

In short, the literature confirms reflection as a prerequisite for defining SL. However, as Ash and Clayton (2004) admitted, it is still difficult to put it into practice, despite the pedagogical virtues of the DEAL model (Ash and Clayton, 2009). For this reason, our objective in this work is to provide evidence of how reflection should be carried out to maximize the development of certain mediating variables that affect the improvement of the academic performance of university students (Furco, 2005). Specifically, we are going to analyze when making a reflection, who should participate in this process and what the reflection is for.

Materials and Methods

Participants

The research involved 295 students (intentional non-probability sampling) who participated during the 2016–2017 academic year in six SL projects belonging to the degree programs of Optics and Optometry, Early Childhood Education, Primary Education, Pedagogy, Veterinary Medicine, and Forestry and Environmental Engineering of the University of Santiago de Compostela. Each project was developed over a 4-month period. The highest number of students were from the area of Social and Legal Sciences (60.7%), followed by Health Sciences (37.3%) and Engineering and Architecture (2.0%). 23.4% were men and 76.6% were women, with a M_age = 21.39 and SD = 3.18. They were mainly enrolled in the first (49.2%) or the third year (35.6%) of their university degree¹.

Instrument

We used three instruments, two for the students and one for the professors responsible for the project (Santos Rego and Lorenzo, 2018).

Firstly, the students were given four Likert-type scales, already validated in a previous research study (Mella, 2019):

(a) University training: to understand students' motivation and perception of the training they received at university. With seven items (with five options ranging from strongly disagree to strongly agree), this scale generated three factors (Link between the subjects and real life, Satisfaction with the training and Uncertainty about the future), which explained 66.11% of the variance and showed a good internal consistency (α values ranging from 0.50 to 0.79);

(b) Social participation: made up of five items that assess the frequency of students' social involvement (never, annually, quarterly, monthly, weekly), based on their participation in civic matters, both within the university and outside it. It yielded a single factor, thereby explaining 42.39% of the variance with α = 0.60.

(c) Civic and social competences: to study the degree to which students consider that they possess competences related to civic and social matters. The 20 items of this scale with five options (ranging from strongly disagree to strongly agree) yielded four factors (Prosocial behavior, Leadership, Intercultural competence, and Teamwork and relationship with others), which explained 50.96% of the variance, with the α values varying between 0.70 and 0.79;

(d) Self-efficacy: there are 14 items that refer to the degree of self-efficacy perceived both in their daily life and in the academic context (with the same response options as the previous one). These items are grouped into two factors (Academic self-efficacy and General self-efficacy) which explained 54.03% of the variance and showed an internal consistency of α = 0.88 and 0.82, respectively.

Second, 165 of the participating students also filled out an Osgood scale, a bipolar scale made up of antonyms, to find out their level of satisfaction with different elements that define an SL project: the project in general, the activities in relation to the subject, the service provided, the lessons learned, and the involvement of the professor, the community partner and the student in the project. More specifically, in this research, we only used the item referring to the level of satisfaction with their learning, made up of the items usefulness (useless-useful), satisfaction (dissatisfaction-satisfaction), quantity (few-many) and applicability (inapplicable-applicable). Five response options were established (from 1 to 5), with one being the most negative, and five the most positive.

Thirdly, the professors responsible for each of the projects filled out a Registration sheet of SL projects/experiences at university level designed by Villa (2013) and which was validated by a group of experts in SL. It included a total of 28 questions grouped in the following modules: data from the university, identification of the project/experience, partners, assessment, and dissemination of the project/experience. Three reflection questions were included in the project/experience identification section. Specifically, the structure of reflection by the students (before, during, and after), with whom the reflection is carried out (community, collaborating entity, course and professor, SL work group), and orientation of the reflection (sharing feelings, relating the service to the contents of the subject, analyzing the problems of the community, and developing attitudes and values).

Procedure

The instruments were administered during the 2016–2017 academic year, at the end of each of the SL projects, with the permission of the teachers responsible for the projects and the consent of the students. The administration of the scales was carried out collectively in the classroom, coordinated by a member of the research team especially trained for this task, and within the framework of a broader data collection for an educational research project. In some projects, the Osgood scale could not be administered at the same time, which resulted in the loss of those students in the sample. The professors' sheet was individually filled out by each of the project coordinators

At the beginning of the study, the governing team of the University of Santiago de Compostela was also informed of the aim of the research project and of the procedure which was to be followed. In applying the instrument, we followed the recommendations of the University of Compostela's Bioethics Committee and of the Helsinki Declaration, complying with the stipulations which, in the case of Spain, are laid down in the Organic Law 3/2018, of December 5, on Data Protection and Guarantee of Digital Rights.

Data Analysis

We used the IBM-SPSS statistical package, version 24, which allowed us to organize the data, as well as its subsequent analysis. Given that the sizes of the groups were not balanced in the analyzed variables, we first checked the assumptions of independence of the observations, as well as the homoscedasticity of the variance. Both assumptions were fulfilled, thus parametric analyses, specifically one-way analyses of variance (ANOVA) with post-hoc Scheffé tests, were performed to study differences between groups.

Three matters relating to the reflection were taken as independent variables: time, participants, and their goal. As dependent variables, we took into account the mediating variables of educational performance that were reflected in the factors of the Likert scales, as well as the variable related to satisfaction with learning.

Results

Time for Reflection

Reflection could have occurred at different times during the development of the project: before starting the service (T1); at the end (T2); during and after (T3); before, during, and after (T4). No project included only reflection during its process.

When a reflection was made had no relationship to the association that the student makes between the subjects studied and real life, but it was associated with the other two factors of the first scale (Table 2). The best results in satisfaction with university training occurred when reflection took place before the start of the project, at the end of the service or when the three times (before, during, and after) are combined. The lowest incidence in this variable occurred when this process took place during and after. The statistically significant differences occurred when comparing the reflection during and after with any of the other options. The time for reflection also was associated with the perception of uncertainty. Specifically, the students who reflected during and after or at all three times were those who reported a positive attitude toward facing the future. At the opposite end, there were those who only reflected on the meaning of what they were going to do (before), or those who made the reflection only at the end of the project. In any case, significant differences only appeared when comparing T1 and T3 or T4. Therefore, students' opinion of the university training is better when, in the context of SL projects, the reflection process is continuous.

The social participation variable obtained considerably low mean values in all cases, thereby indicating the low participation of students in different formal initiatives of the same university, even if the literature sheds some light on this link with SL. In any case, the best indicators were obtained when the reflection was made throughout the entire project compared to when it occurred only at the end.

In the scale of civic and social competences, there are differences regarding its four factors. When reflection occurs during the entire process, a higher score on prosocial behavior and intercultural competence is obtained. Significant differences were observed when comparing this option with reflection only before (only on prosocial behavior) or after the process. Making the reflection during and after the project turned out to be the category for which students reported a higher score in teamwork.

In leadership, the highest scores were found when reflection was made before or during the entire process, but there were significant differences when comparing the latter with the reflection made only at the end. Time for reflection seems to have had no relation to self-efficacy.

In short, the moment of reflection was best associated with the mediating variables when it was made throughout the entire project (before, during, and after). In the same way, it seems that making a reflection only after the project ends has significantly less effect on the studied variables.

For satisfaction with learning, significant differences were obtained on the items of usefulness, satisfaction, and quantity (Table 3).

The greatest sensation of usefulness with the learning acquired occurred when the reflection took place at all three times, a statistically significant difference when compared to making the reflection only afterwards. The same conclusions can be drawn regarding satisfaction with learning and quantity perception.

Therefore, the assessment of the self-reported qualities of learning were better when the student was introduced to a process of reflection from the beginning to the end of the SL project.

Participants in the Reflection

Another variable that can condition reflection and, therefore, affects the results of SL are the actors participating in this process: the entire classroom group and teacher (P1); only the SL work group (P2); community partner, and classroom group and teacher (P3); and community partner, classroom group and teacher, and SL work group (P4). In some of the projects, not all the students enrolled in the subject participated in SL, which led to the identification of the SL work group, which includes only those who did participate.

With whom a reflection is made seems to have had no relationship to the association that the student made between the subjects studied and real life, but it was related to the other two factors of the first scale. With respect to satisfaction with the training the students who presented the best scores were those whose projects incorporated reflection with the community partner, and classroom group and teacher. The students who reflected with the greatest number of actors involved were those who reported a positive attitude toward facing the future (Table 4).

On the civic and social competence scale, leadership was not related to participants in the reflection process. For the rest of the factors, the university students who had the opportunity to reflect with all the participants in the process had a clear advantage, followed by those who did so with the SL work group only. Statistically significant differences were found when comparing P3 and P2 or P4.

Finally, taking into account all the actors involved in reflection (P4) implies obtaining statistically significant differences against not considering the working group (P3).

The reflection should certainly be made either in the SL project work group or with all the participants in the process.

In terms of learning, significant differences in the items related to usefulness, satisfaction, quantity, and applicability were found (Table 5).

Just as in the civic and social competence scale, a clear advantage can be observed in making the reflection with all those involved in the project. This gave students the feeling that the knowledge obtained is more useful, satisfactory, abundant, and applicable, unlike those projects whose reflection process involves fewer actors.

Goal of the Reflection

The last of the variables considered that may be related to the reflection process and, therefore, affect the result of SL is the goal or goals of the reflection sessions: relating the service to the contents of the subject (G1); sharing feelings about the service experience and relating it to the contents of the subject (G2); sharing feelings about the service experience and developing attitudes and values (G3); sharing feelings about the experience, relating service to contents and developing attitudes and values (G4); and all of the above and analyzing community problems (G5).

The first thing to take into account is that the goal which is most valued in all cases is that reflection is an opportunity to share feelings about the service performed, but also to seek the connection with the contents of the subject or course.

In line with the above ideas, once again, reflection does not seem to have any relationship to the association that the student makes between the subjects studied and real life, but it is associated with the rest of the factors of the first scale (Table 6). A reflection oriented toward sharing feelings, relating the service to the contents and developing attitudes and values (G4) was associated with a reduced perception of uncertainty about the future and higher levels of social participation.

Within the social and civic competence scale, option G4 turned out to be the category for which students obtained a higher score, both in prosocial behavior, intercultural competence, and better teamwork. The G5 option, which includes all the possible options, acts to the detriment of the factor scores on this scale.

Undoubtedly, the best option is when reflection was oriented to share sensations about the experience, service is related to the contents, and attitudes-values are developed. Secondly, the results showed that even when removing the option of relating the service to the contents from the previous one, good results were equally obtained.

In the case of the reflection orientation, the learning variables which showed differences were usefulness, satisfaction, and quantity (Table 7).

As shown in the analysis of the scales, one can observe how, in case of focusing it on all the options except for the analysis of community problems, students experienced a greater sense of usefulness, satisfaction, and abundance with respect to the learning acquired. Significant differences appeared when compared to considering all four options simultaneously.

The option of analyzing the community problems did not introduce any element of improvement. This may be due the lack of criteria for defining the goal of the reflection, or to the fact that it is a goal that should be closely linked to the initial reflection before carrying out any type of service, which, as shown, has little impact on the variables studied.

Discussion

A quality reflection is the real challenge for service-learning. This is due, according to Ash et al. (2005), to the difficulty of developing effective structures to guide that process, and meaningful strategies to assess the associated learning outcomes.

We agree with Sturgill and Motley (2014) that reflection is a key component of service-learning, but research showed that in order to maximize learning, the reflection must be of high quality. This article shed light on three variables that university professors should consider in this process: when, with whom, and why the reflection should be used.

These results have shown that not all reflections have the same value or the same effect on students. If the purpose is a positive impact on learning as well as the perception of the training they receive, the social participation, the development of civic and social competences, and the perception of self-efficacy, reflection should have its place, its protagonists and its goals. Ash and Clayton (2004) concluded that a model of reflection that pushed students beyond the superficial interpretations of complex issues and should facilitate academic proficiency, personal growth, civic engagement, critical thinking, and the meaningful demonstration of learning.

Our results coincide with those found by Eyler and Giles (1999), who concluded that the more rigorous was the reflection in SL, the better was learning in general and the academic results in particular: deeper understanding of a subject, better analysis and problem solving, openness to new ideas, and critical thinking skills.

Thus, we found that the assessment of learning and the impact on the mediating variables were higher when the student was immersed in a process of reflection from the beginning to the end of the SL project. In other words, reflection must be a continuous process, parallel to the entire educational process and the service activity (Eyler et al., 1996; Hatcher et al., 2004). Choo et al. (2019) have recently found evidence that more frequent reflection was associated with better connection of the project to academic content and vocational training, but they observed no effect on civic development.

Sturgill and Motley (2014) compared three different models of reflection on students' thinking processes. Specifically, they compared the students' reflections across axes of guided vs. free response, dialogic vs. expressive reflection, and public vs. private reflection. The results indicated that dialogic and guided reflection produced greater integration of knowledge of learning activities within the service.

In addition, reflection benefits from engaging the SL project work group or all those involved in the process. This may be due to the importance of group work in this type of dynamic. Chavez-Yenter et al. (2015) confirmed this, through a study that analyzed the connection between the development of competences and attitudes and the team dynamics in service-learning. The researchers identified team dynamics as a factor associated with civic attitudes and skills developed through a SL experience.

Finally, one of the best options is when reflection is oriented toward sharing feelings about the experience, relating the service to the contents of the subject, and developing attitudes and values. In addition, the data showed that even when removing this option of relating the service to the contents from the previous one, good results were obtained. Bringle and Hatcher (1999) also referred to the fact that reflection activities should clearly link the service to the subject content and learning objectives. In any case, Hatcher et al. (2004) stated that the regular reflection was likely to help develop an educationally more meaningful reflection. Furco and Norvell (2019) stated that reflection was a central element in the design and fulfillment of curricular objectives that should take place before, during, and after the service, and used multiple methods to promote critical thinking. However, reflection is a learned skill, which improves with practice (Ash and Clayton, 2004).

Following the same research line, Goff et al. (2020) identified three categories for understanding the impact of SL on students that were linked to the work of Eyler et al. (1996): first, the service enables the creation of a learning laboratory; second, there are challenges that young people face; and third, the service encourages the creation of meaningful connections.

In short, our work confirms that the nature of reflection is a central element when it comes to the quality of SL projects. That reflection, understood as reasoning about the entire experience, must be defined and structured in order to maximize the cognitive and non-cognitive learning options in students. For this reason, teaching staff must be aware that this should be a continuous process, thus they should take time for reflection during its development, they must involve all actors, from the university to the community, and they must have a clear objective of establishing links between service and academic content of the subject, otherwise this could lead to a volunteering action. Reflection in SL should not be improvised or left to chance.

For this reason, the connection between evaluation and critical reflection activities is also important (Ash and Clayton, 2009). The assessment, like the reflection questions, must be explicitly linked to the learning goals. Hence, these authors suggested the need for the assessment to incorporate the “Describe” and “Examine” aspects of the DEAL model. The summative assessments would evaluate the level of students' achievement of the learning objectives, while formative assessments would provide the necessary feedback for students to reflect on their learning. In addition, the students' responses should also be used as an opportunity for teachers to learn from the process in order to improve the project and maximize their students' learning.

In order to make this possible, SL should be institutionalized in our universities, which must be translated into the support, training and preparation of the teaching staff in order to develop quality SL projects (Santos Rego and Lorenzo, 2018). Meijs et al. (2019) proposed a guide for institutionalization based on three pillars: National and institutional prioritization, institutional support, and cooperation. Precisely, in terms of institutional support, these researchers included allocating resources for staff to learn and adopt this methodology, but also assigning and establishing budgets and financial incentives for SL development or even other important incentives for teachers.

Finally, we should note that the teachers responsible for each SL project were the ones who can reported on reflection through an Experience Sheet. We believe that further research should also include the perspective of the participating students, which would allow triangulation of the perspectives and make the results more rigorous.

Data Availability Statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics Statement

The studies involving human participants were reviewed and approved by Bioethics Committee of the University of Santiago de Compostela (Spain). The patients/participants provided their written informed consent to participate in this study.

Author Contributions

ML and DS-G collected and analyzed data and wrote the paper. MF and CV analyzed data and wrote and reviewed the paper. All authors contributed to the article and approved the submitted version.

Funding

This work was derived from two projects of the Spanish State Program of Research, Development and Innovation oriented toward the Challenges of Society (EDU2013-41687-R and EDU2017-82629-R) and a Research Network of the Spanish State Program for Knowledge Generation and Scientific and Technological Strengthening of the R&D&I System (EDU2017-90651-REDT)—http://www.usc.es/apsuni/.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnotes

References