Professional Development in Mathematics Education—Evaluation of a MOOC on Outdoor Mathematics

Abstract

In this paper, we examine the impact of a massive open online course (MOOC) in the context of outdoor mathematics on the participating teachers’ professional development. We firstly introduce the theoretical background on outdoor mathematics, focusing on math trails with the digital tool MathCityMap and professional development to be accomplished using MOOCs. By taking into account the MOOC “Task Design for Math Trails”, with 93 finalists, we analyze the learning progress of 19 selected case studies from different nations and learning levels by taking into account their answers in a pre- and post-questionnaire and their posts on a specific communication message board, with a special focus on the MOOC’s topics’ task design for outdoor mathematics and the digital tool MathCityMap. The analysis is performed using different quantitative and qualitative approaches. The results show that the teachers studied have benefited from professional development, which is evident in the expansion/evolution of their knowledge from a content, pedagogical, and technological perspective. Finally, we formulate consequences for professional development in STEM education, and conclude the paper with limitations to be drawn and a perspective for further research.

1. Introduction

The Erasmus+ strategic partnership “Math Trails in School, Curriculum, and Educational Environments of Europe” (MaSCE³; 2019–2022: http://masce.eu/ (accessed on 1 October 2021)) focuses on the idea of outdoor mathematics, in particular, math trails. By enriching an old idea (see, for example, [1]) through the digital tool “MathCityMap”, the MaSCE³ project aims at promoting math trails in an educational setting. In order to facilitate the preparation, conduct, and evaluation of math trails, seven partners from Portugal, Spain, Italy, France, Estonia, and Germany cooperate in the realization of different outputs (see Figure 1).

In this paper, we focus on the massive open online course (MOOC) elaborated as a MaSCE³ output, which aims at educating teachers to design tasks for outdoor mathematics teaching and learning. The MOOC, delivered from March to May 2021, gathered a massive number of mathematics teachers, around 500, from all over the world, among which almost 100 teachers completed the course. It is our aim, in this paper, to examine the learning outcomes and the professional developments of these finalist teachers. To do so, we will first introduce the idea of outdoor mathematics with a focus on math trails and, in particular, the MathCityMap system. Afterward, the focus will be on the professional development in mathematics teaching and on the potential role of MOOCs as a suitable format for this development. After presenting the research questions, the MaSCE³ MOOC itself is described as the context of the current study. Then, we will outline some clues on the methodology of the study, the analysis of the selected participants, and the considered data. Finally, the research questions will be answered by taking into consideration quantitative and qualitative analysis tools. The paper ends with a discussion of the obtained results, their relevance, and limitations.

2. Theoretical Background

2.1. Outdoor Mathematics

Teaching mathematics does not have to be limited to the well-known classroom and textbook setting. Following the Experiential Learning Theory [2], the possibility of concrete experiences and their reflections form the basis for learning processes. [3] use the term out-of-class-experiences. In addition, an embodied perspective on learning, describing that mathematical concepts can (only) be understood through activities with an appropriate object or situation [4], underlines the potential of leaving the classroom for discovering and applying mathematics in the real world.

One approach to mathematics outdoors is mathematics trails (referred to as math trails). A math trail describes a route with mathematical tasks [5]. The tasks are related to real objects or situations and can only be solved at a specific location and through mathematical activities [6]. The first math trails followed primarily the goal to popularize mathematics [1]. The math trail approach is also transferred to the educational context and used in school contexts. One project that focuses on math trails in the school context is the MathCityMap project. It has been developed at Goethe University Frankfurt in cooperation with international partners since 2012. The context of this paper is the Erasmus+ project MaSCE³ that aims at the further development, didactical enrichment, and dissemination of MathCityMap.

MathCityMap is a two-component system including a web portal and a smartphone app (see Figure 2). The web portal is an international database of tasks and math trails. In September 2021, while this paper was being written, there were more than 27,000 tasks by more than 8800 authors in 12 different languages, available in more than 40 countries. Teachers can either use this material or create their own tasks and trails that are individualized and suitable for their students. The tasks are related to real existing objects. Therefore, they are linked to the object’s location and have a task picture. In addition, the author formulates the actual task, up to three hints, and a sample solution. To check the solution, the system provides different answering formats: exact value, multiple-choice, interval, set, vector, fill-in-the-blanks, and GPS (for more information, see https://mathcitymap.eu/en/the-mathcitymap-task-formats-2/ (accessed on 1 October 2021)). Through the combination of different tasks, the authors create a math trail for their students.

The students—typically in groups of three—go on math trails with the corresponding smartphone app. The app guides and supports the students along the route of a math trail through a map view with their own location and the tasks’ locations, the task picture, task formulation, and optional hints that the teacher provides in the web portal. In addition, the students can enter their solution in the app and it is automatically validated in accordance with the beforehand chosen answering format and the quality of the students’ solution. The app includes different gamification options, e.g., points, a leaderboard, and a narrative story in which the learners take over the role of a pirate that is accompanied and supported with hints and motivations by his parrot Perry (for more information, see https://mathcitymap.eu/en/update-narratives/ (accessed on 1 October 2021)).

In the interface of the web portal and the smartphone app, the “Digital Classroom” allows teachers to follow their students’ progress on the math trail in real time. They can see what students enter as a response and the location of the students. In the case of problems, either the students or the teacher can start a conversation via chat that allows them to send text, audio, and photos. As the system does not store any personal data, it conforms to GDPR.

The math trail approach—in the sense of a digital enrichment through MathCityMap—supports mathematics teachers in their teaching of mathematics. It creates a frame in which students can learn mathematics in an authentic way. In addition, the students use a digital tool, their smartphone. This combination seems to have several advantages. Studies show in particular that students tend to remember the mathematical contents longer [7] and that they are motivated while solving math trail tasks [8].

Despite these advantages, our experiences show that math trails are often solely used for excursion days and that teachers have concerns about using outdoor mathematics in their teaching [9]. For example, teachers raise concerns about the supervision of their students and the time needed to prepare a math trail. In addition, they mention the lack of familiarity with the system. This last issue points out the need for different professional development actions aiming to make teachers more confident with outdoor mathematics and, in particular, with math trails through MathCityMap.

2.2. Professional Development in Mathematics Education

Professional development (PD) activities in mathematics education aim at increasing the teachers’ knowledge of mathematics [10] and at using new approaches and methods in their teaching [11]. Ref [12] formulates the following goals of PD in mathematics education: “(a) a shared vision for mathematics teaching and learning, (b) a sound understanding of mathematics for the level taught, (c) an understanding of how students learn mathematics, (d) a deep pedagogical content knowledge, (e) an understanding of the role of equity in school mathematics, and (f) a sense of self as a mathematics teacher” [13] (p. 87). With a special focus on PD of technology, the following factors seem to be relevant for increasing acceptance/integration of technology in teachers’ teaching practices: “(1) the perceived ease of use of technology, (2) the perceived usefulness of technology, (3) the attitudes toward the use of technology, and (4) the frequency of use of technology. If teachers perceive that the technology is easy to use and effective in teaching math concepts, this perception can lead to positive attitudes toward the use of that technology” [14] (p. 55).

The content and formats of PD activities cover a wide range: “Content may consist of the mathematics topics covered, a focus on student learning of particular topics, or a focus on mathematics curriculum. Format describes how opportunities for learning are organized and presented. It includes number of contact hours, span, location, type of contact (in person, distance learning, mixed), the activities carried out, and the artifacts used” [13] (p. 87). However, limiting oneself only to content and formats does not give an indication of what impact a training course has had on teachers’ PD.

As an example of a framework to measure the impact of PD, TPACK focuses on different aspects of pedagogical content knowledge and adds a technological domain. Figure 3 shows the three different domains of teacher’s knowledge involved. Content knowledge is related to mathematics and includes the topics that are taught. Pedagogical knowledge summarizes the knowledge about practices and methods of teaching and learning. Technological knowledge focuses on the knowledge of technological tools and how they can be integrated to convey the content [15]. The core TPACK, the intersection of all three components, is the technological pedagogical content knowledge.

With this framework, it is possible to specify the goals and impact of PD activities in mathematics education. Still, it remains a challenge to design PD initiatives, especially in the context of international projects such as the MathCityMap project.

In fact, there are several requirements that must be taken into consideration in this particular case; on the one hand, the fact that the project allows a personalized use of the system, being adaptive for teachers and their students. On the other hand, it is the aim of the MaSCE³ project to reach as many teachers as possible and on an international level. For both reasons, the focus will in the following be on a specific format of PD, namely the MOOC.

2.3. MOOCs in Mathematics Education

MOOCs are courses offered openly to anyone with Internet access, and without having to meet any course prerequisites, anyone can register and complete the course [16]. Thus, MOOCs can be accessed globally and can accommodate a large number of learners through the web [17], appearing as dynamic and diversified learning spaces with varying factors, such as flexible time frames, a massive number of participants from different demographics areas, motivation to continue learning, and opportunities for designers to implement novel pedagogies [18]. The activities provided in these courses range from watching certain videos, posting on forums or blogs, sharing experiences on social media, responding to quizzes, doing learning tasks for individuals or workgroups, and/or conducting peer review activities [19]. Learners are involved to various degrees; many just want to check out the resources and the new educational model, while others are really motivated and follow every aspect of the course, often interacting with other MOOC participants [20]. Likewise, instructors’ involvement varies substantially: in some courses, the educators disappear when the course starts; in others, they are intensively involved, injecting dynamism to the proposed activities and providing their students with feedback [21]. It is worth noting that the emergence and use of MOOCs for professional teacher development is still uncommon, especially in mathematics. In fact, although there is a wide choice of many different topics, when looking specifically for a MOOC aimed at mathematics teacher education, the range is limited [22]. Nevertheless, there is a growing interest in MOOCs involving mathematics teachers as participants, and MOOCs for teacher education are on the verge of gaining a foothold.

The theoretical framework MOOC-MDT [22] aims at understanding the complexities of the learning trajectories of the participants in a MOOC for teacher education. A learning trajectory, according to [23], studies how these participants interact online, both with the platform and with each other. In particular, the term addresses if and how these interactions change participants’ knowledge and beliefs and generate the perception of change in their practices, namely PD. Here, we briefly recall the salient points of this framework.

A MOOC is a very dynamic and complex environment, in which different actors alternate with each other. Indeed, in the design phase, the MOOC is ‘inhabited’ by the instructors who propose the materials and resources to be included in it. Once ready, it is still in an inert state; we can consider it as an artifact [24], i.e., a static set of materials. Connectivism [25] allows us to consider that the MOO-artifact has its own network of knowledge, the nodes of which are the resources (e.g., content, ideas, images, and videos) present in the various modules of the MOOC, and the connections are the links between pairs of nodes. When a MOOC module is activated, i.e., when access is allowed to the learners, it dynamically creates a complex structure that [26] call an ecosystem, meaning “all the connections (exchange of materials, experiences, and personal ideas/points of view) put in place by participants of an online community thanks to the technological tools through which they interact with each other, establishing connections within the given context” (p. 2481). The MOOC-ecosystem’s network of knowledge is dynamic; it evolves as a MOOC-artifact’s network thanks to the contributions of the learners. Moreover, the learners’ network of knowledge evolves as the personal self-organization of the ecosystem [23]. This process of dual evolution, both of the MOOC’s network of knowledge and of the learners’ network of knowledge, is named by [23] as a double learning process, and it has the following components, intertwined and self-feeding each other (Figure 4):

i. instrumentation/self-organization (from the ecosystem to the individual = E → I): process by which the MOOC-ecosystem’s network of knowledge expands the individual’s network of knowledge. In particular, the instrumentation is the phase by which the chaos (The complexity and multiplicity of connections can easily be perceived as chaos, information overload, in which it is difficult to find meaning of coherence in information. [25] talks about chaos as “a cryptic form of order” (p. 4). Moreover, [25] (p. 4] states: “Unlike constructivism, which states that learners attempt to foster understanding by meaning making tasks, chaos states that the meaning exists—the learner’s challenge is to recognize the patterns which appear to be hidden”. Chaos becomes a new reality in the people’s online learning process) of the ecosystem network reaches the individual. The many novelties of views and experiences ensure that the individual compares herself with new utilization schemes. A phase of self-organization of the MOOC information follows; namely, when the individual selects which resources proposed by the MOOC are valuable and which are not.

ii. instrumentalization/sharing (from the individual to the ecosystem = E ← I): process by which the individual’s network of knowledge expands the MOOC-ecosystem’s network of knowledge. The instrumentalization is the phase by which the individual, with her renewed network of knowledge, independently builds new connections. The individual is stimulated by a task requested by the MOOC and she caters to the ecosystem to turn it according to her own (new) connections. The individual wants to integrate it with her own cognitive structures. Sharing is the phase by which the MOOC welcomes the contribution of the individual and makes it available to all: information goes towards (is available to) all members.

As the number of MOOC participants is massive, the process is iterated: a phase of sharing is followed by a new instrumentation; a self-organization by an instrumentalization. It should be emphasized that the two processes are ‘intertwined’; that is, there is no moment in which one ends and another begins. This theoretical framework will be used, in this paper, to analyze the PD of MaSCE³ MOOC participants. Consequently, both the theoretical background concerning outdoor mathematics and the MOOC will be relevant for the creation of the methodology of this study as well as the analysis of the data gathered. In order to carry out finer analysis, another methodological instrument will be used, the reflective narratives or discourse analysis [27] that will be introduced in the research methodology section.

3. Research Question

The theoretical considerations have shown the relevance of MOOCs in mathematics teachers’ PD. Especially, the idea of the MOOC-ecosystem with a dynamic network of knowledge in line with TPACK seems to be beneficial for the teachers’ learning outcomes. Nevertheless, it is the individual design and conduct of a MOOC that might influence the learning outcomes and effects of the participating teachers’ PD. In the following, it is our aim to examine the MaSCE³ MOOC’s impact on the professional development of the participating teachers with a special focus on the MOOC’s topics’ task design for outdoor mathematics and the digital tool MathCityMap. In order to reach this research aim, two questions are formulated:

(i) 

Within the MaSCE³ MOOC-ecosystem, did the MOOC participants benefit from a double learning process?

(ii) 

Did the participants benefit from an expansion/evolution of their network of knowledge, in line with the three dimensions of TPACK?

4. Research Context

The research questions will be answered in the context of a specific MOOC that has been created in the context of the MaSCE³ project with the aim to educate mathematics teachers in designing outdoor mathematics tasks and using them in the context of the MathCityMap system. In the following, we will describe the MOOC and the participants (The enrolled members are all mathematics teachers teaching in different school grades or pre-service teachers. We will, from here on, refer to them all as teachers) as the context of the paper’s study.

Description of the MOOC

Preparations: The design of the MOOC “Task Design for Math Trails” started in November 2019 and the digitization of all resources in the platform was completed in December 2020, about one year later. The MOOC was advertised from January 2021, through different channels (MaSCE³ website, mailing list, and social media such as Facebook and Twitter). It was made explicit that the target audience of the MOOC was mathematics teachers from primary to higher secondary schools, eager to learn how to do mathematics outdoors, to learn to use MathCityMap, and to design their own tasks and trails to walk with their students both in their own neighbourhood and across Europe. Thus, enrolments were free and voluntary and started from February 2021 until three weeks after the start of the MOOC. The official opening date of the MOOC was 8 March 2021.

Technical Details: The MOOC was delivered via the DI.MA. Moodle platform (https://dimamooc.unict.it/ (accessed on 1 October 2021)), managed by the University of Catania. It used the Moodle 3.10+ version. The server where the MOOC platform was deployed is Server Debian 4.19.171-2 (2021-01-30) x86_64 with PHP version 7.4.15. The deployment details of MOOC server are in the local data center. From a technical and IT point of view, the platform was set up and monitored by the IT team of the University of Catania, which was also responsible for matters relating to possible attacks on the server (which did not occur). The components enabled in the platform are all the plug-ins available in Moodle, particularly the forums and the assignments (which we have referred to as homework). The former were used to allow participants to interact with each other in asynchronous mode, and the latter to allow instructors to monitor the learning progress of participants.

Structure and Contents: The MOOC was organized in modules, with each module’s respective data collection instruments defined as shown in Figure 5. Besides general information about the MOOC, its homepage allowed the participants to access the module’s resources when they were made available. On the Introductive module, participants found the presentation and introduction resources; namely, a presentation video made by Prof. Matthias Ludwig (the MaSCE³ project coordinator and one the co-author of this paper), another video introducing the instructors, a brief overview of the MOOC setup, and videos concerning outdoor learning and math trails. This was the module wherein a pre-course questionnaire was administered.

In module 1 “Outdoor Learning and Task Design”, the following topics were covered: an overview of the MCM web portal; an explanation about math tasks’ creation using different answering formats; the criteria for meaningful math tasks; and, finally, the math task publication process. In these two weeks, different data collecting instruments were used; namely, two discussion forums and the Excel review file used by the instructors to register the reviewing process for the two submitted math tasks, particularly multiple-choice and exact value.

In module 2 “Task Wizard and Subtasks”, to explore the content task wizard (a template used to implement generic tasks), four resources were used, namely, three videos and a static tutorial: a video that explains the task wizard concept with an example; the tutorial made using Sway (https://sway.office.com/ (accessed on 1 October 2021)) to explore other task wizard topics; a video exploring the creation of one of the GPS tasks; and a video with other examples of GPS tasks. Moreover, to explore the subtasks content, a webinar was used to present this new feature of MCM. The instruments used to collect the data were the task wizard forum, the subtasks forum, and the Excel review file of the four requested math tasks.

Module 3 “Interval” began with the exploration of interval tasks. The two resources available (a video and a pdf) teach strategies to set the bounds of an interval. As in the previous modules, a forum and review file in Excel were available for the two tasks.

Module 4 “Trail & App” included the following topics through nine resources (videos): How to create a math trail? How to modify it (if necessary)? How to request its publication? As well as an overview of the MCM app; the feature narrative that places math trails in some fiction context (e.g., a pirates’ adventure), so that the students can feel along the trail as characters of the story; and the digital classroom, a tool to monitor, interact, and follow the math trail with the students. The Excel review file and the forum were the instruments selected to collect the data.

Finally, the Final Module lasted for three weeks. The aim of this module was to put into practice the implementation of the trail, providing all the guidelines. As a conclusion of the MOOC, the webinar “Closing Ceremony MOOC” was also realized to have the opportunity to enter into dialogue with and greet in synchronous modality the MOOC participants. The chosen instruments were the final questionnaire and the padlet (https://it.padlet.com (accessed on 1 October 2021)), a communication message board. Here, the teachers were invited to post photos of the trail implementation with the students and a short sentence about the experience (Figure 6).

Each MOOC module was associated with a badge, which was automatically issued by the platform if and only if the homework foreseen in the module was fulfilled. Some of the homework was fulfilled by simply putting a tick in the box, which allowed the teacher to declare that he/she had watched that certain resource (video, tutorial, and so on). On the other hand, in the end-of-module homework, the tick ensuring completion was automatically inserted by the platform if and only if a specific resource was uploaded by the teacher. This end-of-module homework consisted of the design of a math task on the MCM web portal, in one of the following languages: English, Estonian, French, German, Italian, Portuguese, or Spanish. Once the task was created, it was submitted for expert review (there was at least one reviewer for each of the languages). The task could be directly approved for publication if it complied with the criteria for meaningful math tasks. Otherwise, the reviewers declined the publication by sending an e-mail to the teacher through the MCM review system explaining the reasons for the refusal. The teacher then had to revise the task and resubmit it for revision until it was published. If the publication was obtained, s/he could upload a screen (.pdf or .png file) on the MOOC platform, showing the title of the task, the name of her/his author, and the status of the publication. If all the homework of the modules was fulfilled, i.e., if all the badges were collected, at the end of the MOOC, a final certificate was issued by the University of Catania, certifying 30 h of training.

Participants: The DI.MA. platform shows that the number of registered participants, excluding instructors, is 503. Among them, 87 had never entered the MOOC, thus the learners’ population is reduced from 503 to 416. Of these, only 236 were active (those who filled in the pre-course questionnaire). Figure 7 gives an overview of the active participants’ nationalities. The number of participants who completed the MOOC, satisfying all the homework and thus collecting all the badges, is 93 (for detailed information, see Figure 8 and

Table 1. Detailed information about the 93 MOOC finalists.

Country (# of Teachers from that Country)	Gender	Age Interval	Current Status	Grade Teachers Declare to Teach	Already Participated in MOOCs
Brasil (3)	F = 1 M = 2	30–39 = 1 40–49 = 1 50–59 = 1	in-service teacher = 2 employed at university = 2	lower secondary school = 1 higher secondary school = 3	Yes = 1 No = 2
France (18)	F = 14 M = 4	20–29 = 1 30–39 = 1 40–49 = 14 50–59 = 2	in-service teacher = 14 pre-service teacher = 2 teacher educator = 2	primary school = 8 lower secondary school = 5 higher secondary school = 8	Yes = 9 No = 9
Germany (6)	F = 5 M = 1	20–29 = 2 30–39 = 1 40–49 = 1 60–69 = 2	in-service teacher = 4 pre-service teacher = 2	primary school = 5 lower secondary school = 1	No = 6
Greece (2)	F = 1 M = 1	30–39 = 1 40–49 = 1	in-service teacher = 2	primary school = 1 higher secondary school = 1	Yes = 2
Italy (22)	F = 18 M = 4	20–29 = 7 30–39 = 5 40–49 = 4 50–59 = 5 older than 69 = 1	in-service teacher = 15 pre-service teacher = 5 employed at university = 2	primary school = 2 lower secondary school = 6 higher secondary school = 11	Yes = 9 No = 13
Peru (1)	F	50–59 = 1	in-service teacher	lower secondary school	Yes
Portugal (24)	F = 20 M = 4	20–29 = 13 30–39 = 2 40–49 = 2 50–59 = 5 60–69 = 2	in-service teacher = 9 pre-service teacher = 13 employed at university = 2 teacher educator = 1	primary school = 5 lower secondary school = 7 higher secondary school = 5	Yes = 15 No = 9
Slovakia (1)	F	40–49 = 1	in-service teacher	lower secondary school	Yes
Spain (13)	F = 7 M = 6	20–29 = 2 30–39 = 1 40–49 = 5 50–59 = 5	in-service teacher = 11 pre-service teacher = 2	lower secondary school = 7 higher secondary school = 11	Yes = 5 No = 8
The Netherlands (1)	F	30–39 = 1	in-service teacher	lower secondary school	No
United Kingdom (1)	F	20–29 = 1	pre-service teacher		No
Uruguay (1)	F	40–49 = 1	in-service teacher	lower secondary school	No

). Therefore, the completion rate of the MOOC (calculated on 236 active participants) is 39%. This is significant for us as instructors because, in general, the completion rate of international MOOCs for mathematics teachers’ education is 12% [28]. Moreover, this percentage was achieved during a particular period: the COVID-19 pandemic, which affected the whole world, with uncomfortable impacts on the world of education, forcing the mode of distance learning and teacher overload [28,29].

It is interesting to remark on the gender difference among the MOOC participants and finalists. For all 236 active participants, the percentage of female participants (67) is more than twice the percentage of male active participants (32). Furthermore, in terms of the finalists, Table 1 shows about 71 female finalists in contrast to 22 male finalists. Thus, the ratio here is quite similar to the previous one, although the female participants have been a little bit more successful than male ones. It is also remarkable that the predominance of female finalists occurs in almost all countries except Greece and Spain, which have a gender-balanced number of finalists.

In the following, we will focus on the finalists of the MOOC. For this group, we can assume a continuing and great awareness during the MOOC because the finalists received all their badges and submitted all mandatory math tasks, which were approved by the MOOC instructors.

5. Methodology

5.1. Description of the Sample for the Case Studies

With a focus on the research aim of this paper, the individual learning progress of the participating teachers is of high relevance. Therefore, checking the Excel file used by the instructors to accomplish reviews of math tasks designed by teachers, we selected several case studies for the research on the MOOC. The following criteria were made for the selection of the case studies:

• Complete participation: To measure the learning outcomes and PD during the MOOC, it is important that the case studies completed the MOOC and received their certification. Therefore, only the finalists can be chosen for the case studies.
• International level: In order to allow an international interpretation of the results, the case studies are selected from different participating nations. Therefore, all project countries of MaSCE³ (France, Germany, Italy, Portugal, and Spain (of the enrolled Estonian teachers, none completely finished the MOOC; therefore, no teacher from Estonia can be taken into consideration for the case studies)) are taken into consideration.
• Different learning levels: As the individual learning outcomes might differ concerning the previous knowledge, digital competence, and task design experiences, we choose teachers of different performances. For each country, two teachers of a very good level (i.e., designers of math tasks generally accepted already at the first revision submission) and two teachers of a medium level (i.e., designers of math tasks with one/two revision rounds needed) are chosen (exception: for Germany, only one very good teacher could be chosen).

The above criteria are selected to fit the MOOC described here. Accordingly, this is an initial experience with this selection of criteria. With these selections, 19 case studies are chosen for the analysis concerning the research questions. As for the active MOOC participants and finalists, we observe that the previously mentioned high gender difference blows up in the collection of selected participants for case studies; that is, there are only two men (1 in Germany and 1 in Spain, along with 17 women). In summary, there is a clear gender bias in the MOOC and, furthermore, in the selected participants contributing to the MOOC analysis. In the following, we will present the data considered for the analysis.

5.2. Description of the Pre- and Post-Questionnaire

Two questionnaires were administered to the teachers enrolled in the MOOC: a pre-course questionnaire and a final questionnaire (see Appendix A). Throughout the questionnaires, we made several connections to the different sections of TPACK, which will be added in brackets in the following lists.

The pre-course questionnaire consists of three sections, aimed at investigating in a respective manner the following: teachers’ personal and professional information (PK); their previous experience with MOOCs (TK); and their knowledge of outdoor mathematics and mathematical trails (CK).

The final questionnaire, in contrast, is composed of six sections. The first one aims at collecting the personal and professional information of the teachers. The other sections are aimed at investigating the following aspects about the teachers: conducting the final teaching experiment with their students and the digital classroom (TPACK); their opinions on the MCM web portal and app (TK); comparing their professional experience and knowledge of outdoor mathematics and mathematical trails before starting the MOOC with their professional experience and knowledge of the same topics after attending the MOOC (PCK); and their professional training experience during the MOOC to assess the degree of satisfaction and impact of this online educational experience on the teachers (TPACK).

In both questionnaires, the different sections contained open-ended, semi-open, closed, and Likert scale questions. The questionnaire was produced using Google Forms, an open-source application for online surveys, and uploaded on the DI.MA. platform. The analysis of responses was performed with Excel. Both questionnaires are based on previous MOOC experiences (e.g., [22]) and on experiences in outdoor learning settings (e.g., [30]).

For the qualitative and quantitative analyses derived from the questionnaires, we consider the MOOC-MDT framework. Our focus is on the MOOC-ecosystem, i.e., the space hosting the teachers who can interact with the content prepared by the instructors (the MOOC-artifact) as well as add new content, allowing modifications to both the MOOC’s network of knowledge and their own network of knowledge. Modifying one’s own network of knowledge implies that either a new node is added to it or new connections to pre-existing nodes are generated. In our analysis, we will speak of a new node when teachers claim not to have any knowledge about a certain notion before starting the MOOC. On the other hand, if a certain notion is already included in their network of knowledge, it means that the node related to that notion exists, because they claim to have even a smattering of it before the start of the MOOC. In this case, we are going to investigate the creation or renewal of connections of this node with others present in the network. Thus, to describe the evolution of the teachers’ network of knowledge, we will consider the double learning process. The process of instrumentation/self-organization (E → I) includes the interactions that take place with new elements of the online world, from the platform in general to the resources in the modules and on the MCM web portal in particular. The process of instrumentalization/sharing (E ← I) concerns dynamics that are implemented when one is familiar with the online environment and its resources.

5.3. Description of Qualitative Data and Discourse Analysis

For a finer qualitative analysis, which takes into account not only the open answers to the questionnaires, but also the teachers’ posts in the padlet (there are other sources of information on teachers’ opinions about the MOOC, such as those posted on MOOC-related forums, but they are not analyzed here, as we consider they are less formally relevant than questionnaires’ answers or padlet posts), we make use of another methodological instrument. Following the recent work of [27] in the context of a pre-service teachers’ course, the authors of the current paper considered also using the analysis of reflective narratives or discourse analysis—from the perspective of [31]’s “Discipline of Noticing”—of some selected participants. We are aware of the use of the same theoretical framework in previous experiences, such as the one already mentioned by [27], the work of [32] concerning university lectures, or the work by [33] on a primary teacher, but we do not know of the existence, so far, of an application of this methodology involving MOOC participants.

Here, the reflective narratives of the 19 case studies are analyzed from the perspective of [34]’s Discourse Analysis and of the Reflection on Practice by [31,35,36], applied to the acquisition of competences for outdoor mathematics and, in particular, to the use of math trails with MathCityMap, in the context of PD. Their answers to the different questionnaires will show their reflections about what they had observed (“noticed”) and what they had estimated that they had learned from the different resources available in the MOOC in relation to their future teaching practice.

The discourses analyzed here show traits, or what [37] calls “habitus”, of different level, quality, structure, function, form, and professional content (related to the teaching profession). The teachers attending the MOOC, as participants in the locutions, did not simply construct the semantic representation of the verbal presentations in their episodic memory, but also a representation (with their own contributions) about the event of the training sessions. Linked to the MOOC-MDT framework, this means not only that is a node generated or activated in the teacher’s network of knowledge (E → I), but also that it is connected to other nodes in the network of knowledge (E ← I).

The field of pragmatics studies verbal utterances as speech acts and analyzes the speech function or locution force as a speech act. Thus, our goal in this context is to evaluate the degree of this force, studying the content of the verbal emissions of the selected participants in relation to the teaching of outdoor mathematics and the use of technology, as well as their predisposition to possible future teaching work. All this is valued for the level of depth and detail shown by the teachers in the training course.

As in [27], three degrees of reflective narratives will be identified, freely adapted for this case (of mathematics teachers in a training MOOC), based on the ideas of [31]:

The low grade, which will correspond to features of a narrative with a low degree of reflection, in which the trainee teacher only describes or refers to anecdotal events of the training sessions and that we could relate, in some way, to the “account-of” dimension.

The intermediate grade, in which the teaching disposition is naive and slightly conscious. Thus, in this case, we consider that their degree of narrative reflection is valued at a midpoint, which we could identify with a new dimension, called “account”, between “account-of” and “account-for”, for expressing attempts of interpretation of crucial ideas from training sessions.

Finally, the high grade (“account-for”), which already indicates a reflective and personal teaching disposition, with an attempt to conceptualize the key ideas of the sessions and a teaching expectation for the professional future.

The final goal is about obtaining information—analyzing the reflective narratives present in the answers of the selected participants to the questionnaires and padlet—on the effect of the transmission of certain technological innovations on the convictions of teachers about the teaching of such a discipline, thus helping to develop plans that improve the training of teachers towards the acquisition of professional skills for using outdoor mathematics in the classroom.

In what follows, we will see that their answers show different degrees of reflection, ranging from the naivety of certain assessments to professional initiatives and purely mathematical perspectives. The didactical and mathematical consequences of this analysis will be presented and discussed in the Conclusions section.

We will also see that the TPACK perspective spreads this analysis along various dimensions, from purely technical or pedagogical for mainly naive occurrences, towards an integrated and articulated perspective for more in-depth reflective narratives. This allows identifying areas of improvements in the teachers, whether based on the quantitative analysis and comparison of the questionnaires or on the more qualitative discourse analysis. Therefore, the questionnaires include questions related to the Technological, Pedagogical, or Content knowledge sections of TPACK and its intersections. For example, the questions “Have you previously taught mathematics outdoors?” and “Describe what mathematical topics you covered” are clearly Content based. The questions “Have you ever heard of Math Trails?” and “What do you think they are or what are they for?” are closer to Technology. Finally, the questions “Was there a task (or more than one task) that created problems for your students?”, “If yes, how did you intervene?”, “After this experience, do you think you will change one or some of your tasks? Why?”, “Describe the positive aspects you have experienced running the course with the digital classroom”, and “Describe the negative or difficult aspects of running the course with the digital classroom” are related to Pedagogical and Methodological issues and, although related to technology, sometimes reach the central node of the project, articulating all the different aspects of the TPACK.

6. Results

6.1. Changes in the Attitude towards Outdoor Mathematics

In the pre-course questionnaire, the teachers were asked the following open-ended question: “Why did you join MaSCE³ MOOC?”. In the following analyses, we consider the 93 finalists. About half of the participants declare quite superficial or anecdotal reasons (“account-of”) for entering into the MOOC-ecosystem. The remaining half are split into those who think occasionally (“account”) of the MOOC regarding its potential role in their PD and those who seriously focus on this opportunity (“account-for”). This question was corroborated by a multiple-choice question: “What is your primary goal in taking this course?”. It emerged that about two-thirds of the participants declare quite superficial or anecdotal reasons (“account-of”) as their primary goal in taking the course, whereas the remaining third think occasionally (“account”) of the MOOC regarding its potential role in their PD. Only less than 10% seriously focus on this opportunity for their PD (“account-for”).

In order to obtain more information about the teachers’ network of knowledge regarding their attitudes on outdoor mathematics, before they started their training with the MOOC, a series of questions were asked in the pre-course questionnaire, remarking that, sometimes, the precise nature of the questions makes it impossible to apply the reflective narratives methodology here, being replaced by simple numeric outputs. The data we report refer to the 19 selected case studies. The question “Have you previously taught mathematics outdoors?” showed that 9 teachers out of 19 had experience in teaching mathematics outdoors. Among the 10 who had never taught mathematics outdoors, 5 belong to teachers labelled as the very good level. We also asked, “Have you ever heard of Math Trails?” Here, 5 out of 19 teachers answered No; that is, one very good level teacher and one middle level teacher from Italy and Spain, and one middle level teacher from Germany. It is observed that, in general, both those who were previously acquainted with math trails and those who were not were able to give a quite reasonable description of what math trails can be used for.

The 14 teachers who responded that they had knowledge about math trails were asked the following multiple-choice question: “Have you already created your own math tasks and/or trails for outdoor mathematics?” Only one Italian, one French, and two Spanish teachers (all middle level participants) answered Yes and that they do it by themselves, declaring to have already created a math trail, either with a class or with colleagues or on their own. The remaining 10 answered No, but plan to do it in the future. For us, it was important to know the reasons why these teachers who are familiar with the math trails had not yet tried to create them. Thus, we asked the multiple-choice question, “What are the reasons why you have not been active so far?” We found that most of them declared a lack of time, lack of ideas to implement trails, or ignorance about the MathCityMap project (a mixture of “account-of” and “account” comments), as well as difficulties in introducing this activity into the school curriculum (“account-for”).

For those four who answered that they had already created math trails, we wanted to explore this further by asking the question, “Did you walk a math trail with your class?” All of them answered Yes. They were also asked, “If yes, what are your experiences when walking a math trail with your class?” It is observed that all of them highlighted the interest and pleasure of the students while doing the math trail (“account-for”).

From these preliminary analyses, with the questions asked before starting the training, we can see that our case studies are quite varied. Only 9 out of 19 have the “outdoor mathematics” node in their network of knowledge. Moreover, although most of them (14 out of 19) have heard of math trails, implying that they have this node in their network of knowledge, a minority (4 out of 10) provided some “account-for” explanation, mentioning they have generated a connection of this node with their other professional practices. In fact, most of the 19 teachers had never tried to create a math trail, so they have not connected this node (i.e., content knowledge, in terms of TPACK) with pedagogical or technological knowledge. For some of them, as stated in their own reflective narratives (‘’account”), this is because of not having had, until then, sufficient training in this sense (e.g., difficulty in implementing it within the curriculum and ignorance of the MathCityMap project). Others mentioned rather more anecdotal reasons (“account-of”), including a lack of time or of ideas.

The pre-course questionnaire concluded with a space for free comments; the majority of the 19 case studies expressed their appreciation to the MOOC instructors for the theme of the MOOC, math trails, also declaring their interest in the course.

As anticipated in the research context section, the homework of the final MOOC module involved creating a math trail containing the eight outdoor math tasks designed during the previous modules and experimenting with this trail with one’s own class, using the digital classroom. In the following, we focus on questions from the final questionnaire that investigate how the 19 teachers conducted the outdoor experimentation. First, we wanted to investigate which of their classes they had involved in the experimentation, with the open question, “If you teach in more than one class, which class have you chosen to run your Math Trail with the Digital Classroom?” Two teachers chose grade 1, one with a group of 21 students and the other one with a mixture of 14 students from grades 1 and 2. Another teacher practiced with 12 pupils from grade 4 and another one with 4 pupils from grades 5 and 6. Three teachers ran the trail with students from grade 7, two of them with 20 students and the other one with 16. There also was a group of 19 students from grade 8 and two groups of 20 and 19 students each from grade 9. The most chosen grade was grade 10, selected by four teachers, with a range of students from 6 to 34. A math trail with 22 and 19 students of grades 11 and 12, respectively, was run only by two other teachers. Another teacher worked with 15 adult students. There was another participant who selected 18 volunteer students and another one who did not have students and chose family members. All the teachers were asked to motivate their choice, by answering the question, “Why did you choose this class?” It emerged that participants declared that the election was made for practical reasons (there was time in the class for running the math trail, the topics in the trail fitted better the curriculum of the class, and so on). Furthermore, they also referred to the quality and interest of the chosen students (again, a kind of “account” narrative, midway between anecdotal and professional reasons).

Subsequently, it was in our interest to understand how they had set up their options to run the trail. We asked if and why “Have you set the narrative mode?” and “Have you set the gamification mode?” It turned out that the majority did not set the narrative mode (14/19), declaring reasons (“account” or “account-of”) including a lack of time and lack of confidence in how to handle it, considered as something optional, not actually relevant, or adequate for their students. Only two teachers did not set the gamification mode, for simplicity. The rest used it, for fun, to raise more interest among the students (“account-for”).

It was also important to understand with which materials the teachers had instructed the students to equip themselves. We thus asked the following multiple-choice question, “Each group of students was equipped with the following materials”.

As shown in Figure 9, the most used material was the calculator. The MCM app was used by all the teachers who were graded very good. Of the four teachers who did not use the MCM app, three used the pdf of the trail, but one did not use it, which leaves doubts about how the trail was implemented. It is also found that most teachers (14/19) selected the notebook/sheets on which to write, as well as the rigid meter.

To find out whether teachers made use of anything other than the standard materials, the following open question was asked, “In addition to those already indicated, what other tools were needed to run your trail?” It turns out that the teachers highlighted other materials such as tape measure, flexible meter, ropes, level, manipulative materials, set of rulers, chalk, adhesive tape, and a homemade instrument for measuring angles.

In order to explore with what level of confidence the teachers had performed the outdoor teaching experiment, we asked them to express their degree of agreement (from 1 = strongly disagree to 4 = strongly agree) with specific statements. It turned out that 16 out of 19 of the teachers agreed or strongly agreed to the statement “You started running the trail at the time you had set with the digital classroom”. The three participants that disagreed (one of them strongly disagreed) were of medium level. All the teachers who agreed with the previous statement also agreed or strongly agreed with the further statement “Thanks to the methodological indications proposed by MOOC, it was easy to divide the students into groups and prepare the necessary material to be provided to each group”. Only three teachers, two from France (very good and medium level) and one from Portugal (medium level), disagreed with the statement “It was easy to monitor my students through the digital classroom”. The same three teachers, together with the other French medium level teacher and one of the Italian medium level participants, also disagreed that “It was useful to monitor my students through the digital classroom”. What all the participants agreed upon, and especially 14 of them strongly agreed, is that “Students had fun doing maths outdoors”.

The teachers, as required by the MOOC, were to conduct the experiment by also starting their own trail by activating the digital classroom, one of the latest innovations of MathCityMap. It was thus interesting for us to see what use they and their students made of it. We asked a multiple-choice question regarding their use of the chat, and it turned out that only one teacher fully used the technical possibilities offered for textual chatting and exchanging pictures, while six only used textual chat and the other two-thirds did not use any synchronized communication tool.

Furthermore, we investigated the actions carried out during the digital classroom, with a multiple selection question, “During the monitoring of your students with the digital classroom, what actions have you carried out?” It was found that all of them used it to manage their groups in one way or another, except two, who could not do so for technical reasons. Most of them used the feature to track students’ movements on the map (12/19) and focus on individual groups (5), as well as to reach a particular group in difficulty (8/19) or chat with all or reply privately to a selected group. All in all, this showed a reasonable appropriation of the communication tool, answering the needs of running a trail in the field, although the final questionnaire raised some safety issues not addressed by technology (i.e., difficulty of keeping an eye on the students, so the digital classroom is not sufficient, as well as a lack of maturity among students). These technical competencies seem almost transparent, as their chat interface is pretty well conceived and close to current social media usage, compared with the more mixed feeling (4/19 negative feedback) regarding overall students monitoring usefulness.

Using the digital classroom, teachers also have the opportunity to understand how students relate to the math tasks that make up the course. Therefore, we asked the question, “Was there a task (or more than one task) that created problems for your students?” The majority declared that there were some tasks that created problems for the students, and we asked them, “If yes, how did you intervene?” It emerged that the teacher helped them by recalling some concepts, explaining the involved task in person, telling them to consider the hints, to read the question more carefully, and so on. Only two Italians and one Portuguese teacher answered No, without further explanation.

Finally, we asked the teachers to reflect on this outdoor experimentation experience with the following questions, “After this experience, do you think you will change one or some of your tasks? Why?” “Describe the positive aspects you have experienced running the course with the digital classroom” “Describe the negative or difficult aspects of running the course with the digital classroom”. It turned out that 12 participants said they would change some tasks, feeling that some tasks are too long or too difficult (“account-for”). Regarding the positive aspects, it emerged not only that teachers appreciated the possibility of following the activity of the students in real time, as well as at a distance (using the digital classroom), but also that they noticed motivation in the students (again, a rather “account-for” reflection). Concerning the negative aspects, most of the answers referred just to technical, and thus in some sense anecdotal (“account-of”), issues. In a general sense (“account-for”), teachers reported a lack of maturity in some student groups regarding leaving them alone, particularly under COVID restrictions, and walking outside the school; in order to take this into account, it might be better to have smaller groups, as “difficulty to keep an eye on the students, digital classroom is not enough”; to have smaller walking teams to make each member feel like a protagonist; and to give some previous advice to the students in the classroom, among others.

Most of the reflections on the padlet (10 out of 17, as two teachers in the case study did not reflect anything) correspond to the “account-for” dimension, referring mainly to the positive motivation of students in engaging in outdoor mathematics with MathCityMap. There are also some references to students’ specific mathematical skills. Four cases are considered as the “account” dimension. In three of them, the usefulness of the technology in classroom management is reflected, although one teacher reported difficulties with this tool. The fourth declared the (not very relevant in this context) discovery of the Fibonacci series (Italian teacher at medium level). The remaining three teachers correspond to the “account-of” dimension.

The 19 case studies concerning the experience of the trail the teachers designed for their students cover almost all school grades (from 1 to 12). One of them also involved adults and another involved family members. Concerning the organization from the methodological point of view of running the course, we observe that all of them integrated into their network of knowledge the necessity of having to give the students specific materials to solve the math tasks and, in addition to those indicated by us as instructors, they were also invited to use other materials. This indicates that, in their network of knowledge, they have created valid connections between the nodes learned during the training and the pedagogical knowledge they already possess. Moreover, from the technological point of view, we notice that new nodes have entered their knowledge network: the narrative mode, the gamification mode, and the digital classroom. The first two were not connected to other nodes in everyone’s network of knowledge (the narrative mode was not used by most, because it was considered childish in relation to the age of the students with whom they experimented; indeed, it is an incentive designed for primary school students). The digital classroom instead was used with familiarity by all, also showing appreciation for the possibility of following the students’ actions at a distance and being able to interact with them at a distance via chat, immediately having an idea of which tasks were too long or difficult for the students. This favored, on the one hand, the implementation of pedagogical practices aimed at immediately helping students who showed difficulties (going to explain the exercise, indicating to consult the suggestions, and so on). On the other hand, it allowed them to reflect on how to modify certain tasks for future use of the trail.

The padlet is a proof of the evolution of the teachers’ networks of knowledge. Indeed, recall that, in the initial questionnaire, only 4 out of 19 case study teachers had previously created and experimented with an outdoor math trail. Although with varying degrees of reflective narratives, all 19 teachers completed the experiment, demonstrating that they were able to create connections allowing interaction between their content, pedagogical, and technological knowledge.

6.2. Learning Progress and Usage concerning the Digital Features of MCM

In the final questionnaire, we investigated the E → I process by means of a Likert scale question (from 1 = strongly disagree to 4 = strongly agree), which asked teachers to express their degree of agreement to specific statements (Figure 10). For the 19 teachers in the case studies, the steps of the instrumentation of the MOOC-ecosystem and their personal self-organization within this virtual environment occurred intuitively and/or spontaneously. In particular, we focused on the actions taken by these teachers in drawing information/training from the MOOC in order to orient themselves in the MathCityMap web portal, the space in which they played their role as designers of math trail tasks. In fact, we observe that (considering together items 3 and 4 of the Likert scale, “agree” and “strongly agree”, respectively, in Figure 10) all of them declared it easy to register on the MathCityMap web portal, as well as to orient themselves in it. They also all considered the creation of tasks and trails on the web portal to be intuitive. Therefore, it was easy for them to understand how the MathCityMap web portal worked. Interactions between the web portal and the app were also considered simple by the majority; only one teacher disagreed with this statement. All agreed that the MOOC offered enough material (e.g., tutorials, exemplar tasks, and so on) to learn to use MathCityMap.

With respect to the creation of the math tasks, we also wanted to investigate, with a Likert scale question (from 1 = not at all easy to 3 = very easy), “How easy do you think it is to create a math task with the following answering formats: exact value, multiple-choice, interval” Figure 11 gives an overview of the judgement of the teachers concerning the ability of the creation of tasks with the different task formats introduced in the MOOC. The teachers judge the multiple-choice tasks to be very easy (73.7%). This is followed by the creation of tasks with an exact value (57.9%). Finally, only every second teacher reports the interval tasks to be very easy.

Regarding the process E ← I, by means of a Likert scale question (from 1 = strongly disagree to 4 = strongly agree), we asked the teachers to express their degree of agreement with specific statements related to the review they received on their tasks from the instructors (Figure 12).

From the quantitative analysis concerning the reviews within the MOOC, we see that most teachers strongly agree that the MOOC improved their math task design skills (63.2%).

For example, teachers were made to think about modelling, measurements, and uncertainty, both in the complexity of the measurements they made and in those made by the students. A typical beginner’s mistake is to model an object in a questionable way without discussing the problem, allowing for other models, or checking that the acceptance of the answer can accommodate other legitimate points of view. Teachers have been asked to invent a task to illustrate a particular piece of the curriculum they have to teach, from a specific point of view. A narrow uncertainty estimation and implicit assumptions such as units are at the top of this list (see also [38]). However, the teaching convention that prevails in the classroom is clearly challenged in outdoor mathematics. In the MOOC, the focus was on raising awareness of the subtleties and depth of modelling; most teachers underestimated the beautiful and useful mathematics hidden behind both the curriculum they routinely taught and the problems arising from questioning the world around us. Outdoor mathematics has shaken up the prevailing classroom convention, changing the perspective on teaching mathematics.

Considering Figure 12, most of the teachers do not judge the revisions to be excessive, but to be clear (73.7%) and to be carried out in a short amount of time (47.4%). Therefore, for the 19 teachers in the case studies, the steps of the instrumentalization of the MOOC-ecosystem occurred in the MCM web-portal, and the sharing of what they have gained from it, obtaining the publication of their math tasks, is due to the revision work suggested by the instructors on their productions. We can witness here the crucial role of instructors in facilitating the conscious shift from a merely technological or content perspective towards a more integrated TPACK perspective.

The familiarity of the 19 teachers with MCM, i.e., the network of knowledge that the teachers had before starting the MOOC and that they had after finishing the training, was investigated in the final questionnaire with a pair of similar questions, on a Likert scale (from 0 = none to 6 = excellent): “Before the MOOC, rate your awareness of your skills in …” and “Now that the MOOC is finished, rate your awareness of your skills in …”. The response options included, in both cases, “Designing math tasks to practice maths outdoors”; “Designing math trails to practice maths outdoors”; “Using the MCM web portal”; and “Using the MCM app”.

Comparing the before (Figure 13) and after (Figure 14) MOOC graphs, we can see that there was a positive evolution of the 19 teachers. Regarding the ability to design math tasks to practice maths outdoors, before the MOOC, there were two teachers who did not have this competence, nine teachers who were divided between little and enough ability to design them, and eight teachers who had good to excellent ability to design math tasks outdoors. After finishing the MOOC, two teachers claimed to have acquired sufficient knowledge, while all the others felt much more competent than at the beginning. We observe a similar migration in the design of math trails for the practice of outdoor maths. In fact, before the MOOC, 8 out of 19 teachers declared to have no or little competence. After the MOOC, three teachers declared to have enough, and all the others felt sufficiently competent or much more competent than at the beginning. From the network of knowledge point of view, this implies that, for those who had no knowledge at all, a new node has become part of their network of knowledge. Meanwhile, for those who had little competence, the task and trail design nodes were present, but not sufficiently anchored in the network. After the MOOC, on the other hand, these nodes are not only present in all teachers, but have even established stronger connections with other professional mathematics teaching competencies in their network of knowledge. We can also observe an evolution in the usage competence of both the web portal and the MCM app. Before the MOOC, there were eight and nine teachers who had no or little competence in using the portal and the app, respectively. After the MOOC, they all claim to have good or excellent competence in their use. This testifies to the fact that not only have these two technological elements entered as new nodes in the teachers’ network of knowledge, but also they are also connected with the network of knowledge skills that the teachers have developed or strengthened during the training period in the MOOC.

In order to better understand whether teachers have benefited from an expansion/evolution of their network of knowledge, the following open questions were asked: (a) “Which elements of the MOOC have fostered your reflection on the possibility of doing maths outdoors?” (b) “Which MOOC elements best supported your development as a math tasks/trails designer on MathCityMap?” (c) “Which MOOC elements were ineffective or what adjustments could be made to improve the experience?”. We can observe that some answers to question (a) are quite generic. The teachers did not write a precise answer to the posed question. Here are some examples: “Even simple tasks can be fun” and “learning something outdoors in the real world could help students”. Other answers to question (a) are more precise. In fact, some state that the MOOC was their first introduction to math trails or to MathCityMap, while some others mention video introduction, examples, and materials. Concerning question (b), again some answers are quite generic. For example, “todos = all”; or “to do mathematics closer”; and “ability to use MCM for different school levels”. Other more concrete answers, in contrast, referred to specific MOOC elements: videos, tutorials, wizards’ tasks, interval tasks, and reviews by the reviewers. Regarding which MOOC elements were ineffective, nine answers stated no elements at all. Other answers mentioned the following: more examples, more time and flexibility, task wizard, the fact that the pirate narrative is not suitable for older students, to clarify better criteria for good tasks and subtasks, to adjust intervals perhaps with a percentage option, and pdf versions of trails that sometimes do not include hints. In summary, we could say that the narratives showed, mostly, a mixture of “account-of” and “account” dimensions: anecdotal or (perhaps superficially) curricular-driven.

We also wanted to investigate if teachers had difficulties in using MathCityMap with the following open question: “What difficulties did you face when using MCM (creating a task/trail or running a trail)?” It turned out that the answers were quite diverse: two (one very good level teacher from Portugal and one middle level teacher from Italy) said nothing, whereas some focused on technical issues (again, could be reported as “account-of”): app crashes (two Germans and one Portuguese), lack of mobile phones (one German), and generic technical problems (one Spanish). Interval tasks were considered as complicated (by one teacher from Italy and one from France). Lack of time to perform the trail and the length of trails were also considered as problems (by Italian and Spanish teachers). Intrinsic problems were also mentioned by the majority (“account-for”); that is, about 50% of the teachers complained about creating an original task, thinking of tasks beyond geometric aspects, thinking of valuable tasks, and structuring tasks through subtasks.

We then asked the following open question: “Which functions do you recommend for the next version of MCM?”. The teachers answered that it would be appreciated to have a paper format math trail (there is already such a feature) and that, on Android, the function of the back button to go back and not to exit the app. Other suggestions were to enlarge photos on the mobile phone to be able to read the suggestions, to be able to write formulas and math expressions more easily and to write text and at the same time insert images, and the possibility to choose different narrative modes and add more narratives. It was also advised to allow open responses for some tasks and the evaluation of procedures, not only answers, as well as to have access to tasks without necessarily being included in a trail. From these answers, we can deduce that teachers have incorporated well-anchored technological nodes into their network of knowledge, i.e., they have a clear idea of the potential and limits of MathCityMap. In fact, suggesting new features or improving some existing features is an indication that teachers have tried to create connections between the new technological nodes and the existing pedagogical nodes in their network of knowledge. The implementation of these suggested features would make their connections more stable and lasting.

It was in our interest to understand the teachers’ intentions regarding the future use of what they learned after the training. We thus asked the following multiple-choice question: “Are you planning to use MathCityMap in the future?”. All 19 teachers answered Yes.

6.3. MOOC Ratings

In the use of the MOOC-ecosystem by teachers, regarding the E → I process, we used a Likert scale question (from 1 = strongly disagree to 4 = strongly agree) that asked teachers to express their degree of agreement with specific statements (Figure 15).

The process by which the MOOC-ecosystem network of knowledge influences the participants’ individual network of knowledge was intuitive and effective for the 19 participants (instrumentation). Every participant considered the course layout easy to follow. Instructions, recommendations, and communications were clear, and the teachers confirmed that they were able to self-organize themselves to carry out the proposed activities, both in terms of deepening the resources provided and in terms of time management.

Regarding technical conditions, only five of the participants did not agree that the resources ran smoothly on their computer/Internet. Only three participants did not agree that the communication message boards (forum, padlet) encouraged interactions with other participants.

In relation to the E ← I process, we asked the following multiple-choice question: “As a MOOC participant, to what extent did you feel you were part of a community?” and asked them to motivate their answer. It emerged that two participants (from Germany) stated Little, owing to feeling alone in their institution concerning this MOOC because of language barriers (remember that the official language of the MOOC was English). There were eleven participants who answered Enough. They remarked on the feeling of belonging to a community through the chat, forums, emails, twitter, and their involvement in the MOOC activities. As for difficulties, they mentioned language barriers and a lack of time. The other six participants selected Very much: one from France, all from Portugal, and one from Spain. Again, forums, chats, and the sensation of interchanging information with teachers from different countries were the reasons exposed.

For us instructors, it was significant to understand how much personal commitment the teachers had spent in attending the MOOC. Thus, we asked the following multiple-choice question: “How much did you use the materials (e.g., videos, exemplar tasks, …) provided in this MOOC?”

The way each teacher expanded their network of knowledge and self-organized themselves, deciding which available resources were valuable to them, proved to be different from teacher to teacher. It is observed in Figure 16 that 47.4% of the 19 teachers used more than 80% of the materials. About 26.3% of teachers used between 61% and 80% of the materials and 21.1% of teachers used between 41% and 60%. Only one teacher claimed to have used 21% to 40% of the materials.

In any case, from the multiple-choice question: “How committed do you think you are to participating in MOOC?”, we can see that most participants declared that they were committed to participating in the MOOC (Figure 17).

It was important for us, the instructors, to understand the level of satisfaction experienced by teachers in attending the MOOC. We thus asked the following multiple-choice questions: “(a) Did MaSCE³ MOOC meet your expectations?” and “(b) How do you judge the duration of the course with respect to the topics covered?” We asked them to express a level of agreement (from 1 = strongly disagree to 4 = strongly agree) with the following statement: “(c) This MOOC was a valuable use of your time?” and we asked “(d) Would you recommend this course?”.

From the overall judgement of the teachers, the following conclusions can be drawn. All teachers state that the MOOC met their expectations—question (a)—whereby one teacher explains, “I really didn’t know what to expect and it resulted in a marvelous experience”. The duration of the course with respect to the topics covered—question (b)—was judged to be good by 94.7% of our case studies, with one teacher claiming it to be excessive. Concerning question (c), 84.2% of the teachers stated that the MOOC was a totally valuable use of their time, while 15.8% agree that it was a valuable use of their time. All of the case studies would recommend the MaSCE³ MOOC—question (d).

The last question of the final questionnaire invited teachers to “Leave a comment or any additional feedback/information you would like to provide.” In general, the comments were very positive. Here are some examples: “inspiring course”; “I will use MCM in my Erasmus project”; “far beyond my expectation”; and “the app is fantastic”, among others, along with specific thanks to the instructors. However, there were also some critical comments: “preparing these activities will get more complex for more advanced levels”; “the narrative didn’t work”; and “there should be more wizard options”.

7. Discussion and Conclusions

By taking into account the research questions of the paper, in this section, we intend to discuss the presented and analyzed results. We firstly answer the two research questions by taking the results into account. Afterwards, we formulate consequences for STEM PD and conclude the paper with limitations and continuing research ideas.

7.1. Answering the Research Question

Through a back reference to the theoretical framework of this paper, the following research questions were developed and will be answered hereafter:

Within the MaSCE³ MOOC-ecosystem, did the MOOC participants benefit from a double learning process?

(i) 

Did the participants benefit from an expansion/evolution of their network of knowledge, in line with the three dimensions of TPACK?

(ii) 

To answer the two research questions, we selected 19 case studies, following the selection criteria explained in the Methodology section.

Concerning the first research question, we can answer that teachers who attended and completed the MaSCE³ MOOC, by joining the MOOC-ecosystem, benefited from the double learning process.

The E → I process is visible in the MaSCE³ MOOC. In fact, teachers entered the MOOC-artifact, explored the available resources, and self-organized their network of knowledge (i.e., adding new nodes if the information is totally new to them, creating connections with existing nodes in their network if the information in the MOOC is not totally new to them). We, the instructors, had enough proof of this fact, not only from the analysis of the questions in the pre- and post-questionnaires, but also from the fact that all 93 finalists—and, in particular, the 19 case studies examined—produced the eight math tasks required in the modules and merged them into the trail that they then experimented outdoors with their students. This means that the teachers accessed the MathCityMap web portal and designed their math tasks following the indications provided by the MOOC. So, they have put in place the self-organized nodes and connections in their network of knowledge.

Moreover, the E ← I process is visible in the MaSCE³ MOOC. Indeed, on the MathCityMap web portal, the teachers designed math tasks (so new connections in their network of knowledge are stimulated), and the moment of sharing with the MOOC-ecosystem was realized on the one hand with the help of the instructors, who made their productions public on the web portal, and on the other hand with the sharing in the padlet of photos and testimonies of the outdoor experimentation conducted with the students.

Regarding the second research question, we can state that teachers have benefited from an expansion/evolution of their network of knowledge, which is in line with the three dimensions of TPACK.

Let us start by mentioning that, out of the 19 case studies, only 9 had experienced outdoor mathematics, only 5 had heard of maths trails, and only 4 had created and experienced an outdoor maths trail. By the end of the MOOC, all of them had both designed and experienced their own outdoor math trail tasks with students. Therefore, we could talk about expansion for those for whom MathCityMap and outdoor mathematics was a total novelty (a new node is added to their knowledge network, so, compared with TPACK, there is the integration of technological and pedagogical knowledge into the content knowledge). We could speak of evolution for those for whom MathCityMap and outdoor mathematics were not totally new, but who benefited from insights and awareness of these topics to improve professionally. This means that, in their network of knowledge, new connections are made on existing nodes, and new nodes are added too. For example, the digital classroom is a new node for everyone and, from the TPACK perspective, there is a deepening of technological and pedagogical knowledge.

In terms of TPACK, we can infer that the main concern of teachers was related to technological knowledge when using MathCityMap either as a user, author, or teacher. However, the reviews showed that issues of content, and thus knowledge of mathematics, came up on many occasions. The reviews were highly valued by teachers (Figure 12) and contributed to their PD. In this paper, we did not go into the details of the revisions; however, we can observe that, especially for the middle level teachers (those who went through more than one round of revisions), the comparison with the reviewers showed progress in improving their math task design skills. We can assume that the teachers had a solid knowledge of the curriculum, but, by adapting the wording of the question to the answering format allowed by the MathCityMap mathematical tool, they actually improved their inner understanding of the curriculum, as evidenced in some responses to the open-ended questions of the questionnaires and by their appreciation of the revisions and suggestions received from the instructors.

Moreover, the questionnaire and the padlet posts showed that technological/pedagogical issues, such as the constraints of the lessons to be moved out of the classroom and the maturity of the students in terms of respecting the rules outside the classroom, had to be taken into account and overcome. Many observables (Figure 18), therefore, point in the direction of strengthening all TPACK dimensions in the 19 teachers considered.

If we focus on the answers to the questions in the questionnaires, we can observe that the comments of the 19 selected teachers reflect a positive evaluation of the following: math trails as a methodology, MathCityMap as an app for developing math trails for the classroom, and the MOOC as a useful and inspiring course. One of the teachers, from Germany, quite often expressed some criticisms, involving all aspects of the experience, but the others relayed comments of different kinds (sometimes critical, sometimes anecdotal, and sometimes quite positive or suggesting improvements, among others). Many of these comments provide very useful hints for the MathCityMap developers, calling their attention to some methodological needs or to some technical problems that should be solved.

Moreover, we can observe the presence of many reflections from an “account-for’’ perspective on the experience, very professional, by the selected teachers. Obviously, it must be recalled that the 19 case studies were chosen from very good and medium level teachers. This is in stark contrast with what we have summarized for the questions “Why did you join MaSCE³ MOOC?” and “What is your primary goal in taking this course?”, showing that the term “account-of” better describes the narrative of the general attitude (of the global collection of participants, the 93 finalists, not of this selection of case studies).

As a conclusion, participants came for the MathCityMap app, students’ anecdotal motivation through outdoor math, and the digital classroom, and brought home a deeper understanding of modelling, uncertainty, and outdoor activities. The technical nodes somehow allowed for the exploration and deeper connection of already existing knowledge nodes in the content and pedagogical dimensions.

7.2. Consequences for Professional Development in STEM Education

We have observed in previous sections that many answers to the final questionnaire, as well as the padlet posts, reflect a very plural perspective concerning the whole MOOC experience. This could be expected, as the participants, even the selected ones, have very different personal and—most relevant—professional contexts.

However, for the same reason, it is remarkable to confirm the prevalence of reflections visualizing the different items of the general experience (the MOOC, its content and development, the involved technological novelties, their pedagogical relevance, the required didactical experiences with students, and so on) from the perspective of a teacher that is mainly thinking about why, how, and when he/she could bring these novelties into the classroom.

It is certainly true that the current didactical context, the appearance in different countries in Europe of new curricula relying on a methodology that emphasizes the need for a problem-based approach, collaborative work, and transdisciplinary content—let us call it STEM (Science, Technology, Engineering, and Mathematics) in a simplified way—greatly favors the development of initiatives such as this one, focusing on outdoor mathematics; that is, a way of approaching and of doing mathematics that provides a clearly pertinent answer to all STEM requirements.

The MathCityMap technology, and this MOOC that disseminates it, have arrived at the right moment and with the right purpose: to decisively contribute to solving the main problem posed by the implementation of whatever new methodologies that come, always unexpectedly and without serious preparation, to the classroom. Namely, the difficulty, for the teachers and the educational community at large to learn about how to conceive, how to work out, and how to evaluate STEM-related activities, in this precise context, is where we consider that the analysis performed in this paper shows the relevance and success of the MOOC experience for the professional development in STEM education.

7.3. Limitations and Future Perspectives

The described study aims at providing an insight into the benefits participants can obtain from their participation in a MOOC. To do this on a preferably holistic level, we chose 19 case studies from different countries and on different levels of knowledge. To analyze the actual impact of the MOOC, different research perspectives were taken into consideration.

Even though it was our aim to provide a holistic insight, this perspective is limited in the context of the study. We hereby see the following reasons to formulate these limitations:

• The case studies were chosen on different learning levels. From only the finalists, it is a rich selection. Still, the active participants that did not finish the MOOC were not taken into consideration for the analysis. Therefore, the 19 case studies could be interpreted as a positive selection, especially if one was interested in the reasons behind teachers quitting the course. As it was not the purpose of this study, the selection of the 19 case studies can be justified. Still, the selection must be taken into consideration when interpreting the results. In addition, it might be interesting for future research to see investigate the reasons for active participants not finishing a MOOC.
• The results have to be interpreted in the context of the specific MOOC that has been created for the purpose of presenting outdoor mathematics, designing outdoor tasks, and getting to know MathCityMap. In particular, the last focus is relevant in the formulation of limitations, as the participants that enrolled for the MOOC might be outstandingly interested in technologies (i.e., the MathCityMap system) and, therefore, might encounter less technical problems during their MOOC participation. It would, therefore, be of high interest for future studies to evaluate other STEM, but not technology-related, MOOCs in the presented way and see whether they resemble the ones presented here.
• Finally, the impact and benefits were directly measured after the completion of the MOOC. The presented results are thus to be interpreted on a short-term level. As a long-lasting impact and benefits are of high relevance in PD, it would be interesting to see whether the results can be confirmed after a certain period of time, i.e., one year. Questions that arise in this context are “Have the teachers used outdoor mathematics and MathCityMap since the MOOC has ended?”, “Have the teachers changed their attitude towards outdoor learning and MathCityMap since the MOOC has ended?”, and “Do the teachers feel more comfortable in the use of the digital components of MathCityMap after one year?”.

The limitations show that the results of this paper are of high relevance for future PD research in STEM education, in particular in the context of MOOCs and digital tools in education. During the study, many aspects on the impact of the MOOC could be shown and clarified. Still, many questions remain open and can be seen as points of reference for future research in PD in STEM education.