Blogged into the System: A Systematic Review of the Gamification in e-Learning before and during the COVID-19 Pandemic

Abstract

Gamification is becoming more relevant, especially after the start of the COVID-19 pandemic and quarantine. The purpose of our paper is to analyse the changes which occurred in the effects of gamification on e-learning as a result of the COVID-19 quarantine emergence in the spring of 2020. This paper describes the scientific contributions about the game elements most frequently implemented and their effects on those that use e-learning platforms, as well as the factors that contribute to the development of effective gamification in e-learning before and during the pandemic. Drawing upon the PRISMA framework, a number of 103 articles were identified in two databases: the Web of Science and Scopus. This paper discusses the previous works associated with the corpus of knowledge built around gamification in the past decade. This research shows that before the COVID-19 pandemic, even though many papers were written on gamification in education prior to 2019, there is a trend regarding the multiple mentions of using storylines, challenges, or badges in order to create and maintain competition among users, which in turn may influence and increase the level of social interactions and the coalescence of communities.

1. Introduction

In our paper, we aim to examine the changes that were documented to occur in the effects of gamification in e-learning both before and during the COVID-19 quarantine emergence in the Spring of 2020. Accordingly, we review articles detailing the effect of gamification in e-learning before and during the quarantine. Gamification is not a new topic of interest. However, the unique conditions imposed by the multiple quarantine stages from various countries between 2020 and 2021 brought new unforeseen challenges in education. The various e-learning systems used also entailed new opportunities. Gamification in e-learning is one of the processes that came into the forefront particularly during the pandemic. It entails the application of new technologies or the enactment of new strategies in the use of some of the old ones.

Gamification became more relevant in the past decade, particularly during the COVID-19 pandemic. Its relevance resides in the accessibility and possible applications gamification entails for increasing the efficiency of the activities or services from multiple fields. Our excursus, in which we examine several aspects pertaining to the relevance of gamification in e-learning, is complicated by the diffusion of the term from marketing in numerous sciences and fields of study: anthropology, sociology, management, game studies, and communication sciences, to name only a few. On the one side, some authors use different terms to designate the same concept, and on the other, the same umbrella term is used for several more or less related concepts [1,2,3,4,5]. In a nutshell, the several meanings associated with the term can be resumed in two senses:

(a)

In a “broad” sense, gamification represents any endeavour that entails the development of an activity using games: serious games, features of games, or game-based learning.

(b)

In a “narrow” sense, it entails extracting features from a game that can be incorporated into a coherent “package” or solution for increasing the apparent efficiency of an activity or service.

Nevertheless, it should be noted that [6,7] have clearly emphasized the “differences between gamification and game-based learning and explained that in game-based learning, students reach their educational goals by playing games” [7] (p. 142). Accordingly, we use, in our article, a “narrow” definition of gamification. Game-based learning designates the conscious use of games in order to improve the students’ learning experience. Gamification is the process of introducing game elements in a non-game context, such as an e-learning platform. The pervasiveness of this process has determined an increased interest in its elements and effects [6] (p. 142).

Before the COVID-19 pandemic, gamification was already a popular process used to engage employees, customers, and students. It was used to increase enjoyment and loyalty, incentivise behaviour, and develop engagement. However, the social distancing entailed by the pandemic has significantly accelerated the adoption of gamification as a means of keeping people motivated during a period in which the traditional means of interaction are limited.

The fact that the number of published peer-reviewed articles about gamification increased during the two years of the pandemic is a marker for the rising importance attributed to this process. It was used in a variety of contexts, such as remote work activities, team building, virtual events, counselling, e-learning, etc. Gamification was one of the processes associated with online interactions that was considered to be helpful during a time of isolation and uncertainty.

The quarantine imposed in many countries after the emergence of the pandemic determined radical changes in the educational activities from primary schools, secondary schools, and universities. Education moved from the real space to virtual space, from classrooms and course halls to e-learning platforms. Accordingly, gamification was one of the topics approached during the pandemic in the articles dedicated to e-learning. We admit that some of the reviewed articles include data collected before the COVID-19 pandemic. However, they are relevant due to the expectations that they generated: some of the articles published in 2020 anticipated trends about the development of gamification during the pandemic that became a reality in the following two years [8,9,10].Furthermore, during the pandemic, the number of papers published on the subject increased, and thus, we noticed that during the pandemic, regardless of when the research was carried out, the researchers’ interest in the topic increased as well. In this article, we expound a study with a clear goal: to develop a contribution to the current research by configuring an interconnected view regarding the effects of gamification in e-learning, particularly due to the fact that it evolves significantly each year. Furthermore, this paper covers an important gap in the “gamification science” as a subdiscipline of “game science” [11] (pp. 3–4) by expounding an ensemble of meanings associated with various game elements. It also facilitates a better understanding on how gamification was applied in e-learning, what were its results in terms of added value and what were its challenges and trends before and during the COVID-19 pandemic.

In order to analyse the subject of gamification in e-learning before and during the COVID-19 pandemic, we are interested in finding answers to the following research questions:

(Q1)

What were the most used game elements in e-learning before and during the COVID-19 pandemic?

(Q2)

What were the effects of the most used game elements on learners’ behaviour on e-learning platforms before and during the COVID-19 pandemic?

(Q3)

What factors should have been taken into consideration in order to develop effective gamification on e-learning platforms before and during the COVID-19 pandemic?

Our article is structured in the following manner: In the literature review section, the background of the concept of “gamification” is succinctly presented, with an emphasis on its definition, and then the articles related with our topic are expounded with a focus on the main applications of gamification in e-learning; in the material and methods section, we present the research methodology and research questions; then, we present the results of the research. Finally, our article ends by summarizing our conclusions and examining the limitations of the study that we describe herein.

2. Literature Review

At the beginning of the third decade of the 21st century, “gamification” was an umbrella term. It appeared in 2002, and in 2008, it was mentioned specifically in a published work [12]. Since 2010, it has been used in many domains of knowledge. Gamification has been associated with many activities. It has be defined it in the following manner: “Gamification is the use of game design elements in non-game contexts” [13] (p. 9). When expounding the relevance of gamification in e-learning contexts during the COVID-19 pandemic, there are several interesting consequences of using the definition provided by [13]. We agree with [14] that “gamification” encapsulates an essential difference between game and play.

Alternately, Ref. [15] defined gamification as “a process of enhancing a service with affordances for gameful experiences in order to support user’s overall value creation” [15] (p. 19). Intrinsically, this definition emphasizes the goal of gamification: it attempts to give rise to experiences, not methods. Whereas [13] predicated their definition on the thesis that gamification is based on the use of game elements, Huotari and Hamari’s definition appears to be more comprehensive, because “there doesn’t seem to exist a clearly defined set of game elements which would be strictly unique to games, neither they automatically create gameful experiences” [15] (p. 19). Unlike definition from [13] is not predicated on a set of features, mechanics, and methods. It entails a broader understanding of gamification, which is viewed as a process. The gamifier tries, through this process, to increase the probability of gameful experiences to develop and support educational activities.

We consider that the definition given by [15] has the merit of not intrinsically asserting that the process of gamification has to be successful. There are some terms in their definition that require further clarifications. Gameful experiences are a consequence of gamification that may be useful but are not necessarily advantageous. This is relevant also when we discuss gamification in educational activities. The most important dimension of the “gameful experience is that it is voluntary and that it is carried out by having intrinsic motivation” [15]. Notably, the expression “gameful experience” is used by the two authors in order to label the “experiential condition” or type of immersion that is unique to games. The term “enhancement” from Huotari and Hamari’s definition refers to the fact that gamification describes a service system that augments another service or set of activities. Furthermore, in accordance with this definition, not all the systems that combine games and other services represent a form of gamification. For example, the Moodle Learning Platform is not a gamified service platform per se. Nevertheless, it can be potentially gamified through outcomes, feedback, and reward systems. Thus, continuing with the example, if we consider the Moodle Learning Platform as a core service, then gamification would entail that it should be supported by various enhancing services such as plug-ins, compatible apps, various modules, etc. However, in accordance with the definition from [15] if the Moodle Learning Platform as a core service supports the enhancing services, then this process cannot be considered gamification.

The term “game” is associated with a device, while the term “play” designates an activity. As Brougère emphasizes, “game (and therefore gamification) is on the side of the device, it is indeed a device intended for play, a term which underlines that there can be playful activity without a specific device; but in the other direction, even if a game can remain on a shelf or if the mediocrity of its design results in that no one plays it, it has nevertheless been designed to play” [14] (p. 4). The “game” as a term does not have any meaning as long as the activity of “playing” is not related with it. As a device, any game makes possible particular forms of play. The connection between game and play is emphasized in numerous contexts: the concept of game can be ultimately imagined as a reification of play. In Brougère’s terms, “this should be understood as a historical and cultural process that leads to inserting traces of experiences (here, play) into objects (here, games)” [14] (p. 5).

“Gamification” is a concept that involves a paradoxical dialogical relation with its polar opposite: de-gamification. To explain this relation, we need to make several remarks. Gamification entails the careful introduction of various elements from a game into activities that are not correlated whatsoever with playing. Indeed, drawing upon Brougère’s theses, we assert that gamification is a process that ultimately can be described as a form of deconstruction: the game as a “device” is fragmented into elements. They are considered to have an intrinsic play value, which is independent of the game from where they were separated [14] (p. 6). In a nutshell, to deconstruct the game is to de-gamify it. This paradoxical transformation process leads to either a physical or a virtual device that cannot be considered a game anymore, in order to develop an activity that cannot be described as play. Nevertheless, we assume that the aforementioned activity will entail the same form of immersion like playing (i.e., the activity) the game (i.e., the device). This requires the introduction of various forms of achievement systems: levelling up, badges, discount points, etc. Thus, the concept of “gamification”, as it was developed in marketing, entails the motivation for goals that are not necessarily economic in nature. For example, these goals can be educational, social, and, in a broader sense, cultural.

Inevitably, one arrives to a fundamental question: what are the elements or features that are transferred from the game as an artefact or device to the gamified item? The answer to this question is difficult to find, as many of the elements extracted from the game are not specific to it. They are actually easily identified in various types of activities, since any game is inspired from the real world [14,16]. The quantifiable feedback elements transferred from the game are generally considered by those that “gamify” to be “key game mechanics” [17]. This notion is, however, different from the one named by game designers as “key”. Thus, according to game designers, “key game mechanics” represent the features of a game that are “central to the design aesthetics of the medium or to the player’s experience” [16] (p. 68). For the “gamifiers”, “key game mechanics” are the elements that can be extracted and incorporated in a software and/or in various services configured as “solutions” for various organizations.

We assert that gamification cannot be considered a product. Furthermore, while it is associated with various services, gamification cannot be simplified to correspond to a particular type of service. It has a processual dimension. Indeed, as [11,13], have emphasized, “the core of gamification is a design process intended to augment or alter an existing real-world process using lessons (initially) from the game design research literature to create a revised version of that process that users will experience as game-like” [11] (p. 3). While there is a tendency to reify gamification, we should stress the fact that it cannot be considered a product.

As a concept that encapsulates a broadly defined process from an extensional point of view, gamification slowly became, in the second decade of the 21st century, a topic of interest. One may argue that gamification could even be considered to designate a body of knowledge correlated to the “use” defined by [13] and the “process” delineated by [15]. Moreover, Ref. [11] attempted to develop a “gamification science” considered to be “a distinct subdiscipline of game science consisting of researchers adopting a social scientific epistemological footing” [11] (pp. 3–4). Interestingly, in delineating this new subdiscipline, Ref. [11], drew upon the post-positivist perspective derived from Popper’s (1934) theses: there is an objective reality that is viewed “through the lens of subjective interpretation” [11] (p. 2). This is relevant, because it entails that the scientific character of the theories from this subdiscipline is determined in accordance to Popper’s theses regarding their falsifiability. Upon scrutinizing the relevant research literature published before the March 2020 quarantine, one can easily observe that gamification is generally approached from the standpoint of Popper’s philosophical framework. The relevance of his post-positivism in relation to gamification is emphasized by [11] on several wide themes: (a) specifically designed and managed gamification interventions have a potential positive or at least desirable impact on people; (b) the impact of accurate gamification interventions can be appropriately assessed by data collection; and (c) the potential interpretive biases must be reduced using experimental design [11] (p. 3). We may argue that participant observation along the lines developed in the last decade in digital ethnographies may also diminish the various interpretive biases. The difference between the various perspectives regarding gamification, and particularly the gamification in e-learning, are discernible if one compares the various definitions of gamification developed before March 2020 with the definitions attributed to gamification after the quarantine was instituted. During the pandemic, we assert that the concept “gamification” was used in e-learning in a “narrow” sense: games are not used per se in order to facilitate the attraction of a student and stimulate the educational process. However, we consider that the games’ features used to attract students can be subsumed in three main categories: (a) game components, such as points, achievements, leaderboards, levels, etc.; (b) game mechanics, such as exploration, collection, competition, status acquisition, etc.; and (c) game dynamics, which include progression systems, narratives, constraints, and so on.

Incorporating game features into educational activities is meant to motivate students in order to enact a particular behavioural change [11]. More specifically, using gamification in educational contexts has the purpose of increasing engagement and students’ motivation, consequently entailing better skills learning and development [18,19].Bringing key game characteristics to various educational contexts intensifies the immersion into the activities and may offer students a holistic learning experience. Active learning strategies, such as using game fiction in online training and simulation, have an impact both at an individual level [20], and at a group level by providing enhanced learning results, if they are designed to encourage cooperation among group members [18,19,21,22].

The quantifiable results of gamification in educational activities and, in a broader sense, its effects can be analysed and described by the theory of gamified learning [23]. In a nutshell, this theory can explain the manner in which certain game features alter behaviours that have a distinguishable effect on educational results and interact with the curriculum in such ways as to noticeably alter its efficiency. One often mentioned study that seems to illustrate the applications of this theory has been developed by [24]. The authors performed an experiment in which they assessed how a point system and badge system caused a noticeable improvement of students’ grades as a result of preparation behaviours. The experiment’s results have shown that students’ self-testing activities based on the badge system determined a clear improvement of learning and the increase of academic achievements [24] (pp. 1–13).

Current Events Form Future Trends

The second part of this section’s title, which is a quote by [25] (p. 11), has the value of an epigraph for our article, because the evolutions from the past five years were further accelerated in 2020 and were a harbinger for 2021. This is reflected in the systematic literature reviews dedicated to the gamification in e-learning. A number of 15 articles containing systematic literature reviews were identified in the Web of Science and 30 articles in Scopus, according to the criteria that they report the effects of gamification on e-learning platforms. Two of the articles from the Web of Science were excluded due to the fact that they had no full text available or were not relevant. Furthermore, after removing the duplicates that were also identified in the Web of Science and the articles that had no open access text, two articles from Scopus remained for the review. All these articles were published until the 20 May 2022. It is important to stress the fact that we use the terms “e-learning” and “e-learning platforms” as umbrella terms, meant to partially include the extension of the term “online educational settings”. These articles address the following topics:

(1)

The use of augmented reality (i.e., AR) as a pedagogical tool in e-learning contexts “with a focus on the key benefits and challenges related to its adoption and implementation” [26] (p. 1). Augmented reality is reported to increase the motivation, engagement, attention, verbal participation, interactivity, concentration, knowledge retention, spatial abilities, etc. Conversely, the use of augmented reality as a pedagogical tool entails challenges such as: cognitive overload, resistance from teachers, inexperience in the use of e-learning platforms, the loss of time due to complex technologies, the costs of technologies, technological bugs, connectivity problems, etc. [26] (pp. 1–21).

(2)

The attempts to classify the gamification design frameworks emphasized the relatively low number of frameworks that are “designed for gamification within an e-learning environment” [27] (p. 1). Accordingly, it is necessary to develop a gamification framework meant to identify “games elements as motivational affordances that influence behavioural outcome of the learner” [27] (p. 1). The frameworks developed are meant to constitute the basis for an e-learning prototype, which can be integrated in online platforms such as Moodle. Similar frameworks can be used “to design, develop, and evaluate a gamified online tutorial for teaching systematic searching” on platforms such as PubMed and Scopus [28]. The development of new communication technologies and the social isolation caused by the pandemic have brought into the forefront the necessity to further expand frameworks that “can be used in any knowledge field that uses gamification” [29].

(3)

Gamification entails multiple meanings. Not all of them are associated with software. The major trends in the research of gamification before the emergence of the COVID−19 pandemic were correlated with e-learning, proof-of-concept studies in the ecological lifestyle, assisting computer science studies, and improving motivation [30] (p. 33). Particularly, in the subfield of e-learning assessment, before the pandemic, the most used approaches were: blockchain, social network analysis, gamification, process mining, etc. [31] (pp. 1–8).

(4)

The development of a theoretical model of gamification as a “technique to motivate learners and enhance their participation in learning activities by applying game elements and components” [32] (p. 1). This model is predicated upon behavioural science, and it is extended to games in general and entails several recurrent game design elements: points, levels/stages, badges, leaderboards, prizes, progress bars, storylines, and feedback. The study of the effects of the game elements and their applications are extensive. This is particularly valid in the case of the articles published between 2020 and 2022 [33,34].

(5)

The necessity to adapt gamification to the characteristics of the students on various e-learning platforms in order to reduce the negative impact of “same learning content, and static gamification” and to increase the level of motivation [35] (p. 187). Adaptive gamification is sketched in accordance with several elements found in research articles: research components, methods, and framework. It entails three components: adaptive gamification engine, adaptive component, and gamification display.

(6)

The use of smart digital assistants in e-learning inherently has gamification applications. The game-centric components commonly identified in various articles are virtual characters, chatbots, avatars, virtual exploratory environments, points, customization, real-world-inspired scenarios, and social engagement mechanisms [36,37].

(7)

The assessment of the virtual training processes entails the need to establish “what kinds of knowledge are suitable for this type of evaluation, and the challenges and possibilities of virtual tools” [38]. Gamification can be used for the assessment of problem-solving competencies and skill development [39,40].

(8)

The novelty of the study of gamification entails multiple meanings attributed to the term and may entail misunderstandings that emphasize the relatively incongruent character of the writings dedicated to it. While gamification per se has become a hot topic across a “wide range of industries”, it also can be “a potential disruptive force in education” [41] (p. 57), or at least as research findings “suggest that some common beliefs about the benefits obtained when using games in education can be challenged” [42] (p. 380). This entails several potential challenges to the idea that gamification is effective in itself, which are semiotic, epistemological, and methodological in nature. Gamification is not effective per se, but specific game design elements have specific psychological effects” [43] (p. 371).

(9)

The various applications of gamification in e-learning include numerous processes and implementations, mostly targeting e-learning platforms and/or education courses. For example, Ref. [44], compared (a) various educational games and (b) social networking approaches to gamification. They concluded that social gamification is the process that returned the most significant impact on students’ learning performances [45]. From reviewing these similar articles, we assert that it is possible to outline several conclusions: (a) the methodology used in most of the empirical settings is inadequate [36] (p. 83); (b) there is a deficiency of “true empirical research on the effectiveness of incorporating game elements in learning environments” [36] (p. 83); (c) the study designs tend to be heterogenous; and (d) there is a recursive need for further research generated by the small sample sizes employed in the studies [46] (p. 1). Nevertheless, as Alessandra Antonaci, Roland Klemke, and Marcus Specht have asserted “despite all these lacunae, gamification has been judged to be a field with potential” [47]. Accordingly, the level of students’ engagement and motivation can be increased by it.

3. Materials and Methods

In order to identify the relevant literature sources, we used two of the most known online research databases related to the social sciences and technology: Scopus and ISI Web of Science. We used a search system in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), which is represented and detailed below (Figure 1) [48,49]. It is one of the most used reporting guidelines for systematic reviews on a particular topic or on a number of related concepts. We reviewed the literature regarding gamification in e-learning over the last 10 years, emphasizing a comparison between the papers published before the COVID-19 pandemic and those published during the pandemic. In a nutshell, Figure 1 illustrates the identification, screening, eligibility, and the inclusion of the relevant articles in our systematic literature review, according to the PRISMA model. The sources were selected according to the following inclusion criteria: articles focused on game elements, articles that approach gamification with a focus on e-learning, articles focused on the factors of the effectiveness of gamification, articles published in peer-reviewed journals, articles published in English, articles published from 2012 onwards, research articles, and open access articles.

Conversely, we considered the following exclusion criteria: articles that approach gamification in relation to fields of study or scientific domains other than education, articles published in other languages than English, duplicate articles, articles published before 2012, editorial letters, book sections, book reviews in journals, not relevant literature reviews, not related to gamification but connected to educational or serious games, conceptual papers, and conference papers. Furthermore, the articles that presented gamification in general or did not focus on e-learning and did not contain real-world data were excluded from the sample. These exclusion criteria entailed the rejection of articles in the “screening” and “eligibility” phases. The research was undertaken until the 20 May 2022.

The analysis of the literature reviews presented above in the field of gamification in e-learning emphasized the fact that the number of empirical publications is relatively limited. Most of the articles based on empirical studies were published in the two years since the emergence of COVID-19. The keyword combinations for our initial query in the Web of Science Core Collection and Scopus were the same: “(gamification) AND (e-learning)”. This was further refined with the following syntax in the Web of Science: “gamification (Topic) and e-learning (Topic)” refined by “Document Types: Articles”, “Languages: English”, “Document Types: Articles”, “Open Access”. “Topic” represents, in the Web of Science, an umbrella term that includes the reference’s title, its abstract, the author’s keywords and the keywords plus. Furthermore, we have used an equivalent syntax for a second more advanced query in Scopus: TITLE-ABS-KEY ((gamification) AND (e-learning)) AND ((experim*) OR (evaluati*)) AND (LIMIT-TO (OA, “all”)) AND (LIMIT-TO (DOCTYPE, “ar”)) AND (LIMIT-TO (LANGUAGE, “English”) OR EXCLUDE (LANGUAGE, “Spanish”) OR EXCLUDE (LANGUAGE, “Russian”) OR EXCLUDE (LANGUAGE, “Portuguese”) OR EXCLUDE (LANGUAGE, “Arabic”)).

In the initial search performed in each database, we found a total 503 articles in the Web of Science Core Collection and 7239 articles in Scopus. After refining the search in each database using the second search syntax, we found 69 articles in English, which were open access in the Web of Science and 269 in Scopus. The data extracted were added to EndNote X9 in order to develop further analyses. By cross-referencing the results from the two databases, we removed 32 duplicate records. Afterwards, the 306 articles were screened by excluding conceptual and conference papers, and records that included an irrelevant literature review and those that were not related to gamification per se but were connected to educational or serious games. Thus, we identified 252 articles that were eligible. In the “Eligibility” phase, we have assessed the remaining full-text articles by excluding the 149 articles that did not meet the criteria specified above. Afterwards, we reviewed the remaining 103 articles by all fields in accordance with our research questions. We address the insights gained from our analysis pertaining to each question in the following paragraphs.

4. Results

(Q1)

What were the most used game elements in e-learning before and during the COVID-19 pandemic?

From the first query in EndNote X9, three articles explicitly contained the terms “gamification element” and/or “game element” and/or “element” in their abstract and/or their keywords. We undertook a full text analysis of the whole database because we wanted to ensure that we had not missed any relevant reference (see Appendix A). After reviewing the articles, we identified the most used game elements. Accordingly, we consider their relevance in the following paragraphs, both before and during the pandemic.

(a)

“Level”, plural “levels”, represents a game element associated with objectives, or in a broader sense, with goals. It represents a central game element used in gamification. As such, it was used in the scientific literature dedicated to gamification before and during the pandemic. The term, both in singular and plural, was included in 28 articles, out of which the majority were published during the pandemic. Before addressing the references to this element made before and during the pandemic, it is important to make some remarks. The term “level” entails multiple meanings. From a chronological point of view, two meanings are relevant for gamification as a process. However, they are not prevalent.

Firstly, it is used with the meaning of a unit of measurement used to quantify the experience of a player’s avatar or character. Generally, levels are correlated in role-playing video games with experience points (abbreviated as XP). There is a direct correlation between a character’s experience points and her/his level, which materializes in an increase in statistics: strength, dexterity, maximum life, magic, etc. Levelling up allows the player to develop new skills or advance the ones already acquired. Furthermore, levelling up typically allows characters to access new and more difficult areas and use more powerful items. Secondly, the term “level” may also designate a place available to the player. In this sense, in order to ascend to the following higher level, it is necessary to finish all the objectives from the current level, which may be accompanied by a progress bar.

Notably, the term is used with other meanings, such as playthrough level of difficulty, level of user’s engagement, level of satisfaction, level of competency/skill, level of success, etc. Out of the 28 articles that mentioned the term explicitly, six articles contained the term “level” with the first meaning specified herewith and three articles contained the second meaning: the level as a place. Additionally, we identified three articles that entailed the use of both meanings. In 17 articles, the term “success” was used either in a direct and/or an indirect relation with “level”, and in 15 articles, the term “skill” was associated with the same term. In two articles, the term was used in relation to knowledge. Three articles contained the expression “level of engagement”. Moreover, two articles mentioned the level of the user’s satisfaction. In two articles, the term “level” was used in the sense of positioning a character in relation to its potential or storyline. We compared the mentions of this game element from articles published before the pandemic with those published during the pandemic:

α.

Before the pandemic, the term is used in eight articles, which were published between 2015 and 2019. These articles included the term “level” specifically as a game element in order to designate a unit of measurement used (i) to quantify a user’s experience or (ii) a place within a game [50,51,52,53,54,55,56]. In one article, the term was used to designate both meanings in various contexts of discourse [57].

β.

During the pandemic, the term was used with one or both of the relevant meanings in 12 articles. Nine articles contain references to “level/levels” in relation to knowledge and/or experience [9,39,58,59,60,61,62,63,64,65]. In one article, the term was used in order to designate a specific place [66]. Two articles contained in different chapters adopted both meanings of the term [67,68].

(b)

“Challenge/Challenges” designate an umbrella term that can take many forms: quizzes and problems that must be solved, often in a limited amount of time. They are tasks similar to but shorter than missions. A challenge can be undertaken either solo or in team mode by the users. Furthermore, they are generally related with other game elements, such as specific levels or missions, achievements, and badges. We identified the use of challenges in 10 articles. When we compared the articles that were published before and during the COVID-19 pandemic, we obtained the following findings:

α.

Two articles included “challenge” as a distinct game element, associating it with rewards, personal growth, and missions [56,57]. Two other articles referred to challenges as game mechanics [55,69]. In the four articles mentioned, challenges are associated with game elements used to quantify the user’s progress on various platforms (e.g., points, badges, levels, etc.).

β.

Six articles published during the pandemic contained mentions to “challenge” with the above-mentioned meaning [34,70,71,72,73,74]. These articles entailed references to challenges as tasks used to increase the engagement of users on various e-learning platforms.

(c)

“Achievements” are markers of success that tend to have specific logos. Interestingly, the term “achievement” represents an umbrella term that is sometimes considered to be an imperfect synonym for badges and trophies. This game element has been described or explained extensively in four articles [52,70,75]. Both before and during the pandemic, achievements were associated with various types of goals, challenges, badges, and rewards. As a game element, it is correlated with engagement and the positive impact on the users’ educational practices [52].

(d)

“Aesthetics” is a term which designates the multiple graphic and sound elements that convey a specific message and combine themselves into a coherent interface, which elicit the players emotional response. While there were indirect references to aesthetic and/or artistic elements in several articles, there was only one article that specifically mentioned its relevance in relation to the user’s emotional reactions [56]. When we analysed the articles in which this element was approached, we found that [76], were using a definition formulated by [17]: “aesthetics are how the game makes the player feel during those interactions” [17] (p. 36). It is considered to be “the composite outcome of the mechanics and dynamics as they interact with and create emotions” [17] (p. 36). Allegedly, it has an important role to play in users’ immersion and involvement on different platforms.

(e)

“Dynamics” represents the user’s rule-determined interactions with the game mechanics. As a broad term, the “dynamics” of a game have also been applied to some e-learning platforms. However, as some authors emphasize, the implementation of game dynamics on an educational platform must be predicated upon expert guidance [56] (p. 1112). The term has an ambiguous character. In our review, we found four articles in which dynamics was mentioned, particularly in relation to game mechanics, storyline, progression, achievements, action points, etc. [56,77,78]. All the articles identified were published before the pandemic. All of them included correlations between game dynamics and game mechanics.

(f)

“Mechanics”, used in the plural, designates an ambiguous term which is often correlated with dynamics. Interestingly, in the reviewed articles, the term has in certain instances a singular meaning and in other instances a plural meaning attached to it. It is an umbrella term that includes other game elements and rules. Game mechanics include artefacts and/or tools that are meant to determine a meaningful response from the users (p. 36). Five articles include references to this game element. The review of these articles revealed the following aspects:

α.

In three articles written before the pandemic, “game mechanics” was used as an umbrella term which (i) was predicated on the meaning it was derived from or (ii) was based on the definition developed by [17].These articles correlated game mechanics with game dynamics [56,77,79].

β.

In the other two articles written during the pandemic, this game element was either correlated both with dynamics and aesthetics [73], or presented as a stand alone means for gamification [80].

(g)

“Rules” are normative assertions which prescribe and limit a game’s playability and limit the path that must be taken in order to achieve results [21,77]. At the time of our investigation, three articles contained data in which in-game rules were specifically mentioned [3,34,77]. As game elements, the rules are often correlated with other elements, such as goals, achievements, and rewards. Our review allowed us to find the following features associated with this game element:

α.

In the two articles published before the pandemic, rules were considered to provide the frameworks for educational activities, which can have an impact on emotional experiences and the users’ sense of identity [3].In another article, rules were associated with other game elements and circumscribed in the game mechanics in order to prescribe the users’ behaviour [77].

β.

In the article published during the pandemic, rules were directly connected with goals, without characterising them [34].

(h)

“Rewards” are considered by some of the authors to be a form of achievements. They can be planned as an integral part of the in-game learning experience. This is a feature that has been borrowed directly for online educational platforms. Generally, rewards are classified into variable and fixed, based on the criteria of their schedule. As such, they can be obtained after completing a determined number of actions. Alternately, they can be distributed more or less regularly at set intervals. Rewards are another way of providing motivation and recognition for the time spent, the effort allotted, and the skills that have been attained. We were able to identify eight articles that specifically approached rewards as elements used in gamification. These articles can be detailed in the following manner:

α.

Four articles published before the pandemic included “rewards” as an umbrella term for badges, achievements, and trophies [51,52,81]. It was correlated with goals and with the users’ ability to achieve a desired result.

β.

This game element is included in four articles published during the pandemic [34,59,70,82]. Rewards are approached in a systemic manner to cumulatively designate badges, points, medals, levels, advantages, achievements, etc.

(i)

“Badges” are a type of reward offered to users after completing a goal or an objective or attaining a certain accomplishment. In our review we identified 12 articles that mentioned this type of game element. Badges are associated with game elements such as goals, challenges, missions, awards, achievements, points, trophies, and other types of rewards. Using as a criterion the period in which the articles were published, we identified the following:

α.

Seven articles were published before the pandemic in which the term was used in order to emphasize its motivational functions and its goal-setting support [52,56,57,81,83].

β.

Five articles published during the pandemic expand upon the functions previously associated with this game element [44,68,75,84,85]. gives instructions and provides reputation systems and group identification. Accordingly, we state that the scope of this term was expanded during the pandemic.

(j)

“Leaderboards” are systems of social ranking, which allow each user to understand their performance in comparison to the leader and others. This game element has been investigated in seven articles. Three articles containing this game element were published before the pandemic [56,57,83].Interestingly, during the pandemic, there are four articles which included this term [44,82,85,86].

(k)

“Narratives” and storylines entail the use of stories in order to transmit various information to users. We found four articles that used the term “narrative” and “storyline” as imperfect synonyms. Two were published before [87,88] and another two were published during the pandemic [68]. Gamification is directly correlated with storytelling in one of the reviewed articles, which was published during the pandemic [80].

(l)

“Storytelling” implies using an in-game character or an NPC in order to tell a story. This element is related with narratives and storylines. We found two articles that specifically mentioned this as an element used in gamification. Both articles were published during the pandemic [68,74].

(m)

“Feedback” is the information offered to users as a mark of their progress, achievements, and other facets of their activities. Feedback can generally take two forms of information: direct and indirect. Interestingly, e-learning platform aspects can sometimes contain clues (i.e., information on how to solve an exercise or a test) that can also be described as a form of feedback. E-learning platform clues are somewhat similar to in-game clues (i.e., information regarding the quest solving algorithm from a game). Feedback can be used independently or it can be used in combination with other game elements: badges, likes, leaderboards, etc. While it is not exclusively a game element, we identified feedback as an element used in gamification in 16 articles. Seven of the articles were published before the pandemic [52,53,56,89,90]. Interestingly, the majority of the articles that include feedback were published during the pandemic [61,64,71,80,91,92,93,94,95].

(n)

“Progress bars” are graphic representations of the user’s quantifiable improvement. They can be correlated with other game elements, such as: feedback, challenges, levels, points, etc. The term is used in one article published during the pandemic [96].

(o)

“Trophies” are a type of game element that generally take the form of a reward. In two articles, the authors emphasized that trophies are personalized in order to emphasize the goal achieved or the completion of a specific challenge or problem. They are often associated with game elements such as achievements, time limits, tasks, badges, etc. As a game element, this term is mentioned in one article published before [56], and another article published during the pandemic [93].

(p)

“Timers” and the associated time limits are game elements that can be used to expound the effectiveness of a user. These game elements tend to be correlated with feedback, challenges, levels, badges, points, etc. The timer is often implemented for tests, quizzes, and examinations [97]. Basically, it is a method of counting the hours, minutes, and/or seconds allotted for the completion of tests and examinations [86]. Eight of the reviewed articles contained remarks about timers and time limits.

(q)

“Tasks” are game elements similar to missions. However, they are generally smaller in breadth and purpose. They involve specific actions that must be undertaken in accordance with specific goals or objectives. Accomplishing tasks is generally rewarded with achievements, badges, experience points, levelling up, tokens, loot items, in-game or virtual currency, etc. Tasks are game elements often used on e-learning platforms: 77 articles contained references to tasks. However, only some of them approach this term in order to designate elements that are subjected to gamification, both before and during the pandemic [54,65,68,87,88,96].

(r)

“Missions” are a more extensive form of challenges, which tend to be correlated with tasks, time limits, and goal indicators. They are sometimes equated with quests. In video games, missions entail at least one task. Their completion is rewarded with items, in-game or virtual currency, access to other game areas, badges, etc. Notably, when they are used on an e-learning platform, missions are a type of game element that is more heavily customized. The reason for the missions’ heavy customization resides in the fact that just as game missions are a core aspect of the game design, the missions developed for the e-learning platforms are essential for users’ motivation [98] (pp. 277–278). Missions were explicitly mentioned in five of the analysed articles. All were written during the pandemic and approached the ways in which missions can be used on e-learning platforms [8,74,75,84,92].

(s)

“Virtual Currency” is a game element used rather rarely on e-learning platforms. It emulates money in virtual environments. This element was emphasized directly in eight of the articles we identified. Users can be rewarded if an arbitrary mechanism is implemented on the e-learning platform. Virtual currency can be correlated with tokens. For example, some researchers have indicated that a “consensus mechanism” can be developed in order to develop smart contracts that award tokens upon completion. These tokens can be in turn transformed into virtual currency [59] (p. 6). The virtual currency or the tokens can be used for limited-access journal databases or bookstores [3]. Alternately, they can be stored or used for accessing other types of educational and/or research databases or resources.

(t)

“Tokens” represent a type of game element similar to badges that can sometimes be used as currency. They began to be used more and more in the past decade. Tokens tend to be mentioned in relation with badges and in-game or virtual currency. Interestingly, the gamification of an e-learning platform does not require the same level of token customization. This does not mean that the platform is less customized: the various type of tokens are game elements that need less customization [59].

(u)

“Personalising” features are game elements that can sometimes be available for the user from the beginning, or they can be unlocked. The user has the possibility to adapt the look and the outfit of her/his avatar. These features can be correlated with other game elements. Three articles contained mentions of this game element before the pandemic [99,100]. Another three included various references to it during the pandemic [64,71,85].

(v)

“Replayability” represents the number of chances awarded to users to repeat an action or a process, if the first try ended in failure. It is sometimes correlated with open-ended scenarios because they can enhance the relevance of replayability: “Once the students’ curiosity has been energized, a story with several branches can be replayed to experience different game resolutions” [74] (p. 10). The ability to replay certain types of resources, tests, and interactions on e-learning platforms was considered to be useful in articles published both before and during the pandemic for learning and is associated with task completion [101]. Open ended scenarios on topics developed for e-learning platforms can attract the users’ curiosity due to their different resolutions and can determine “spaced task repetition leading to enhanced retention” [74] (p. 10). This theme is particularly relevant during the pandemic.

(w)

Some e-learning platforms can contain “objective” or “goal” indicators, which are commonly combined with elements such as levels, missions, challenges, etc. Learning goals were specifically mentioned in two articles published during the pandemic [60,82].

(x)

“Competition” is not a specific game element. However, it has been usually described as taking place between teams or in a player versus player system. Accordingly, it was used in articles published during the pandemic in order to underpin and accelerate the learning processes [60,84,96,102]. It is associated with game elements such as achievements, badges, leaderboards, missions, etc.

(y)

“Avatars” represent the user’s version and extension within a game or e-learning platform. We identified three articles referring explicitly to avatars as a game element used in gamification [51,82,86]. Both before and during the pandemic, the term was used by various authors in order to designate the same meaning. Interestingly, the dialogical relation between identity and alterity was hinted at in multiple articles. Furthermore, from the reviewed articles, we gained insights regarding the avatar’s role in maintaining the privacy of users “by hiding their identity and activities from others” [86] (p. 413). Avatars are often confused with other game elements such as virtual characters. On the one hand, avatars are representations that are typically animated. They are controlled by users, and their gamification mechanics entail little modification when implemented in e-learning platforms. Avatars offer the simulated presence of a user in real time. On the other hand, animated virtual characters are intrinsic to any structured scenario [37] (pp. 10–13). Avatars tend to be used more often than animated virtual characters on e-learning platforms.

(z)

“Communication channels” are not game elements per se. However, they are used in-game to such an extent that they are intrinsically correlated with them in order to send messages between users or to allow them to discuss in real time. In our research, we found two articles that mentioned this element explicitly. These articles were published during the pandemic [86,103]. Communication channels are implemented in real-time in order to allow users to support each other by sharing information and competencies. These channels also allow users to contact teachers and trainers more easily, “offering them additional academic material or facilitating the permanent accompaniment of the teacher” [103] (p. 147). However, it is important to stress the fact that communication channels can be used for the transfer of personal purposes, which can entail the unintentional dissemination of the users’ private data. Considering the gamification elements used before and during the pandemic, one must take into account the fact that the more refined uses of gamification elements could also be influenced by the greater understanding of the concept in the educational context, or by the fact that teachers may have gotten better at designing or assessing initiatives based on gamification. Thus, the rise of the more refined uses of gamification elements could and should be studied in future research.

(Q2)

What were the effects of the most used game elements on learners’ behaviour on e-learning platforms before and during the COVID-19 pandemic?

In order to ascertain the number of articles that were relevant for our second question, we introduced in our EndNote X9 query of the abstract and the keywords the following terms: “game element”, “effect”, “behaviour” or “behavior”. We introduced the latter term in our query in order to also include the spelling used in American English. Accordingly, we identified the articles that were relevant for our second research question. Afterwards, we combed the full text of the 103 articles again, in order to ensure that we had not missed any relevant reference for our research question. In our investigation, we identified the following types of effects of gamification on users’ behaviour and characteristics: (I) engagement; (II) enjoyment; (III) motivation; (IV) performance; (V) attitude towards gamification; (VI) collaboration; (VII) awareness; and (VIII) satisfaction. Accordingly, it is necessary to expound the effects associated with each category:

I.

Engagement occurs when the users obtain pleasure or satisfaction from the interaction with the gamified e-learning’s mechanics. It entails the practices associated with increasing involvement, attention, and enthusiasm vis-à-vis educational activities.

α.

In articles published before the pandemic, researchers such as [104], asserted that the occurrence of engagement is directly related with the ease of attaining knowledge, which in turn facilitates the learning process [104]. This effect is determined by aspects pertaining to accomplishing tasks and the likelihood of receiving instant feedback. One of the main factors of engagement is the user’s level of activity towards a purpose. However, there is no direct connection between (i) attaining achievements, tokens, and/or badges; (ii) the users’ level of activity; and (iii) the user’s engagement. Antonaci, Klemke and Specht have mentioned studies illustrating the positive effects of badges on engagement [47].Accordingly, Ref. [105] suggested that rewards systems based on badges have a positive effect on the engagement of the students with special needs [105]. Conversely, Ref. [106] (p. 32) stated that there is no a correlation between (i), (ii), and (iii). Another indicator of engagement mentioned in multiple articles is the “social factor”. For example, the study [107] emphasized that when comparing three conditions of gamification on the Moodle e-learning platform (i.e., no game elements, game elements. and social game elements), the users in the social game condition were actually more engaged than those that were not using any game elements. Particularly relevant was their finding according to which social game elements contributed to retention and success [107]. Others, Ref. [108], developed research about the use of four types of feedback mechanisms in order to increase the users’ engagement: (1) gratitude feedback; (2) users’ past activities; (3) relative ranking; and (4) social ranking feedback. Study [108] concluded that providing feedback mechanisms, particularly social ranking feedback, can enhance engagement.

β.

These findings were further expanded in the articles written during the pandemic. Researchers developed an experiment-based study in which they determined that gamification and social networking were evaluated positively by participants. However, there were no notable statistical differences in the participants’ educational achievements or engagement [44]. Engagement was associated with motivation by promoting the users’ participation in reinforcing the taught materials and competencies [58,59,68,70,84,85,86,91,103,109,110]. Enjoyment is a critical effect of gamification, due to the fact that it involves the release of dopamine in the brain. This effect was emphasized particularly in articles published during the pandemic. Dopamine is essential for increasing motivation and attention [38]. Enjoyment influences the users’ e-learning practices: they can appreciate employing the gamified e-learning platform without understanding its potential for increasing performance. However, as [111] have shown, if the e-learning platform is useful but it entails no enjoyment factors, then the proclivity to use it will be affected [111]. Enjoyment was explicitly mentioned in three of the reviewed articles. In the articles published both before and during the pandemic, it was typically correlated with engagement, satisfaction, motivation, and performance [79]. One challenge when addressing enjoyment in the scientific literature dedicated to gamification is the fact that it entails multiple meanings. Furthermore, when compared with other effects, it appears in far fewer articles. It is a term that has been approached from several perspectives, hence, there are some discrepancies in its connotations. Enjoyment has an indisputable emotional dimension that is associated with the notion Mihaly Cziksentmihalyi coined as the flow state: a sense of achievement that ensues when the user’s skills are matched with the task’s challenges [112]. The game elements most associated with enjoyment are leaderboards, achievements, badges, points, and goals [38].

II.

Motivation is an effect determined by the collective efforts which are predicated on the intention of change. Out of the articles published both before and during the pandemic that we reviewed, 41 referred to motivation.

α.

A decade before the pandemic, Ref. [113] emphasized the fact that motivation is critical for meaningful learning. Motivation developed through gamification can take two forms: (I) extrinsic and (II) intrinsic [114]. The former, (I), is determined by factors such as rewards correlated with personal growth. The latter, (II), is based on the user’s interest and allows her/him to have an effective learning process [115]. For example, Ref. [106] examined whether or not rewarding participants with badges determined a change of the intrinsic motivation level. Furthermore, they wanted to ascertain if there was any positive variation between the initial level of intrinsic motivation and the level registered after the users interacted with a gamified platform. The two researchers claimed that there were no discernible effects determined by badges on the users’ level of motivation [106].It should be emphasized that the level of motivation is generally influenced by the type of game element used. Other researchers [116] developed a study in which they gamified a course pertaining to SPSS (i.e., Statistical Package for Social Sciences) on a Moodle-based e-learning platform. They [116], were able to prove that badges and leaderboards increased the motivation level of the majority of users. As an umbrella concept, it is associated with experience and engagement [42]. This effect was included in 18 articles published before the pandemic [3,51,52,56,57,69,76,77,79,81,83,87,99,101,117,118,119].

β.

During the pandemic, motivation was one of the most mentioned gamification effects and it was also correlated with most of the game elements used for gamifying e-learning platforms. Motivation was discussed either in relation to meeting educational objectives or satisfying needs. The two forms of motivation were further expanded upon. Interestingly, the number of articles that contain mentions and/or debates regarding motivation during the pandemic was higher than the number of articles published before it [9,34,58,59,66,68,70,71,72,75,84,85,93,98,102,103,120,121,122,123].

III.

Performance was a recurrent category both in the articles written before 2020 and in those written after the emergence of the pandemic. It denotes the efficiency of accomplishing a task on an educational platform. Other studies [102] compiled a series of factors that can have an impact on the users’ academic performance: (a) the quality of online teaching; (b) the correct revision of teaching to each e-learning platform’s peculiarities; (c) assessment methods; (c) communication channels; and (d) users’ boredom due to the COVID-19 lockdown. The two authors claim that there are no clear conclusions pertaining to a definite correlation between a game elements’ design, achievements, and the use of badges and performance [102] (p. 459). In the articles from the reviewed literature in which the term is mentioned, we identified a limited increase in the users’ performance. The exposure to gamified elements increased the initial users’ engagement. However, it did not substantially improve their performance neither before nor during the COVID-19 lockdown. These results are actually consistent with other studies, such as [124,125,126]. Conversely, there were articles in which elements such as feedback, points, badges, leaderboards, and a clear storyline were considered to have a positive impact on performance [47,127]. Performance offers relevant information, such as the points attained by users. This has a direct effect on stimulating social comparison and competition and determining a higher level of participation and engagement [108].

IV.

Attitude towards gamification was studied in several articles, both before and during the pandemic. There are no marked differences between the discourse dedicated to the attitude towards gamification contained in the articles written before and during the pandemic [34,52,62,72,128,129]. For example, Ref. [52] researched the impact of their gamification design on the attitude towards it. The game elements that are associated in the reviewed literature with appraising the attitude towards gamification are badges, leaderboards, trophies, achievements, tokens, tasks, missions, etc. The source [52] emphasized the positive effect on users’ attitude towards gamification.

V.

Collaboration between users has been studied in various educational contexts. Most articles that include assertions about collaboration were published before the pandemic [54,78,79,101,128,130,131,132]. There were two articles published during the pandemic that entailed mentions to collaboration [8,109].On an e-learning online platform, Ref. [131] researched the effects of a gamified dialogue system on users’ collaborative behaviour. Their study was focused on a sample of 249 respondents. The system analysed by [131] had the purpose of increasing the effectiveness of the users’ interactions through dialogue. The users had the opportunity to offer feedback for their colleagues’ comments through a system of “likes/ dislikes”. Furthermore, they implemented a system of rewards for the feedback of the contributing users. The researchers concluded that the discussion system contributed significantly to the users’ collaboration. It also improved the efficiency of the users’ communication by noticeably reducing the response times to their inquiries [131]. In another article, Muhammad Lukman Arifianto and Iqbal Fathi Izzudin assert that Discord can be used in order to facilitate group work, moderate interactions, and track the students’ activities. The virtual servers created by instructors “act as virtual classrooms” [109] (p. 181). The two authors grounded their assertions on a study that is fragmentary at best (i.e., published as an abstract only before the pandemic). They drew upon a study developed by [132], which emphasized the thesis that the implementation of Discord offers a user-friendly app which can also be used to moderate and to see the students’ involvement. Additionally, it allows instructors to assess the effectiveness of the students’ performance [132]. Interestingly, Arifianto and Izzudin clearly state that there are no studies directed towards observing the implementation of Discord from the user’s point of view. Collaboration was referenced in 47 articles. However, in half of them, the term was not mentioned in relation to users’ interactions [109].

VI.

Awareness was mentioned in seven of the reviewed articles [52,60,118,133,134,135]. In two articles published before the pandemic, there were remarks about the potential of gamification to generate “social awareness” [136].gained insights regarding the ways game elements such as “teams” enable participants to contribute to the users’ community building. The three scientists suggested that in smaller teams, communication is easier. The aforementioned game element had positive effects vis-à-vis learning achievements [136,137] explored whether interacting with a leaderboard (a) inspires social comparison processes or (b) causes the emergence of stereotype threats. Their results exposed the fact that “leaderboards appear to have inspired social comparison processes” to a greater extent than “stereotype threats” [137] (p. 74). During the pandemic, authors such as [74] included social awareness into the social–emotional learning competencies (SEL) by placing it on a “continuum of Awareness—Interest—Reflection—Decision—Action leading to the holistic, communal construction of SEL and sustainability competencies” [74] (pp. 6–7). They claim that integrating social–emotional learning (SEL) into distance programs based on e-learning platforms “can captivate young learners’ attention” [74] (p. 2). Interactive design predicated upon the social–emotional learning competencies is considered to be “an enhancing factor of e-learning” [74] (p. 2). As an effect of gamification, an increase in awareness can be determined by a procedure to “storify” online learning, in order to transform it into an engaging activity.

VII.

Satisfaction represents a state of ease and enjoying the experience derived from using a gamified e-learning platform. The authors of [117] (p. 31) adapted John Keller’s ARCS motivational design model [138], in order to emphasize one of the clearest representations of satisfaction. They claimed that satisfaction entails the positive reinforcement of students’ success on e-learning platforms, proving the fair treatment of users, encouraging learners to seek challenges, ensuring every user’s right of self-expression, and developing existing relationships through generosity. A total of 15 of the reviewed articles mentioned satisfaction. Seven of the articles were published before the pandemic [53,54,100,117,118,139,140]. In these articles, satisfaction was correlated with game elements such as challenges, tasks, achievements, badges/trophies, and tokens. Eight articles were published during the pandemic, marking an increased interest in it [10,70,75,82,84,97,141,142]. During the pandemic, the satisfaction of various e-learning platforms was a focus of research [70]. While satisfaction was generally considered to increase as a result of using gamification, there were few studies emphasizing a real connection between satisfaction and the increased performance during educational activities [84].

(Q3)

What factors should have been taken into consideration in order to develop effective gamification on e-learning platforms before and during the COVID-19 pandemic?

The most difficult to define query was dedicated to the third question, due to the fact that terms such as “factors” tended to be used by the authors with different meanings. Furthermore, the term “effective” was used with variations in its meaning, which made it even more difficult to compare the information from various articles. We chose to introduce in the EndNote X9 query of the abstracts and the keywords for the third question the subsequent terms: “factors”, “effective gamification”, and “e-learning”. Our query returned 33 articles that were relevant for our third question. A total of 14 of them were published before the pandemic [55,56,57,76,77,78,83,88,119,134,140,143,144,145]. Interestingly, 19 articles were published within a much shorter timeframe, during the pandemic [9,34,58,61,62,68,72,74,80,86,94,98,122,133,141,146,147,148,149]. Afterwards, we performed a full text analysis on the articles that were filtered in order to establish their relevance for our research question.

In order to confirm that the data analysis was correct and to avoid biases caused by personal interpretation, the processes described herein are predicated on the following definition of effectiveness: it represents the degree to which a user is successful in attaining a desired result by using an e-learning platform. While this definition is predicated on several articles that were written before and during the pandemic, it is not just an attempt to paraphrase a definition or a meaning attributed to effectiveness [143,144].

Our analysis of the relevant literature allowed us to determine that there are multiple ways in which the factors of effective gamification are described and/or defined. We admit that the process of designing effective gamified e-learning activities is predicated upon multiple factors. However, we cannot ignore the numerous meanings attributed to the term “factor” in itself. This fact makes it increasingly difficult to outline a unified framework containing these factors. The 76 articles that contained references to the term “factor”, both in singular and plural forms, were actually written from quite different theoretical and methodological standpoints. The body of knowledge coalesced around the factors relevant for the development of effective gamification belong to various research traditions and are predicated on theories belonging to various sciences. Accordingly, it is difficult to establish a coherent discourse regarding the meaning of the term “factor/factors”, not to mention the different relevance of various factors. For example, a number of articles addressed the factors that determine the users’ preference for “the use of gamification in virtual classrooms” such as: engagement, ease of use, motivation, and knowledge [103].These factors cannot always be considered relevant for the development of effective gamification of the e-learning platforms.

Another manner of describing the term “factor” is as intrinsically entailing a cumulative function in relation to users’ achievement: (1) exercise difficulty; (2) time required; (3) number of corrections; (4) degree of success; (5) the use of the metronome; and (6) the number of checks and submission attempts [64] (p. 10). This topic was addressed before the pandemic by [17]. Other authors underline the relevance of the social and contextual factors [57] (p. 32094). Thus, implementing the aforementioned game elements does not guarantee an increase in the users’ engagement, but the social and contextual factors actually play a role in gamification’s effectiveness [150].

In their attempt to review the extant empirical evidence for the effectiveness of gamification, Ref. [34], found that certain game “attributes” and elements tend to be prevalent in certain contexts. Conflict and challenge attributes were more often implemented in digital contexts than in analogue contexts. Furthermore, a combination of the two attributes was more often identified in the studies developed by researchers from Europe and the USA than those from Canada. Interestingly study [34], was not able to ascertain any direct signs “that the effects of game attributes were dependent on these contextual factors” [34] (p. 691).

One aspect that is often ignored when expounding the factors relevant for effective gamification is the fact that most of the studies that mention them are descriptive in nature. This is a feature of both articles published before and during the pandemic. Basically, by observing and describing the interactions on e-learning platforms, the potential scientific advantages that arise from using comparisons, in terms of groups, game elements, and contexts are lost. Additionally, it is very difficult to draw “strong” conclusions from an epistemological standpoint, because the majority of the studies were not only descriptive, but also confused: their theses entailed either attempts to justify or clarify whether or not “the reported effects on academic performance can be solely attributed to the gamified interventions due to the absence of (non-confounded) control-groups” [34] (p. 693). The researchers claimed that using the conflict/challenge and assessment attributes had a positive influence on learning. However, due to the fact that the studies conducted were small-scale, it is important to admit the possibility that positive results may have been exaggerated [36]. As a result, while the number of articles published during the pandemic dedicated to these factors is higher than the number of articles published before the emergence of the COVID-19 emergency, most empiric studies were on a small scale [147,149]. Alternately, they detailed systematic reviews which entailed references to articles that belonged to different research traditions and/or disciplines [34]. Notably, during the pandemic there were a number of articles that suggested new approaches, frameworks, and protocols for studying these factors [68,98,133]. These features also characterize the articles published before the pandemic. Thus, in the reviewed articles we have noticed an evolution of the scientific literature and not a revolution determined by the pandemic.

Several of the reviewed articles contained statements regarding effective gamification. Indrel Doney reviewed the literature dedicated to the approaches and the factors that increase the effectiveness and the engagement of adult learners through the inclusion of game elements into e-learning platforms [76] (p. 1). The trends delineated by [76] were intensified during the COVID-19 pandemic. He showed that game-based learning was one of the main trends in e-learning. This trend had already emerged by 2011. The prospects for game-based learning identified by [151] developed in the past decade into many collaborative activities in which game elements can increase the effectiveness of learning.

After reviewing the most widespread contributing factors to effective gamification on e-learning platforms, Ref. [76] lists the following: (i) challenge refers to the level of difficulty of activities; (ii) the level of difficulty, which is correlated with the ability to stretch the learner; (iii) competition against the platform and/or other users must be assessed in relation to challenges and goals; (iv) control is the ability of users to manipulate or influence game elements [152,153]; (v) feedback entails offering users the information on the choices already made and updating the progress achieved [154]; (vi) interaction represents the connection established with nonplayable characters, equipment [153], or other users [152], (vii) representation designates the users’ perceptions of the reality constructed in-game or on the e-learning platforms; (viii) rules and goals shape the framework upon which games and/or gamified e-learning platforms are transposed, or occur within a “fixed space and time period with precise rules governing game play” [155] (p. 448); and (ix) reflection offers users the opportunity to re-examine the process of learning through interactions mediated by e-learning platforms and the new knowledge and competencies obtained.

Effectiveness of learning is often correlated with the narrative used as a game element on specific platforms. For example, Ref. [68] (p. 76), stressed the role of stories “as a crucial element for the development of an effective gamification experience”. The researchers used the game element hierarchy developed by [156] in order to attain a comprehensive summary of the game elements that can be used synergistically [68] (p. 76). The effective gamification is predicated on the story in a ternary manner: (a) it “represents the gaming environment element” that coordinates their gamification model in a manner that allows one to achieve “effective pedagogical objectives”; (b) the story acts as the engine or the “driving system” of the gamification, considered as a process, because the curiosity of finding the story end is encouraged throughout the learning experience; and (c) the story evolves in parallel with the educational interactions of the e-learning platforms, because it is created “by the actions of the participant playing a specific role within the game” [68] (p. 77). This underpins one of the most important factors of gamification which is “the context of application” [157] (p. 3).

The most important characteristics of effective gamification are summarized by [94]. According to them, it needs to be “justifiable, adequate and easy to set up” [94] (p. 5). The separation of the educational content from the gamification layer allows the potential revision of gamification rules and/or educational activities. The scientists assert that there is a direct relation between the simplicity and reusability of “gamified programming exercises” on platforms such as the Framework for Gamified Programming Education (FGPE), which permits users to adapt elements from various widespread exercise formats [158].

The personality of every learner is impacted by each game element in a specific manner [15,159]. Furthermore, the learner’s personality, culture, age, knowledge level, background, languages, and intentions determine the effectiveness of her/his gamification.Others, Ref. [160], aimed to study the relation between game elements and some of these factors, such as personality, to gain insight into “the suitable effect for each character” [160] (p. 4050). Effectiveness of the educational outcome in its broad sense was investigated in relation to the students’ retention to join the course: the students’ behaviour to access the course, which is predicated on data statistics. The factors used by [160] in the design of their study were drawn upon “the big five factor model”, which in turn is based on the studies of [95]. These include extraversion, neuroticism, conscientiousness, agreeableness, and imagination/ openness. The researchers compared the data obtained from evaluating the activities of two groups of students: the first group used a gamified e-learning platform and the second used a non-gamified platform. They determined that initially, the first group had a more intense activity than the second one. However, by the third and the fourth week, the students’ activities started to decline in both groups. Interestingly, the “gamified” group’s activity actually had a sharper decline. Accordingly, the two scientists concluded that gamification can be used in order to improve the students’ activity in the short term, but it does not entail a greater effectiveness in the long term (2020, p. 4060). Thus, the gamification of e-learning represents a tool useful to engage students in e-learning, but in order to effectively enhance the learning experience, the presence of a teacher or trainer is recommended.

One of the aspects generally omitted in the literature dedicated to the factors of effective gamification on e-learning platforms is the fact that it can also have unwanted effects [161]. For example, competition, which is often mentioned in the articles dedicated to the above-mentioned topic, represents one of the most evocative game elements in this regard. It can affect the educational process by drawing the users’ interest towards winning instead of acquiring knowledge and competencies. This thesis was initially expounded by [162]. They asserted that the change in focus from task towards competition can affect the users’ performance and motivation to learn [162]. In a review developed by [34], dedicated to the use of simulators, it is suggested that most studies asserted that competition generates the increased use of simulators, without a discernible contribution to the improvement of learning outcomes [163,164,165,166]. This aspect was discussed mostly in the articles published before the pandemic.

In a nutshell, we found no grounds to assert that competition is a noticeable factor for developing effective gamification on e-learning platforms. Actually, multiple authors clearly state that repetition, as a factor that affects memory, is less effective when the users direct their focus towards competition [34,167,168]. Therefore, one potential consequence of e-learning gamification that is emphasized by [34], is the loss of interest in educational activities, if certain accomplishment markers are initially used and then removed: “giving rewards for a previously unrewarded activity can lead to a shift from intrinsic to extrinsic motivation and even loss of interest in the activity when the rewards are no longer given” [34] (p. 694). This is actually a seldom approached effect in the literature dedicated to the effectiveness of gamification: the over-justification effect [169,170]. This effect entails the change of users’ motivation drive from internal towards external sources, namely game elements. In turn, this engenders the possibility of disrupting motivation, when the game elements are removed and the associated activities are ending.

5. Discussion

The articles reviewed represent a plethora of gamification studies and analyses. They were examined across multiple topics associated with gamified activities on e-learning platforms. One of the strong points of our review is the comprehensive search approach, which entails using two of the most important databases: the Web of Science and Scopus. Furthermore, we used clear criteria for inclusion and exclusion.

Our article expounds a review on gamification of e-learning platforms centred on three questions. First, we wanted to ascertain the meanings associated with the game elements used on e-learning platforms before and during the pandemic. After describing them, we presented the effects of the most used game elements on learners’ behaviour on e-learning platforms. Finally, we reviewed the factors relevant for developing effective gamification on e-learning platforms. An intrinsic part of our article is establishing an accurate terminology that emphasizes the processual dimension of gamification, the integration of game elements on e-learning platforms, and the classification of the factors that may influence the effectiveness of e-learning platforms in terms of engagement and completion rates.

In our review, after we undertook the two stages from the “Identification” phase, we were able to screen a total of 338 records out of the initial 7742. We removed 32 duplicate records. Subsequently, we screened the remaining 306 records by title and abstract and excluded 54 records that were (a) conceptual papers, (b) conference papers, (c) literature reviews irrelevant for our research questions, and (d) those not related to gamification. In the “Eligibility” phase, we assessed the remaining 252 articles, eliminating those that expounded gamification in general, those that did not focus on gamification per se, and those that did not focus on e-learning or were published before 2012. Thus, we proceeded to do an in-depth analysis of 103 articles in accordance with our research questions. We have noticed a growing trend over the last two years. This is predictable given the unique events that happened since March 2020.

Furthermore, our research also contributes to sustainability through the fact that gamification elements can help learners personalize their process of learning, and by this, they can access and search only for the type of resources that are most relevant to their learning process depending on the type of subject they learn, thus reducing waste. Moreover, gamification elements can also help students improve or increase their social abilities by allowing them to collaborate and divide their tasks during the educational process. Hence, by collaborating and searching online only for the type of documents that are indeed relevant for the type of subject addressed, students also diminish their carbon footprint.

Our first question addresses the most frequently implemented game elements on e-learning platforms. The order of the most used game elements based on their frequency is the following: tasks, levels, avatars, challenges, badges, rewards, trophies, ranking systems such as leaderboards, feedback, missions, and timers (i.e., time limiting systems). In order to respond to the second question, we clustered the effects of various game elements into eight types: engagement, enjoyment, motivation, performance, attitude, collaboration, awareness, and satisfaction. We emphasised the game elements most often associated with each effect generated on users. Consequently, by relating the elements that are most frequently implemented on e-learning platforms and their effects, we were able to outline the following remarks:

(A)

The effects of tasks, levels, and avatars were noticed in relation to satisfaction and attitude towards the gamified e-learning platform. If the tasks are perceived as being too difficult, controlling, or restrictive, it can have a negative impact on both types of effects. Achievements, badges, and rewards in general can be used on e-learning platforms to delineate goals and/or provoke comparison, which positively affects performance.

(B)

Badges, rewards, trophies, and leaderboards are considered to have a positive impact on attitude toward gamification, performance, engagement, enjoyment [47] (p. 12). In our analysis we have concluded that leaderboards offer information about the points achieved by users. In a manner similar to badges and trophies, leaderboards can determine social comparison and competition and cause a higher level of participation and engagement [108]. However, it should be stated that the number of articles in which this element is directly connected with the above-mentioned types of effects is limited. Furthermore, while researchers like [47] describe such connections in unequivocal terms, the studies they mention, while varied, are limited at best. There is no holistic attempt to configure a methodological model meant to include the nature of the relations between game elements and their effects. Even though multiple models have been suggested from various fields, the epistemology of gamification is still an insufficiently explored domain. The map of gamification is not the territory: the latter is a vast domain with meanings developed in various fields of research that juxtapose, overlap, and more often than not diverge.

(C)

Missions, timers, and feedback have been associated with engagement, enjoyment, attitude towards gamification, motivation, performance, collaboration, and satisfaction. The effects of missions and feedback can differ depending on gender and personality. In a similar manner to points (i.e., especially experience points), these game elements improve comparison and subsequently improve performance. Interestingly, in the articles written both before and during the COVID-19 pandemic, there are clear mentions of these elements as increasing the users’ level of engagement in ways that determined them to undertake considerably more difficult tasks and challenges [171]. In turn, when their use is determined by engagement, tasks and challenges have a positive effect on satisfaction and attitude towards gamification. This chain connection between game elements and their effects, which in turn determine the pervasive use of other game elements that cause a second wave of effects, is rarely mentioned in the specialized literature.

(D)

Leaderboards as ranking systems, rewards, and the various types of points have been associated with motivation, particularly in its extrinsic form, attitude towards gamification, performance, engagement, and enjoyment. Furthermore, there was a reported correlation between the game elements that positively impact enjoyment and satisfaction.

(E)

It is important to stress the fact that some elements cannot be considered as belonging strictly to games, such as communication. However, they were relevant for all the reviewed studies: in the case of e-learning, communication tends to borrow the characteristics of game channels and dedicated Voice over Internet Protocols (VoIPs) that have a negligible effect on the capabilities of any specialized gamified platforms. Accordingly, the communication used on gamified platforms tends to offer numerous tools that were initially imagined for online games. Inherently, on any e-learning platform, communication relates to all the types of effects of gamification. However, the gamified communication channels facilitate an acceleration of online interactions and were associated in the reviewed articles particularly with performance, collaboration, and awareness [102,109,131,132,136].

Regarding our third question, which refers to the factors that should be taken into consideration for developing effective gamification on e-learning platforms, we gained several insights about the most relevant ones. First, the effectiveness of gamified e-learning platforms has been approached in many disciplines, which makes it more difficult to coalesce a working definition of effectiveness. Second, the multidisciplinary character of gamification entails several classifications of factors that have their terminological extensions more or less overlapping, which determines a higher level of vagueness from a conceptual standpoint and ambiguity from a methodological perspective. While we admit that some game elements are equated with certain factors of effectiveness, we have reservations vis- à-vis considering the former to be the equivalent of the latter. However, the process of choosing game elements for an e-learning platform should take into consideration their impact on the effectiveness of gamification.

External rewards are extensively applied on e-learning platforms. Admittedly, they are easy to implement and use, but can be perceived negatively by the users, because they can harm intrinsic motivation [169] (p. 153). Thus, gamification can have negative effects on the effectiveness of e-learning platforms. Correlated with external rewards such as badges is competition, which can hinder repetition. Subsequently, competition can have a negative influence on the development of effective gamification. Conversely, as a factor of memory retention, repetition is a factor that contributes to the effectiveness of gamification, as long as the game elements (e.g., challenges, tasks, missions, achievements, levelling up, loot items, etc.) that it entails are not too limited in number and features. Otherwise, by increasing the repetition of the above-mentioned elements, the effectiveness of gamification decreases as well.

The context of application is a factor that is mentioned by several authors [34] (p. 691), [47] (p. 13). It can refer to the type of virtual environment used. However, it also may connote to the fact that the designers must configure e-learning platforms by keeping in mind the fact that their creations should be in accordance with the needs of numerous users, each having their own culture, knowledge, background, language, intentions, age, gender [112].

The level of a challenge’s difficulty determines the motivation, engagement, and enjoyment of a user. It has a definite impact on the effectiveness of gamification: if the selection, ensuing design, and implementation of certain game elements (e.g., achievements, storyline, avatars, etc.) are inadequate, then they can have a detrimental effect on the users’ learning outcomes. It is relevant to properly evaluate the designers and developers’ intervention on e-learning platforms.

Feedback is an extremely important factor in the design and development phase of an e-learning platform, in the process of gamification, and during the users’ online activities. Interestingly, feedback is considered in some articles to be a game element, in others a factor, and there are articles in which it has a dual meaning. An increase in the feedback-oriented activities also engenders an increase in the level of interaction. This is regarded by some authors to be a factor that is partially overlapping with feedback and also has a role in the effectiveness of gamification: the two factors allow users who have different roles to share information and competencies. The interaction with other users is generally provided either through a chat function, forum/discussion board, audioconference, and videoconference. The reviewed authors stress the necessity to reflect upon the ways in which these mediums could and should be moderated and managed [76]. Reflection is actually another factor that is often associated with feedback. For example, ref. [172] stressed the importance of reflection by requiring users to expound their choices from a predefined list of options (2010). When gamification involves opportunities for users to share and discuss their experience by using questions as reminders, then reflection is beneficial for the effectiveness of the e-learning platform.

The users’ immersion on an e-learning platform is fundamentally determined by the limits of the game environment which are implemented. These limits are not necessarily negative for the learning process. The proper contextualization of information on a platform requires the use of visual and audio representation of concepts. It is important to stress the fact that in the case of this factor, there is a chain effect: the combination of particular visual, audio, and dynamic game elements leads to the appropriate representation of concepts, ideas, theories, methods, and values, which in turn increases the users’ immersion and can lead to an increase in engagement. Quite a few authors emphasize the fact that if the e-learning platforms’ designers and developers deliver information in multimedia formats in a stable and balanced manner and the users’ cognitive processing is not overloaded, then “representation” per se can be considered a beneficial factor for the effectiveness of gamification [76,152,153]. These researchers use a narrow definition of representation, which is predicated on the user’s perception of the gamified platform’s reality and its immersion levels. However, an aspect that should not be ignored is the way immersion is described as the user’s perception of her/his role within the interactions that take place on the platform. If the immersion is successful, the users will accept the illusion of the e-learning platform’s “reality” and engage with it [152]. If they do not accept it, then the users can become distracted, and this can have detrimental consequences on the effectiveness of gamification.

6. Conclusions and Future Work

From a processual standpoint, gamification is considered in most of the reviewed articles as propitious for improving the effectiveness of e-learning: learning behaviours are deemed to be strengthened. A trend that we have identified in the reviewed articles was their descriptive nature. The researchers seldom attempted to provide more than ambiguous descriptions and definitions for the concept of gamification per se. Equally before the COVID-19 pandemic and during it, the studies that have been undertaken are limited at best. They cannot convey to teachers and students with a clear empirically based list of recommendations pertaining to the activities and the game elements that should be implemented in order to increase the effectiveness of e-learning platforms. Furthermore, the negative effects of gamification are presented in the relevant literature to a lesser extent. While gamification was already a popular approach before the pandemic, the events that took place between 2020–2022 have increased its importance in relation to e-learning. Gamification has become an essential process for educational organizations and/or institutions to engage both teachers, academics, and students in a world where traditional forms of interaction and engagement have been severely limited.

Our article has several limitations. We have attempted to develop a comprehensive study of game elements and factors pertaining to the increased effectiveness of e-learning on the most relevant two databases dedicated to the social sciences in general and gamification in particular. Nevertheless, it is possible that we have disregarded certain articles which contain references to the same topic, because we have used specific keywords and keyword connections. Another limitation is the fact that we have omitted a part of the literature which contains keywords that were either too ambiguous or were returning irrelevant research articles in our queries. We tried to include the most relevant articles by drawing upon the insights provided in their framework by [11]. We wanted to have an extensive search, although we were mindful of the various meanings associated with “game elements” and “gamification”, particularly during the COVID-19 pandemic [34,173].

Another aspect that hinders the research dedicated to gamification is the fact that there is no agreement pertaining to the definition of the term “game”. In our article we suggest a meaning for “game”, “gamification”, and “game element” that can be used both for descriptive and argumentative studies. Our study emphasizes the necessity to clarify the terms “gamification” and “game element” from a semiotic standpoint. This is the reason why we present the meanings that were most often associated with these terms. Furthermore, we have attempted to offer a holistic perspective regarding the umbrella term, “game element”, emphasizing that it entails different meanings which are predicated on multiple research traditions. We bring forth in our paper a multitude of meanings associated with a wider ensemble of game elements than other papers.

One direction for future research that may yield interesting results should be expounding and/or developing the grounded theories that may analyse and explain both the beneficial and the detrimental effects of gamification on clearly delineated control groups in a diachronic manner. The existing literature dedicated to gamification can be used as a basis for an epistemological analysis predicated on the purpose of verifying and expanding the existing theories.

6.1. Gamification before the COVID-19 Pandemic

The review of the articles which were published before the emergence of the COVID-19 pandemic allowed us to emphasize the fact that in the past decade the number of elements used in the gamification of education has increased dramatically. Unlike most of the reviewed articles, we expounded a wide range of features that were or could be considered game elements. However, the body of knowledge that was developed between 2012 and 2019 is predicated on various disciplines. This means that reviewing the three research questions postulated in this article also entails a challenge from a semiotic and an epistemological point of view. The same (Q1) game elements, (Q2) effects, and (Q3) factors could receive different meanings and encompassing frameworks which are based on various research traditions.

The articles published before the pandemic contained various connections between game elements, their positive effects, and the factors for an effective gamification. However, our review emphasized the fact that most articles that approached game elements were narrative-based. Accordingly, it is necessary to emphasize the necessity of a closer alignment between the educational objectives and the way they are expressed on e-learning platforms.

In some of the reviewed articles there is no clear connection between the factors of effective gamification and its effects [3,55,76,77,78,88,134,143,144]. Moreover, the conclusions from some articles tend to emphasize the positive impact of game elements on the effects of gamification, without clearly establishing the scientific limits of the theories and/or the frameworks employed by the researchers [81,87].

6.2. Gamification during the COVID-19 Pandemic

The sharp increase in the number of articles dedicated to the gamification of e-learning platforms during the COVID-19 pandemic has accelerated the emergence and implementation of various theories [58,61,64,68,72,86,98,122,133,146,147]. However, this did not necessarily entail the identification of the most effective game elements and factors in order to improve the learning process through gamification on e-learning platforms. In the reviewed articles we found a trend regarding the multiple mentions of using storylines, challenges, badges, and leaderboards in order to create and maintain competition among users, which in turn may influence and increase the level of social interactions and the coalescence of communities. Nevertheless, while we admit that in the studied articles there are clear mentions of competition as influencing intrinsic motivation, there is no well-grounded reason to claim that it contributes to the effectiveness of the learning activities developed on e-learning platforms. Competition is considered in most articles a factor that improves engagement. Actually, in the relevant literature, it has a dual character: it is sometimes considered to be a game element and at other times, a factor of e-learning effectiveness.

Our review emphasized the fact that the number of articles published during the pandemic dedicated to factors of effective gamification and some game elements was higher than the number of articles written prior to the COVID-19 emergence. However, our findings denote more of an evolution of the relevant literature and not a revolution determined by the state of emergency. Furthermore, we have approached a large part of the game elements that were and still are discussed in the articles published on this topic. The emphasis during the pandemic was centred on the e-learning platforms used for educational purposes, not on serious games [38,59,60,66,74,75,86,92,94,174].

From the articles published during the pandemic, it is clear that the users’ perspective is predicated to a high degree on their background. The viewpoints of e-learning platform designers, students, academics, and scholars, in general, differ when addressing a “gamified platform” and a “game element”: meanings are imagined as being intrinsic to historical, cultural, and social circumstances. The introduction on an e-learning platform of an achievement system based on awarding badges may be considered to be a game element by some of the students. However, there is a clear possibility that other users may consider it a game in itself. As there is no consensus on the definition of “gamification” and “game element’, we noticed a high degree of subjectivity when we reviewed the various descriptions and the correlated interpretations of gamified interventions.

e-learning platforms have their own particularities that distinguish them from serious games and Massive Online Open Courses (MOOCs). This distinctiveness is a result of the audience it is tailored to, the technology used, and the degree of open source features it entails. The e-learning platforms’ designers and developers have a high degree of flexibility in terms of intervening, modifying, and customizing them, which in itself is a factor that can influence not only the mechanics and the dynamics used for gamification: it affects the effectiveness of implementation and the users’ learning process.