Towards High Usability in Gamified Systems: A Systematic Review of Key Concepts and Approaches

Abstract

Gamification is widely used in various domains as it helps increase user motivation and engagement. Therefore, it is important to ensure that the gamified system meets usability requirements and conforms to user experience and human-centred design guidelines, as poor usability may lead to undesired side-effects, such as a loss of user retention. Unfortunately, it is not entirely clear what usability guidelines are related to gamified systems and how they depend on the gamification target audience and on the gamified system application domain. This paper provides a systematic review of usability, user experience and human-centred design guidelines for gamified systems. A systematic search was performed in Web of Science, Scopus, IEEE Xplore, ScienceDirect, and ACM Digital Library databases, following the PRISMA guidelines. The results show that most of the recommendations listed in analysed papers are related to improving learnability of gamified systems. Furthermore, the system application domain is generally viewed as very important when making usability-related decisions. The key distinctive gamification application domains are medicine, education, and business. The findings may be useful to usability and gamification researchers, as well as the developers and owners of gamified systems who want to grasp the state of research into the specifics of gamified systems usability.

1. Introduction

Gamification encompasses multiple areas, such as technology, culture, society, healthcare, and economics, while helping make the reality more gameful to make humans more creative, skilled, productive, engaged, and motivated [1]. A gamified system is an information system which uses game principles in a non-gaming context for increasing user motivation and engagement. The purpose of a gamified information systems is to provide an experience and motivation similar to games, affecting the user behaviour [2]. Therefore, when developing a gamified system, one should strive towards not only a good functionality but also player engagement, as that is necessary to achieve the main goal of gamification—user motivation and active involvement in the activity that is being gamified [3]. This creates new challenges compared to the development of traditional information systems.

Gamification is increasingly associated with the concepts of user interface (UI) and user experience (UX) [4], a high quality of which is crucial in order to meet the rapidly changing user expectations and demands [5]. The user interface defines components that allow the users to interact and communicate with the interactive system in order to complete specified tasks [6]. The user experience, on the other hand, is the quality of the user’s interaction with the system, which includes user reactions and impressions [6].

The crucial component of user experience is usability, which is a software quality attribute that determines the extent to which the specific user in a specific context can use the system to achieve specific goals effectively and efficiently [7]. According to the ISO 25010 standard, there are six characteristics of usability, namely: appropriateness/readability; learnability; operability; user error protection; accessibility; and user interface aesthetics [8]. However, these criteria can be broken down into even finer characteristics, for example, learnability can be treated as dependent on predictability, synthesisability, familiarity, generalisability, and the consistency of the system. The generalised usability guidelines, also known as heuristics, typically build on such a set of usability characteristics. For example, the widely cited Jacob Nielsen heuristics for user interface design cover ten principles: visibility of system status; match between system and the real world; user control and freedom; consistency and standards; error prevention; recognition rather than recall; flexibility and efficiency of use; aesthetic and minimalist design; helping users recognize, diagnose, and recover from errors; and help and documentation [9]. At a more detailed level, there are hundreds of concrete usability recommendations and guidelines, which, in most cases, are of a universal nature.

Usability is important in the development of gamified systems, as it has a direct impact on the user engagement and the achievement of individual or organizational goals [5]. If the system does not ensure sufficient usability, it can cause problems and affect the user, for example, to annoy, distract, or mislead him [10]. This fact is among the basic principles behind the human-centred design approach, which incorporates the knowledge and techniques of human ergonomics and usability into the development of interactive systems, with the aim of making them usable [6].

Therefore, it is important to ensure that the gamified system meets the usability requirements and complies with user experience and human-centred design guidelines, as poor usability can lead to a loss of user engagement. In addition, if the system is insufficiently easy to use, the user may choose to avoid the system or use it improperly, even if other aspects of the system (functionality, security, speed of operation, and others) are very well thought out.

As far as usability in gamified systems is concerned, this field is under-explored and the systematic reviews on usability recommendations for gamified systems are lacking. However, there are several studies focusing on the user experience and user interface recommendations for learning-oriented gamified systems. Handayani et al. [5] presented a systematic review of the gamification UX and UI concepts, aimed at identifying specific design recommendations for the gamified e-learning systems. They specified three research questions about criteria, usability evaluation tools, and gamification elements for the e-learning application. The review shows that the gamification elements must be tailored to the target audience (gamified system users), while also stressing the importance of the user experience and user interface evaluation and testing with actual users. Mustafa and Karimi [11] reviewed the elements of gamification in gamified online learning and their impacts on user experience and engagement. They outlined four research questions related to targeted education level, gamified subjects, combination of gamification elements, and the influence of gamification on the user experience. The authors concluded that the elements of gamification could affect the online learning user experience and engagement.

As a result, even though concepts such as usability, user interface, user experience, human and user-centred design are used in systems development in general, it is not clear what specific usability, UI, and UX guidelines and methodologies would be relevant when developing new gamified systems and evaluating existing ones. Having such a tailor-made toolkit would allow those working with gamified systems to concentrate their usability-related efforts more efficiently. This review aims to collect and analyse usability and user experience guidelines, intended specifically for gamified systems, and identify their relations to the gamification target audience and application domain. The findings are relevant to usability or gamification researchers seeking to better understand the current trends and possible work directions. On the other hand, they are also valuable to the developers and owners of gamified systems. Here, there are several potential benefits, including:

(a)

improved ability to concentrate on the usability essentials when designing or developing a gamified system, instead of trying to meet all of the universal recommendations out there, thus saving time and resources;

(b)

increased likelihood that the usability improvement measures taken will indeed be beneficial to the efficiency of the gamification, rather than some other aspect of the gamified system;

(c)

improved ability to pick the usability recommendations based on the characteristics of the gamification in question, i.e., gamification domain and features of the target audience.

2. Methods

A systematic review is performed following the recommendations outlined in the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) [12] methodology.

This review aims to examine the state of research in usability and related issues in the context of gamified systems. Three research questions were raised to further specify the review goals:

• RQ1. What is the nature of specific usability recommendations for gamified systems?
• RQ2. Do gamified systems usability recommendations depend on the gamification target audience?
• RQ3. Do gamified systems usability recommendations depend on the system application domain?

The above questions influenced the key literature search criteria. Additionally, publication date, type, and other similar attributes were also defined. The full list of constraints applied to basic properties, as well as the titles and abstracts of papers, is given below.

Paper inclusion criteria:

• Publishing date 2008 to the first quarter of 2022;
• Document type is article and/or conference paper;
• Usability, user experience, human-centred design, user-centred design guidelines for gamified systems are given;
• Discussion is related to software gamified systems (i.e., not just gamification per se).

Paper exclusion criteria:

• Publishing language is not English;
• Paper is not in the field of computer science;
• Full text unavailable.

The literature search was carried out between February 2022 and March 2022 in five databases: Web of Science, Scopus, IEEE Xplore, ScienceDirect, and ACM Digital Library. The relevant papers were identified by searching in papers’ metadata (title, abstract, keywords), using keyword search queries. The keywords identified from the research questions were as follows: gamification; usability; user experience; human-centred design; user-centred design; guidelines; and their synonyms. The search queries and number of search results for each database can be found in

Table 1. Searches in databases.

Database	Search	Results
Web of Science	(gamifi* OR “serious game*”) AND (system OR software OR app*) AND (usability OR ux OR “user experience” OR “user interface” OR “human-centred design” OR “human-centered design” OR “user-centred design” OR “user-centered design”) AND (guidelines OR recommendations OR requirements)	158
Scopus	(gamify*) AND (system OR software OR app*) AND (usability OR ux OR “user experience” OR “human-centred design” OR “user-centred design”) AND (guidelines OR recommendations OR requirements)	286
IEEE Xplore	(gamify* OR “serious game*”) AND (system OR software OR app*) AND (usability OR ux OR “user experience” OR ui OR “user interface” OR “human-centred design” OR “user-centred design” OR “human-centered design” OR “user-centered design”) AND (guidelines OR recommendations OR requirements)	65
ScienceDirect	(gamification OR gamified) AND (system OR software OR application) AND (usability) AND (guidelines OR recommendations OR requirements)	57
	(gamification OR gamified) AND (system OR software OR application) AND (“user experience”) AND (guidelines OR recommendations OR requirements)	3
	(gamification OR gamified) AND (system OR software OR application) AND (“user-centred design”) AND (guidelines OR recommendations OR requirements)	1
	(gamification OR gamified) AND (system OR software OR application) AND (“user interface”) AND (guidelines OR recommendations OR requirements)	3
	(serious game OR serious games) AND (system OR software OR application) AND (usability) AND (guidelines OR recommendations OR requirements)	58
	(serious game OR serious games) AND (system OR software OR application) AND (ux) AND (guidelines OR recommendations OR requirements)	1
	(serious game OR serious games) AND (system OR software OR application) AND (“user-experience”) AND (guidelines OR recommendations OR requirements)	3
	(serious game OR serious games) AND (system OR software OR application) AND (“user-centred design”) AND (guidelines OR recommendations OR requirements)	1
	(serious game OR serious games) AND (system OR software OR application) AND (user interface) AND (guidelines OR recommendations OR requirements)	8
ACM Digital Library	(gamify*) AND (system OR software OR app*) AND (usability OR ux OR “user experience” OR “human-centred design” OR “human-centered design” OR “user-centred design” OR “user-centered design”) AND (guidelines OR recommendations OR requirements)	170

.

The following data about the identified papers were imported into the reference management tool: publication type; year of publication; title, abstract; author(s); publication language. Based on the eligibility criteria, the titles and abstracts were manually screened to determine which of the papers were relevant, irrelevant, or indeterminate and thus requiring full-text screening. The manual screening of both the abstracts and full-text was performed by two of the authors in a pair and, in case of disagreement, the paper under consideration was discussed to reach a consensus.

3. Results

The databases’ search resulted in the identification of 814 candidate records in total. Before screening the abstracts, filters were used to exclude the papers that were not articles and/or conference papers, were published before 2008, were not in the field of computer science, and/or were not in English. The selected references were imported into Zotero [13], a reference management tool, which was used to remove the duplicates and manage the list. The number of papers included and excluded in each stage is shown in Figure 1. A total of 396 records were removed before screening: 242 were removed using filters and 154 were duplicates. After screening the abstracts, another 177 records were removed. After full-text screening, 104 articles were excluded for the following reasons:

• did not provide usability, UX, or human/user-centred design guidelines;
• were not written in English;
• papers were review articles;
• full-text versions were unavailable.

Finally, an additional paper search was carried out following the process of snowball search for primary paper references and citations, and one additional paper that met the inclusion criteria was detected. In the end, 20 papers were selected as eligible for inclusion in the analysis.

After selecting the papers, the years and types of publications were analysed. The findings show that the selected papers were published in the last ten years, and even though the years 2017 and 2018 were the most productive, the totals were low. Most of the papers were conference proceedings (n = 12). The numbers of papers per year and per publication type are given in Figure 2 and Figure 3, respectively.

After analysing the relevant guidelines and recommendations for gamified systems described in the selected papers, it was discovered that the recommendation lists provided by the different authors overlapped heavily. The different authors would often mention the same principle or recommendation with the only difference being the wording. Therefore, to facilitate the analysis of the recommendations, an initial grouping was carried out. In the end, the thorough review of 259 guidelines and recommendations resulted in the list of 30 items or generalised recommendations R1–30, most of which were mentioned in one way or the other in more than one publication (see

Table 2. Recommendations found in the selected publications.

No.	Generalised Recommendations	Comment	Publications
R1.	Ensure user control and freedom	Concerns the availability of alternative paths towards the accomplishment of user tasks within the system and the means for the user to choose what fits him best at any given moment.	[14,15,16,17,18,19,20,21,22]
R2.	Provide feedback	Concerns the use of messages, visual or audio signals, progress indicators, busy indicators and other means used by the system to communicate its status or other relevant information to the user, especially in response to user input.	[14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33]
R3.	Use familiar vocabulary	Concerns the use of terminology that is known to typical users and represents the particular domain.	[14,16,30,31,33]
R4.	Ensure that available actions are relevant to goals	Ensuring that actions available to system users at any given point in time are in sync with the typical steps of the activity in progress and no distracting, misleading, or irrelevant options are presented, or they are clearly distinguishable.	[14,15,16,17,18,20,22,23,24,25,26,27,28,33]
R5.	Use modality principle	Concerns the use of narration instead of just text when conveying important information to inexperienced system users.	[14,16,17,21,30,31,33]
R6.	Provide help to users	Concerns the clarity and usefulness of instructions and hints presented by the system to the user, especially where the user must fill in the forms or carry out complex or important tasks.	[14,16,18,23,31]
R7.	Ensure onboarding	Concerns providing all relevant information about the task to the system user before he begins the said task.	[14,15,16,18,22,28,30,31]
R8.	Use information segmentation	Concerns improvement of the visual structure of textual or other type of information presentation.	[14,16,19,20,23,24,27]
R9.	Provide rewards	Concerns the availability of rewards system within the gamified solution. Rewards can be both digital (e.g., achievement badges) and material (e.g., money, candy).	[14,15,17,18,19,20,21,23,28]
R10.	Use signalling principle	Concerns adding visual cues to multimedia learning materials within the system to help users find elements that are relevant at any given moment fluidly.	[14,16,27,31,32]
R11.	Ensure consistency of elements	Concerns making sure that the same colour, visual style (e.g., raised rectangle), term, command, etc., carry the same meaning throughout the system.	[14,16]
R12.	Ensure aesthetic and minimalist design	Concerns limiting the graphical user interface to necessary elements only, both in terms of quantity, and graphical complexity or style.	[14,16,28,30,33]
R13.	Provide clear error and warning messages	Concerns the usefulness and clarity of error and warning messages displayed by the system to the user.	[14,16,18]
R14.	Provide challenges that grow with the user’s skills	Concerns ensuring that difficulty of tasks that system user must perform corresponds to the user’s ability level (i.e., is neither too difficult, nor too easy).	[15,18,20,22,23,27,28,29]
R15.	Ensure short response time	Concerns ensuring that there is no unwanted system lag or delays in response to user input.	[23]
R16.	Provide fatigue management	Concerns providing system users the tools to manage the speed of system operation and not enforcing unreasonable, fixed time constraints on user actions.	[18,23,31]
R17.	Provide personalization controls	Concerns availability of system customisation options in terms of looks, functional complexity, localisation (e.g., choosing language, measuring units), etc.	[17,23,28,29,33]
R18.	Clearly communicate progress	Concerns the availability, visibility, and clarity of user action or system process progress indicators.	[15,24]
R19.	Facilitate competition	Concerns the use of a comprehensive user ranking and encouragement system in a gamified solution.	[15,17,19,20,21,22,24,25,28,29,33]
R20.	Provide narrative	Concerns connecting user actions within the gamified process to a fictional story in order to increase the user engagement.	[15,18,22,28]
R21.	Ensure the process is enjoyable	Concerns the subjective emotional pleasure the user experiences (typically determined through testing with users) and is largely dependent on the way gamified process is organised.	[15,17,26,28,30]
R22.	Use badges	Badges can be both material and digital and are rewarded to users for certain commendable actions or achievements.	[20,22,24]
R23.	Use points system	Concerns the awarding/deduction of points for actions users make. Points are then used to rank the users.	[19,20,22,24]
R24.	Ensure error prevention	Concerns the use of graphical or textual hints in user forms and menus that reduce the risk of user errors (not to be confused with software bugs).	[26]
R25.	Ensure recognizability of functions	Concerns the use of graphical, terminological, and other conventions when displaying user menus, buttons, links, et al. that are both different from one another and well known to users of the system.	[18]
R26.	Facilitate virality	Concerns the ability to share experiences with other people thus making the gamified solution more widely known and appreciated.	[19]
R27.	Facilitate social interaction	Concerns the availability of means or tools for the users to communicate with each other either within the gamified system or otherwise (in team meetings, through message boards, etc.).	[15,17,19,21,25,29,30]
R28.	Ensure easy navigation	Concerns the use of clearly visible and understandable system user menus, links, buttons, as well as well thought-out tree of website pages and current user locations hints.	[14,16,31,33]
R29.	Allow using avatars	Concerns the ability to use fictional alter-egos, typically expressed in fictional names and pictures.	[21]
R30.	Ensure visibility of objects	Concerns facilitation of discoverability of required elements on screen by using well thought-out placement, size, colouring of buttons, form fields, messages, etc.	[30,31]

). It must be noted that the level of detail that the different authors provided in their recommendations varied significantly. Below are some of the recommendations and guidelines that were generalised into R1 and R2:

R1. Ensure user control and freedom:

• “Allow users to control of the application.” [14];
• “The system provides users with choices on what to do or how to do something, which are interesting but also limited in scope according to each user’s capacity.” [15];
• “Allow the children/user to be in control of the application through self-pace progression, navigation and support elements.” [16].

R2. Provide feedback:

• “The feedback is affirmative, positive in tone, short, precise, polite and non-offensive.” [14];
• “The systems always inform users immediately of any changes or accomplishments in an easy and graspable way…. Feedback always tells users where they stand and what is the path ahead for progression.” [15];
• “When a system provides feedback, users are aware of the impact of their actions. In a game this enables them to know when they succeeded or failed and why.” [17].

The list of generalised recommendations served as an input for the second phase of classification, as described further in this section.

The selected publications were grouped based on the target gamification domain and the audience outlined by the authors of the particular paper. The authors of seven papers did not define the characteristics of their recommendations’ target audience, while others concentrated on either children, the elderly, or adults (without defining whether such positioning includes the elderly) only (see

Table 3. Distribution of papers based on the gamification application domain.

Gamification Application Domain	Percentage	Publications
Education	6 (30%)	[14,16,19,20,25,27]
Medicine	5 (25%)	[23,24,29,30,31]
Sport	1 (5%)	[27,33]
Business	1 (5%)	[28]
Undefined	7 (35%)	[15,17,18,21,22,26,32]

for distribution data). Similarly, there was a good variety of gamification application domains that the presented recommendations were aimed at. Those included the development of gamified solutions for medicine, education, sport, and business. Only a minority of the analysed recommendations’ lists were positioned by the authors as applicable independent of the nature of gamified activity (see

Table 4. Distribution of papers based on gamification target audience.

Target Audience	Percentage	Publications
Children	6 (30%)	[14,16,20,23,24,25]
Adults	2 (10%)	[27,29]
Elderly	4 (20%)	[17,28,30,31]
Undefined	8 (40%)	[15,18,19,21,22,26,32,33]

). It was presumed that not defining the target audience and/or target gamification domain implied that these factors were deemed as having no influence on the importance of the defined usability recommendations.

It is important to emphasise that not all of the authors positioned their recommendations as aimed at improving the usability or user experience of gamified systems, even if most of the given recommendations could, in fact, be interpreted as usability recommendations. Instead, these authors claimed that their recommendations’ lists were intended for serious games developers and/or those striving to improve gamification mechanisms, user motivation, user engagement, adherence to the human/user centred design principles, or a combination of the above.

The relationships between all of the three mentioned aspects of how the analysed lists were positioned are given in Figure 4.

To gain a better understanding of the nature of the recommendations proposed in the selected papers, the generalised recommendations R1–R30 were further analysed one by one and assigned to one or more general classes, based on their nature. Two classifications were used to group recommendations, namely:

• Nielsen’s heuristics for user interface design [9]. Heuristics serve as the most general usability principles available. Each recommendation had to be assigned to one or more of the following heuristics:

N1. Visibility of system status;

N2. Match between system and the real world;

N3. User control and freedom;

N4. Consistency and standards;

N5. Error prevention;

N6. Recognition rather than recall;

N7. Flexibility and efficiency of use;

N8. Aesthetic and minimalist design;

N9. Help users recognize, diagnose, and recover from errors;

N10. Help and documentation.

• Usability characteristics defined in ISO 25010 [8]. Usability is a complex concept and its characteristics provide a more granular view. Again, each recommendation had to address one or more of the following characteristics:

I1. Appropriateness recognizability;

I2. Learnability;

I3. Operability;

I4. User error protection;

I5. Accessibility;

I6. User interface aesthetics.

The mapping to Nielsen’s heuristics was completed, based on the descriptions of the said heuristics given in [9] and its accompanying articles. Similarly, the mapping to the ISO 25010 usability characteristics was completed, based on the descriptions of these characteristics given in the standard and in the textbook literature on usability. Due to the lack of purely formal definitions, such mapping is deemed to be at least somewhat subjective, but this risk was mitigated by the fact that two of the authors of this paper have extensive experience in both the teaching of Nielsen’s heuristics and usability characteristics and in using them as classifiers for software usability problems.

Table 5. Mappings (denoted by ×) between collected generalised recommendations from papers and Nielsen‘s heuristics/ISO 25010 usability characteristics.

Recommendation	Nielsen’s Heuristics										ISO 25010 Usability Characteristics
	N1	N2	N3	N4	N5	N6	N7	N8	N9	N10	I1	I2	I3	I4	I5	I6
R1. Ensure user control and freedom			×										×
R2. Provide feedback	×	×			×				×			×		×
R3. Use familiar vocabulary		×		×							×	×
R4. Ensure that available actions are relevant to goals					×	×					×	×	×
R5. Use modality principle		×				×		×				×	×		×	×
R6. Provide help to users	×								×	×		×		×
R7. Ensure onboarding						×			×			×
R8. Use information segmentation												×
R10. Use signalling principle	×					×					×
R11. Ensure consistency of elements				×								×	×			×
R12. Ensure aesthetic and minimalist design								×								×
R13. Provide clear error and warning messages									×	×		×		×
R14. Provide challenges that grow with the user’s skills							×					×			×
R15. Ensure short response time	×												×
R16. Provide fatigue management			×				×								×
R17. Provide personalization controls			×				×						×		×	×
R18. Clearly communicate progress	×					×							×
R20. Provide narrative		×									×
R24. Ensure error prevention					×									×
R25. Ensure recognizability of functions		×		×	×	×					×
R28. Ensure easy navigation				×			×					×			×
R30. Ensure visibility of objects					×			×						×	×
Totals:	5	5	3	4	5	6	4	3	4	2	5	11	7	5	6	4

shows the mappings between the generalised recommendations collected from the papers and Nielsen’s heuristics, as well as the ISO 25010 usability characteristics. Note that the recommendations R9, R19, R21, R22, R23, R26, R27 and R29 were dismissed from further analysis as they concerned the methodological or social aspects of gamification and were not related to the usability of gamified software systems.

To better understand the importance of each of the usability classes, we traced back from the generalised recommendations to the pre-merger ones to have a look at how many individual recommendations of the given nature were mentioned in the selected papers (see Figure 5).

Although the totals given at the bottom of the Table 5 already provide certain hints, the summary given in Figure 5 clearly shows that the majority of the identified recommendations were aimed at improving the I2-learnability of gamified systems (n = 78), with I3-operability a distant second (n = 40). Speaking of the classes based on Nielsen’s heuristics, it must be noted that there were only 12 recommendations overall related to consistency and standards (N2) and 14 related to help and documentation (N10). The matters concerning user interface appearance or aesthetics seemed to be the least important to the researchers.

Table 6. Mapping (denoted by ×) between selected papers and selected usability recommendations’ classes.

Papers	Nielsen’s Heuristics										ISO 25010 Usability Characteristics
	N1	N2	N3	N4	N5	N6	N7	N8	N9	N10	I1	I2	I3	I4	I5	I6
[33]	×	×	×	×	×	×	×	×	×		×	×	×	×	×	×	15
[22]	×	×	×		×	×	×	×	×	×	×	×	×	×	×	×	15
[17]	×	×	×		×	×	×	×	×			×	×	×	×	×	13
[32]	×	×			×	×			×		×	×		×			8
[21]	×	×	×		×	×		×	×			×	×	×	×	×	12
[31]	×	×	×	×	×	×	×	×	×	×	×	×	×	×	×	×	16
[30]	×	×		×	×	×		×	×	×	×	×	×	×	×	×	14
[20]	×	×	×		×		×		×			×	×	×	×		10
[29]	×	×	×		×		×		×			×	×		×		9
[19]	×	×	×		×				×			×	×				7
[28]	×	×	×		×	×	×	×	×		×	×	×		×	×	13
[16]	×	×	×	×	×	×	×	×	×	×	×	×	×	×	×	×	16
[27]	×	×			×	×	×		×		×	×	×				9
[18]	×	×	×	×	×	×	×		×	×	×	×	×	×	×		14
[26]	×	×			×	×			×		×	×	×	×			9
[25]	×	×			×	×			×		×	×	×				8
[24]	×	×			×	×			×		×	×	×				8
[15]	×	×	×		×	×	×		×		×	×	×				10
[23]	×	×	×		×	×	×		×	×	×	×	×	×	×		13
[14]	×	×	×	×	×	×	×	×	×	×	×	×	×	×	×	×	16
	20	20	14	6	20	17	13	9	20	7	15	20	19	13	13	9	Totals

reveals a relationship between Nielsen heuristics, ISO 25010 usability characteristics and all 20 of the selected publications that helps to better understand the importance of these aspects to the different authors. One usability characteristic and four heuristics have dependencies with all of the publications, namely: N1-visibility of system status; N2-match between system and the real world; N5-error prevention; N9-helping users recognize, diagnose, and recover from errors; I2-learnability. On the other hand, three heuristics and one usability characteristic could only be mapped to less than half of all of the selected publications. Those were heuristics N4-consistency and standards (n = 6), N8-aesthetic and minimalist design (n = 9), N10-help and documentation (n = 7) and the usability characteristic I6-user interface aesthetics (n = 9). Additionally, it is noteworthy that only three publications have outlined recommendations related to all of the usability heuristics and characteristics.

After establishing the dependencies between publications and usability heuristics/characteristics, we wanted to understand the relationship between the way usability recommendations were positioned by the authors and the nature of these recommendations. Figure 6 shows how many of the selected papers mentioned at least one recommendation of the given nature and for a given gamification target audience or application domain. We can see, for example, that all six papers aimed at gamified systems for children, mentioned at least one usability recommendation concerning learnability, but only two of such papers mentioned the recommendation(-s) related to user interface aesthetics. This suggests that user aesthetics were considered less critical than learnability when developing gamified systems for young audiences. Note that there are only seven intersection points on the diagram, where there were no recommendations of the given nature for the specific target audience or gamification application domain.

To gain an even better understanding of these relationships, we again traced back from the generalised recommendations to the initial ones presented in the selected papers and repeated the calculations given in Figure 6. The result is presented in Figure 7. Here, we can see that the learnability recommendations were suggested considerably more often when talking about gamified systems for children (n = 31) than when giving recommendations aimed at gamified systems for the elderly (n = 13). Of course, one must not forget that there were six papers that talked about children-oriented gamified systems and four that discussed those for the elderly (Table 4). In addition, recall that the same recommendation could be assigned to multiple classes, depending on its nature.

4. Discussion

The results of the review presented above served as the input answering the research questions RQ1–RQ3. The key observations are presented below.

RQ1. What is the nature of specific usability recommendations for the gamified systems?

The authors of the selected papers positioned their findings not only as usability recommendations, but also as recommendations for building better, more serious games, ensuring a better user experience, higher motivation or engagement and better compliance with human/user-centred design principles. Despite that, many of the 259 identified guidelines and recommendations addressed the same issues and therefore a summarised list from all of the 20 papers included just 30 items, 7 of which were dismissed as not relevant to this analysis.

In terms of the nature of the recommendations, the variety was good, with all ten Nielsen heuristics and all six ISO 25010 usability characteristics, that were used to group recommendations, covered. The usability characteristic that was addressed the most was learnability, while the user interface aesthetics aspect turned out to be the least important for the gamified systems researchers. The examples of generalised recommendations aimed at improving the learnability of gamified systems were R2-provide feedback, R3-use familiar vocabulary, R4-ensure that available actions are relevant to goals, R5-use modality principle, R6-provide help to users, R7-ensure onboarding, R8-use information segmentation, R11-ensure consistency of elements, R13-provide clear error and warning messages, R14-provide challenges that grow with user skills and R28-ensure easy navigation.

As can be seen from the list above, many of the recommendations were what one could classify as universally accepted, e.g., using familiar vocabulary, providing good feedback, error prevention, information segmentation, ensuring short response times, etc. On the other hand, the authors often recommended using modality and signalling principles as well as ensuring onboarding (relevant information before the task) and providing fatigue management features (adaptable time limitations and speed of work). Such guidelines are not applicable to all software systems, but clearly should be taken into consideration when developing gamified ones. Overall, all ten of Nielsen’s heuristics were covered by the analysed authors in their recommendations, led by N5-error prevention (38 individual recommendations) with N4-consistency and standards coming last (12 individual recommendations).

Most of the recommendations provided in the selected papers were of a general nature, and concrete instructions that could be used by gamification software designers and developers were relatively rare.

RQ2. Do gamified system usability recommendations depend on the gamification target audience?

The majority of the selected papers addressed the usability of gamified systems aimed specifically at either children or adults (30% and 30%, respectively) indicating that age is an important factor when making the decision on how to build or improve gamified systems. Out of six papers presenting adults-oriented recommendations, half were aiming specifically at the elderly gamification audience. The authors of the remaining 40% of the selected publications did not specify any characteristics of the gamification target audience for whom they were providing recommendations. Other user characteristics that typically influence usability-related system development decisions, such as user goals, level of motivation, education or IT skills, were not mentioned.

However, upon further analysis, no drastic differences were found between the nondescript recommendation sets and those aimed at gamified systems for specific age groups. The recommendations concerning learnability were most often mentioned: for children n = 31 (5.16 per paper on average); for adults, including elderly n = 21 (avg. = 3.5); undefined n = 26 (avg. = 3.25). The user interface aesthetics turned out to be the least-addressed usability characteristic. This came as a surprise, as aesthetics have a big influence on the mood the system creates and thus one could presume that for gamified systems, which are of an inherently ambiguous nature and must balance playfulness and seriousness, this factor should be quite important. The principle of consistency and standards was also not stressed by the analysed authors, which may indicate that the gamified systems, as a relatively new class of software, does not rely so heavily on following recognized conventions. On the other hand, it cannot be forgotten that certain usability recommendations are considered to be universally applicable to all systems without exceptions, and hence they may have been deliberately omitted by the analysed authors.

RQ3. Do gamified system usability recommendations depend on the system application domain?

As can be seen from the data given in Table 3, gamification in the domains of education and medicine was discussed most often (30% and 25% of the analysed papers, respectively). The authors of 35% of the selected publications did not indicate any gamification domain(s) that their recommendations were for. Not surprisingly, the largest total number of recommendations were positioned as aimed at the gamification of education. However, similar to the question RQ2, the review results show that the gamified system application domain did not have a substantial impact on the nature of outlined usability recommendations. In all of the application domains (medicine, education, business, sports and undefined), the most mentioned recommendations were related to the learnability aspect of the systems’ usability: medicine n = 18 (avg. = 3.6); education = 30 (avg. = 5); sport n = 5 (avg. = 5). business n = 4 (avg. = 4); undefined n = 21 (avg. = 2.63).

Speaking of the limitations of the conducted review, the generalizability of its results is somewhat compromised by the fact that the analysis only focussed on research journals and conferences. The usability of gamified systems is a highly practical topic and therefore it can be discussed in books as well as online blogs, forums and similar non-academic sources, which were not the subject of this analysis. Furthermore, the number of papers that were selected was not high–20 in total. One could observe a spike in relevant research activity in years 2017 and 2019, but, overall, the numbers of publications on the analysed topics were consistently low. This may have compromised the analysis with regards to the research questions RQ2 and RQ3, as the differences or, more precisely, similarities in the nature of the various recommendation sets could have been due to chance.

5. Conclusions

In this paper, available gamified systems’ usability recommendations were overviewed through a systematic literature review. The review contributes to the area of gamified systems by providing a wider picture to usability researchers. At the same time, the results of the review can be valuable to gamified system developers when specifying system requirements, designing its user interactions or graphical user interface. Here, the summaries given in Section 3 can provide guidance on what precisely must be emphasised in order to expect a higher usability of the developed gamified system, both in general and based on its audience or application domain. Finally, the owners of operating gamified systems can use the findings to direct their system quality evaluation as well as usability (and, by extension, gamification efficiency) improvement efforts.

After conducting a systematic review of the papers from five research publications’ databases, it can be concluded that the number of studies addressing the usability or related properties of gamified systems is not high and the ones that are available typically do not go into fine detail on how to improve the system but rather point out general principles that should guide developers. It can be noted that most of the provided recommendations are related to improving the learnability of systems, while other usability characteristics have received significantly less attention, with user interface aesthetics being the least addressed. The key user characteristic that usability (or related) recommendations for gamified systems depend on is the age of the intended users. A more granular analysis of the differences between the usability requirements of different gamification target groups based on user goals, education, motivations and other characteristics is lacking. The gamified system application domain is generally viewed as very important when making usability-related decisions. The key distinctive gamification domains are medicine, education and business.

Since most of the recommendations outlined in the selected papers are not very detailed (i.e., they mostly outline principles rather than present concrete rules), more research is required to determine the exact differences between various gamification scenarios and develop a practical usability toolset for gamified system developers and owners.