Next Article in Journal
Cavitator Design for Straight-Running Supercavitating Torpedoes
Next Article in Special Issue
Elastic Simulation of Joints with Particle-Based Fluid
Previous Article in Journal
Balancer Genetic Algorithm—A Novel Task Scheduling Optimization Approach in Cloud Computing
Previous Article in Special Issue
User Experience in VR Fashion Product Shopping: Focusing on Tangible Interactions
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Designing Pedagogically Effective Haptic Systems for Learning: A Review

1
Department of Pharmacology, University of Michigan, Ann Arbor, MI 48109, USA
2
Robotics and Haptics Lab, School of Engineering and Technology, Central Michigan University, Mount Pleasant, MI 48859, USA
*
Author to whom correspondence should be addressed.
Submission received: 29 May 2021 / Revised: 16 June 2021 / Accepted: 17 June 2021 / Published: 6 July 2021
(This article belongs to the Special Issue Haptics: Technology and Applications2021)

Abstract

:
Haptic technology enables users to utilize their sense of touch while engaging with a virtual representation of objects in a simulated environment. It is a bidirectional technology in that it facilitates the interaction between the user and these virtual representations by allowing them to apply force onto one another, which is analogous to our real-world interactions with physical objects as action-reaction pairs. The sense of touch is a powerful and innate learning tool that we readily employ starting from very early ages as infants even before learning to walk. Therefore, it is natural that incorporating haptic technology into pedagogical methods has been an active research area as it has significant potential to enrich the learning experience and provide an engaging environment for learners. In this paper, we reviewed studies from various disciplines that incorporate haptics to increase the quality of teaching and learning while emphasizing the underlying cognitive theories. In that direction, we describe two of the most common cognitive theories, the Cognitive Load and Embodied Cognition theories, that developers use to support haptic technology’s implications and use in learning environments. We then explore the effects of haptic design on its current applicability following these two theories. Finally, we summarize the best design practices to develop haptic simulations for learning, address gaps in current research, and propose new research directions.

1. Introduction

In human development, touch is the first sense to develop [1]. Haptic technology is a bidirectional technology that enables its users to interact tangibly via touch usually within a virtual environment. The word “haptic” is thought to originate from the two Greek words haptikos and haptesthai, which mean “able to touch” and “able to lay hold of”, respectively [2]. In the last decade, haptic technology has moved into the commercial spotlight, as tech companies race to deliver virtual immersion to their customers via gloves, bodysuits, and controllers [3]. Haptic technology is popular for the potential it holds for the entertainment market; however, its applicability exceeds gaming. Haptic technology has proven to have effective applicability in pedagogy; from providing medical students with a realistic surgical training experience to keeping children more on task during an educational nature walk, the implementation of haptics increase positive learning outcomes across a wide-range of disciplines [4,5,6,7]. Because the efficacy of these studies across various disciplines have been determined in a mainly compartmentalized manner (i.e., with each study being analyzed on an individual level within the terms of its own discipline or setting), there is a need for a wide scope review that takes different pedagogical contexts into account as to determine key beneficial design features across disciplines.
The purpose of our review is to analyze what has proven effective in the realm of haptic education across fields of study. We approach this review from a cognitive standpoint, aiming to elucidate how haptic information is cognitively processed so that the optimal design of haptic systems for pedagogy may be determined. The theories that we are incorporating in this paper, Embodied Cognition and Cognitive Load Theory, are meant to serve as a psychological framework to both explain previous research and guide the future design of haptic technology for educational purposes. Furthermore, we aim to present optimal design characteristics for educational haptic systems by first exploring the cognitive principles that support and explain the learning that occurs through haptic interaction. We then examine the role these theories play in current studies investigating haptic efficacy and design across various disciplines. Finally, we address gaps in current research, speculate optimal design in accordance with embodied cognition and cognitive load theory, and propose novel avenues of research, which aim to further elucidate optimal haptic design.
The paper is structured to include an introductory description of the Cognitive Load and Embodied Cognition theories with provided references to further engage interested readers, an extensive review of the haptics applications that aim to facilitate learning within multiple domains, and a discussion of future research directions that could potentially help to advance our understanding of the underlying mechanisms that make a specific application successful in terms of its effectiveness as a learning tool. This review should also prove valuable as it synthesizes the best practices as categorized by the learning domain and corresponding discipline.

2. Theories of Cognition for Haptics Applications

The first learning theory we describe is the Embodied Cognition (EC), a theory that holds cognitive processes to be grounded in action. The second learning theory we include in this review is the Cognitive Load Theory (CLT), which posits human working memory to be limited, and states that these cognitive constraints should be taken into consideration when designing educational material.
Embodied Cognition asserts that one’s sensory and motor systems influence one’s cognition [8,9]. This learning theory posits physical activity to be a cognitive anchor from which one can easily understand abstract concepts in learning situations and proposes that the cognitive representations of symbols, such as letters or numbers, are rooted in our sensorimotor experiences. In other words, we understand abstract symbols in terms of our bodies and physical action [10,11]. A good example of this phenomenon in action is when someone is asked to try and understand a physical system. When asked to try and understand a physical system, such as the consequential movement of gears, people will most likely turn to their imagination rather than resorting to complicated inferences about the inner workings of the system [10]. It is asserted that individuals may off-load mental processes that would normally take up cognitive space through their interactions with their body or environment through the phenomenon of Embodied Cognition [12]. This can take form as one counting on their fingers, manipulating puzzle pieces to see where they may fit, or representing information via a graph, equation, etc. These actions free up more mental capacity for the subjects to use in order to solve or attend to the problem at hand, as opposed to holding the entirety of the information in their working memory. In other words, this off-loading of cognitive processes onto the environment or body is thought to lower one’s cognitive load, freeing up more mental capacity to attend to the task at hand [12].
John Sweller developed Cognitive Load Theory (CLT) in the late 20th century. CLT is premised on the fact that human working memory has limited resources [13]. Cognitive load, also referred to as processing load or mental effort, is a term used to describe the loading of one’s working memory in response to a particular task [14]. The goal of an effective education system is then to optimize this cognitive load for the learner as learning is considered to be hindered when the working memory capacity is exceeded during a learning task [15]. Cognitive load levels can be manipulated in part by interaction with the environment or the context of the material being presented. Since the theory’s inception, it has been amended to address evolutionary psychology and information processing discrepancies. Within the updated account of Cognitive Load Theory, there are two main information-type categorizations: biologically primary and biologically secondary [11]. Biologically primary knowledge includes information that the brain specifically evolved to process at an earlier time, such as facial recognition, speaking, or movement. An example of biologically secondary knowledge would be cultural knowledge, as humans did not specifically evolve in order to acquire it. At the heart of Embodied Cognition lies the notion that cognitive processes are ultimately derived from the mental processes that mediate goal-directed interactions between an organism and its environment [11]. As depicted in Figure 1, Embodied Cognition is a cognitive process that is accounted for under the “biologically primary umbrella”. It is asserted that biologically primary knowledge does not experience the same restrictions on working memory because humans have evolved to acquire this information over many generations, thus needing neither explicit instruction nor conscious effort to acquire [11]. It is speculated that learning processes that invoke embodied cognitive processes do not have the same working memory constraints, leading to better learning outcomes. The constraints on working memory are thought to be most prominent when trying to acquire novel information that we have not advertently evolved to acquire, which then requires instruction to be presented explicitly and/or didactically [11]. Because biologically primary information is thought to be less hampered by working memory constraints, it is expected educators attempt to capitalize on our evolution and utilize biologically primary information to assist in the collection of secondary information. A stark and common example of this is the use of gestures and object manipulation during a learning scenario; these physical actions serve as biologically primary scaffolds on which to ground biologically secondary information, therefore assisting in the acquisition of such information. Unsurprisingly, the use of gestures and object manipulation during class have been shown to have positive effects on learning outcomes [11]. Haptic augmentation offers a unique avenue of using biologically primary processing to teach biologically secondary concepts. In a study conducted by Glenberg et al. [16], digital and physical manipulatives were investigated to determine their effects on reading comprehension learning outcomes. Children in the experimental groups were asked to read a sentence and then act out the meaning via physical manipulatives (toys) or digital manipulatives (images on a computer screen). Compared to the control groups, the children in both of the manipulation situations performed better on tests of reading comprehension. In the previous scenario, the manipulatives served as a source of biologically primary information, while the text being read served as the biologically secondary information. It was thought that, through the interaction with the biologically primary information (manipulatives), the children were able to more easily understand the biologically secondary information (text) because of the lower constraints on working memory for the biologically primary information. Another interesting finding of this study was that there was no statistical difference in reading comprehension outcomes between the physical manipulative and digital manipulative group, suggesting that virtual manipulation of objects in the study proved as effective as physical manipulatives at providing a source of biologically primary information.

3. Haptics Applications for Learning

Haptic simulations have proven to improve learning outcomes across many disciplines and contexts including, but not limited to, STEM, medicine, visual disability instruction, and language learning [4,5,6,7]. However, a holistic understanding of effective design factors for haptics across disciplines has yet to be elucidated as many previous studies have investigated haptic pedagogical efficacy in a primarily compartmentalized way. In this section we will be exploring current haptic efficacy across four disciplines in relation to the aforementioned psychological theories of Embodied Cognition and Cognitive Load Theory. STEM, Medicine, Visual Disability Instruction, and Language were chosen as categories to perform the literature review due to the different pedagogical nature and contexts they present to the learner, thus enabling us to achieve a broader understanding of general effective haptic design features across various fields of study.

3.1. STEM

The STEM field is composed of disciplines that require abstract thinking and visualizing. Certain disciplines within the STEM field, such as biology or chemistry, require conceptual understanding of the microscopic and/or atomic world, which cannot be seen with the naked eye. The STEM discipline can largely be classified as biologically secondary information, according to the updated Cognitive Load Theory [11]. The use of haptics can provide an embodied avenue of interacting with “invisible” subject matter in a real and tangible way. This form of embodied learning using haptics provides a novel way for learners to engage with biologically secondary information via biologically primary terms through their physicality and innately understandable sensorimotor processing [17], thus improving learning outcomes.

3.1.1. Physics

Haptic simulation is a unique tool for teaching traditionally literature-heavy subjects, such as physics, because it allows for interaction with subject material in an embodied fashion, which, according to theories of Embodied Cognition and Cognitive Load Theory, can help facilitate positive learning outcomes [11]. Haptic feedback provides a biologically primary avenue through which biologically secondary/abstract material can be comprehended more easily [11,12]. One such study that seems to corroborate this phenomenon is an educational study that considers the effectiveness of a haptic augmented simulation in learning physics [4]. A group of 220 fifth graders without previous exposure to physics education were divided into 3 different groups receiving different simulations: Force and Kinesthetics, Kinesthetic, and Non-Haptic. The Force and Kinesthetic group received force feedback commensurate to the virtual task being performed via a joystick (Microsoft sidewinder). The solely Kinesthetic group operated the joystick but did not receive force feedback. The Non-Haptic group used a computer mouse to complete the tasks, and did not employ/receive kinesthetic or force feedback. The results found that students in the Force and Kinesthetic (haptic) group performed better on tests of recall, inference, and transfer, alluding to better material comprehension and ability to transfer their schematic understanding to comprehend the core physics concepts [4].
One’s physical body has long been known to provide a familiar space to understand abstract concepts through embodied processing. Physics equations and questions may become more than arbitrary representations and can become more easily imagined and used to infer further conclusions. This concept is supported by a study completed by Kontra et al. [18] in which researchers investigated the impact that physical experience had on science learning. Students were divided into two separate groups: one group physically experienced the phenomenon of the physics concept they were learning, angular momentum, via tactile interaction with a bicycle wheel apparatus and one group merely observed the same phenomenon. The students who physically interacted with the angular momentum apparatus scored higher on average on the post-tests measuring comprehension. Individuals in the groups were scanned via fMRI during the training session. The groups were also scanned during their post-test. It was found that the group of students physically interacting with the angular momentum apparatus experienced increased neural activation in sensorimotor areas M1/S1 during the training and post-test. It was also found across groups that activation of these areas correlated with better test scores. These findings provide researchers with a preliminary causal model for how physical experience enhances understanding of science, and further substantiates the theories of how embodied experiences can play a significant role in creating effective learning environments [18]. Experiencing educational material via physical experience and object manipulation is a great example of how biologically primary processes can be utilized to help facilitate learning of biologically secondary knowledge detailed in Cognitive Load Theory [11].

3.1.2. Biology

Frequently, learning biology requires understanding concepts that are invisible to the naked eye and impossible to experience via touch. Attempting to learn about “invisible” concepts can initially be abstract and hard to visualize for some learners. By capitalizing on the principles of Embodied Cognition, haptic technology provides a tangible solution to this problem by enabling the user to physically interact with micro-scale learning material and create a perceptually rich mental model. A study conducted by Jones et al. investigated the ability of haptic augmentation to influence learning outcomes in students being taught microbiology [19]. Three types of haptic and non-haptic interface devices were compared in their teaching ability as opposed to traditional instruction: the PHANToM haptic joystick (a sophisticated haptic desktop device), Microsoft’s Sidewinder (a haptic gaming joystick), and a classic computer mouse (no haptic feedback). In [19], the PHANToM haptic joystick provided the user with the most immersive haptic experience due to the higher degree of sensitivity. The other haptic device that was compared in the study was the Microsoft’s Sidewinder, a less-expensive joystick initially developed for use in commercial gaming. The PHANToM device provided a robust 6-degrees-of-freedom movement while the less expensive Sidewinder provided 2-degrees-of-freedom. A computer mouse was used as the nonhaptic interface. Cognitive and affective measures were used to determine student learning differences; upon completion of the three different instructional models, tests measuring students’ knowledge of content and attitude toward the learning modules were employed.
Students who had experienced haptic feedback stated that they had “participated more fully in the investigations than those students who had only visual feedback” [19]. It was shown that students who had received haptic feedback used more analogies to describe the learning content during following discussions compared to the students who only received visual feedback. Furthermore, the students receiving haptic feedback also included more tactile-related terminology in post-experiment discussions. This finding suggests that the students who received haptic feedback may have had more embodied experiences with the learning material [19]. Interestingly, an increase in the sensitivity of the haptic tool positively correlated with the number of tactile terms students used when describing a virus. This data suggests that the sensitivity of a haptic tool may directly affect the level of embodiment experienced by the user, potentially leading better learning outcomes [11]. The researchers attribute the students’ descriptive differences to “the more perceptually immersive environment”, speculating that the “perceptually immersive environment not only makes the material especially interesting and engaging but also improves the students’ ability to personally connect with material and construct mental models of the abstract concepts based on actual concrete experiences” [19]. Another study completed by Jones et al. [20] explored how the use of a haptic controllable atomic force microscope, called the Nano Manipulator, influenced high school students’ understanding of viruses. The nanomanipulator provided the students with tactile feedback from the properties of the microscopic object being studied and allowed the students to investigate and manipulate objects that are nanometer-size. Adding haptic feedback to traditional microscopy allotted the user with an ability to interact with nanoscale objects in a physical, hands-on way. Results concluded that through the haptic microscope experience, students’ conceptions of the microscale adenovirus shifted from 2-D representations to resemble more 3-D-like representations, suggesting a better understanding of adenovirus morphology [20]. These results demonstrate the power of haptic technology for teaching microscopic and nanoscale morphology; with previously abstract concepts becoming tangible to the user. The two studies conducted by Jones further seem to corroborate the ability of haptic augmentation to provide a biologically primary method of interacting with the initially abstract biologically secondary learning material, and that embodied interaction can lead to improved learning outcomes for the users.
Aside from providing users with an embodied avenue for interaction with subject material, it is thought that haptic technology can also reduce cognitive load associated with certain tasks. For example, Schonborn et al. explored the relationship between students’ modality of interaction and learning biomolecular specific-binding concepts. Specific binding is a foundational, yet often times complex, concept in biology. In this study, Schonborn et al. assigned twenty students to either a haptic or non-haptic condition where they were asked to complete a docking task that resulted in the best ligand-protein binding conformation [21]. It was found that the group of students receiving haptic feedback produced a tighter constellation of the ligand- protein molecule pair, performed half as many interactive behaviors, and had a 15% learning gain, on average, between the pre- and posttest item that measured conceptual understanding (compared to an average learning gain of 3.5% for the non-haptic group). Schonborn et al. suggested that by being able to fully “feel” the biomolecular docking, cognitive load was redistributed from the visual working memory to the haptic working memory, which may have accounted for the faster docking times, lower number of interactive behaviors, and higher learning gain for the haptic group. However, an interesting result to note is that there was no statistical difference in final docking accuracy between the haptic and non-haptic group. Schonborn et al. attribute these positive learning results to reduced strain on visual working memory (reduced cognitive load via visuo-haptic coordination) and inferences gained through embodied cognition. Schonborn et al.’s theoretical approach is in line with the assertion of Cognitive Load Theory that states “Biologically primary processes can help facilitate the acquisition of biologically secondary knowledge” [11]. In the investigation by Schonborn et al., the haptic interaction can be thought to serve as the biologically primary method of interaction, while the biomolecular docking task and related concepts can be considered the biologically secondary knowledge the users hoped to learn. The resulting learning gain of 15% between the haptic and non-haptic group may corroborate this theoretical cognitive mechanism.
Haptic technology can provide a novel and engaging way for students to interact with their learning material. A pilot study was conducted by Hightower et al. [6] regarding an application called Haptic Explorers. A haptic mobile device’s usability was tested by children attending an ecology summer camp. Each child received their own haptic mobile device that worked as a haptic augmented notebook. The children were led on nature walks and divided into two groups with different learning objectives. Throughout the nature lesson, the children were instructed to take notes on relevant material. Half of the students had devices that included haptic augmentation. It was found that the group with haptic augmentation participated in more “science talk” and exhibited more “on task” behavior than the control group, indicating that haptic incorporation can increase learner engagement with material and help keep learners on task [6].

3.1.3. Mathematics

A close relative of the theory of Embodied Cognition, Nathan and Walkington [22] propose a theory of Grounded and Embodied Mathematical Cognition (GEMC). GEMC asserts that a person’s actions serve as inputs capable of altering cognitive states and views mathematical communication as a “multi-modal discourse practice, rather than a formal, written, propositional form” [22]. Studies supporting their theory have found that gesture and directed action can significantly improve learning outcomes. They define directed actions as: “physical movements that learners are instructed to formulate by some kind of pedagogical agent” and define gestures as: “spontaneously generated movements, often of the hand, that accompany speech and thought”. According to Moyer-Packenham et al. [23], a network of conceptual metaphors is the basis of mathematical knowledge. Moyer-Packenham et al. define conceptual metaphors as: “Cognitive mechanisms [that are] used automatically and unconsciously, having abstracted patterns of inferences from physical experiences”. On the premises of Embodied Cognition and Cognitive Load Theory, it is thought that mathematical connections are made when a physical action is connected to an abstract mathematical concept and that physical actions can facilitate the learning of biologically secondary information (mathematics).
According to these principles, mathematical pedagogy should be developed to incorporate physical ways for students to interact with abstract mathematical concepts, which emphasizes a demand for haptic intervention in mathematics education. Capitalizing on embodied cognition research, Davis et al. [24] designed a haptic-supported learning environment and investigated the ability of this system to teach trigonometry concepts to four high school students in a preliminary study. Their research objective was to determine how haptic feedback might impact learning and they propose a theoretical design framework for educational haptics. Davis et al. collected student data concerning three metrics: (1) student interactions with the software, (2) student interactions with the haptic device, and (3) student drawings of the learning material, graphs of sine and cosine, which served as a measurement of conceptual understanding. The haptic device used was created from an open-source kit, called Hapkit. Their haptic device was programmed to be used with a program called Trigonometry Explorer, where the student uses the Hapkit as a joystick to move across the graph of a trigonometric function. The users “feel” the graph by receiving force feedback from the Hapkit joystick depending on the position the user is at in the graph. Results analysis indicated that the haptic feedback helped the students make connections between the different representations for sine and cosine, and that the effectiveness of haptic feedback on learning was tied to the students’ ability to interpret it [24]. For example, one student named Juan systemically manipulated the Hapkit and proposed a hypothesis for how the haptic feedback was related to the sine graph: “So when it’s doing this and it gets harder then you’re going up higher and higher; then you start getting here and it starts getting easier as it’s going down then you start getting around here again and it starts getting a little harder; at that point you’re going up again” [24]. Juan’s personal hypothesis was in line with the force feedback presented by the haptic joystick, the minima and maxima of the graph were located at the two points where the force generated by the Hapkit was highest. When Juan was asked to draw the graph, he returned to the Hapkit several times before completing the task. Juan drew the graph of sine correctly on his first try. One of the other student participants in the study, Manuel, had a harder time interpreting the haptic feedback. Unlike Juan, he did not appear to be making the connection between the haptic feedback and the graph of cosine, saying “I don’t know. I know how it looks, cosine, I’m just seeing how can this help it, how’s it going to help a person understand how cosine works” [24]. When asked to draw the graph of cosine, Manuel failed to get the drawing correct. After his first failed attempt, Manuel was allowed to refer back to the Hapkit once more. After using the Hapkit a second time, Manuel’s drawing was improved, but still contained errors. The results of this preliminary study indicated that learning gains in part depend on the user’s ability to interpret the haptic information in a meaningful way. This illuminates a possible challenge of supplementing abstract learning material, such as trigonometry, with haptics. Creating the link between biologically primary and biologically secondary knowledge may heavily rely on individual interpretation. Haptic design should account for possible individual differences in interpretation and aim to be as generally intuitive and meaningful as possible to the learner.
Intuitive design can be determined via further usability research. A study conducted by Yiannoutsou et al. [25], investigated young children’s use of a haptic device to explore 2D and 3D shapes through a serious game designed to support geometry learning. The research aims to inform haptic educational game design by understanding the usability of the device and the ways that tactile experience shapes children’s concepts of shapes. Twenty students aged 7–8 participated in the study. Data was collected regarding: (1) children’s interaction with the haptic device, (2) children’s interaction with the game, (3) children’s discourse, including discussion with the children before and after the games. Data analysis looked at critical episodes aiming to capture student experience with haptic feedback and repertoires of haptic device manipulation. The two haptic games studied were “Space Shapes” followed by “Balloons”. The haptic game “Space Shapes” aims to support 3D shape exploration through a narrative in which the user helps a fictional character named “Mia” build her rocket. In this game, the user explores physical properties of a cube (i.e., number of faces, vertices, internal corners, 2D met representation). The second haptic game that was studied was called the “Balloon” game. The Balloon game was played before the Space Shapes game in the study, and was designed to familiarize the user with the perception of depth allowed by the haptic device. In the game, lines of balloons situated in a 3D space and children pop the balloons using the haptic device. It was found that children adopt different grasp styles of the haptic device than intended, which influences the control, angle, and rotation of the pen. It was also found that for typically developing children, haptic feedback presented at the same time as visual feedback tends to be overlooked [25]. These findings also reinforce the importance of determining the most intuitive design. The significant amount of variability in grasp-styles between the subjects alludes to potential usability barriers in the design of the haptic hardware system. The phenomenon of visual dominance (Colavita Effect) must also be addressed as to improve haptic design, not hinder working memory, and understand when haptic feedback is most effectively implemented.

3.2. Medicine

“Medicine, from its inception, has been about connecting clinicians with patients through the sense of touch” [1]. Haptic technology has broad applicability within the medical field, and is continuing to expand. It can offer a cost effective and reusable alternative to current methods of pre-clinical preparation, such as cadavers, live human subjects, animals, or other physical methods of simulation [26]. It can provide a valuable tool for training, evaluating clinical skills, and performing interventional procedures [1,27,28]. The embodied experience haptics offers provides a realistic environment for trainees to develop the motor skills involved with medical procedures and also provides an embodied route for the user to off-loading cognition during training [27]. When learning motor-skills, vision can only partially provide the learner with information. Learning motor tasks, such as medical procedures, proceeds at least in part through haptic exploration and trial and error [29], thus making haptic technology a viable educational solution.

3.2.1. Palpatory Diagnosis and Treatment

The use of haptic or tactile learning plays an important role in medical professions. There are concepts and procedures that require tactile feedback to determine success, such as: palpatory diagnosis, injections, or surgical procedures. The implementation of haptic interfacing for these professions have shown great promise, with positive subjective reports [30]. Palpation is the process where physicians analyze tissues or organs via their fingers to detect anomalies [31]. This assessment technique employed by medical professionals is important for determining patient characteristics and plays a large role in human and veterinary medicine [30]. Palpation can be utilized to effectively locate medically relevant anomalies (e.g., tumors) and targets (e.g., veins) underneath the skin [31,32]. While the procedure of palpation is economical and quick, it relies on the presence of real-life patients with specific medical problems to ensure adequate training. The limited supply of real-life examples can significantly hamper teaching palpation [30]. Haptic technology can enable the learner to experience a variety of different patient problems with lower restrictions on time, allowing the user to spend more time becoming familiar with a wide variety of different diagnoses, thus likely improving the user’s palpation skills. A study conducted by Karadogan and Williams [30] examined the implementation of a palpatory diagnostic haptic module on a group of medical students. Palpatory procedures included used touch to determine common afflictions, including: bump height, bump location, fascial drag, pitting edema, ropey, and stiffness. The participants in the study were twenty-two osteopathic medical students from the Ohio University College of Medicine. After completing the haptic modules, the subjects completed a subjective evaluation questionnaire about their experience. Results showed that the majority of the students believed that the practice with the haptic modules helped to improve their palpatory skills, suggesting that the haptic module would make a great addition to their pre-clinical training.
Li et al. [33] explored a pseudo-haptic device and its ability to teach palpation for tumor identification under different conditions. Pseudo-haptics is a feedback method which visually mimics force feedback through appropriately adapted visual cues. Pseudo-haptic feedback (PHF) generates virtual “forces” through visual feedback only (i.e., cursor displacements), so no force feedback devices are necessary. However, PHF can be easily combined with haptic feedback. In Li et al.’s study, four different palpation scenarios were deployed to investigate the effects of different types of feedback on the users’ nodule detection time during palpation and determine which combination may best strengthen haptic perception. Four different tests were conducted by the twenty participants: (1) Manual palpation, (2) PHF, (3) FF, and (4) PHF and FF. The manual palpation was conducted on silicone blocks with hard inclusions inside which represented tumors. Participants reported the locations where they believed to have sensed the hard nodules and were timed until they thought they had identified all of the nodules. For the PHF group, participants were asked to palpate the virtual soft tissue block with the embedded hard nodules inside using only PHF and to indicate the positions of the hard nodules they believed to have found, while being timed until they believed they had found all the nodules. The force feedback group completed the same procedure as mentioned above relying on force feedback only. The final test was completed with the combination of pseudo-haptic feedback and force feedback. Participants were asked to palpate the virtual soft tissue block with hidden nodules by using the combined feedback method and then asked to indicate the different positions where they believed to have found the hard nodules and timed until completion. Compared to FF alone, the PHF device increased sensitivity by 5%, the positive predicative value by 4%, and decreased detection time by 48.7%. The combination of PHF and FF enabled subjects to find hard nodules via palpation faster, compared to both PHF and FF individually [33]; this effective combination of visual and physical feedback is a unique property haptic technology can afford its users.

3.2.2. Surgical Training

Surgical procedures have been canonically practiced on cadavers or animals. While these modalities of training allow for an immersive and realistic experience, they are expensive and non-reusable. It has been stated that on average 750 operations are needed to improve surgical procedure ability and it has been found that patients who have been treated by doctors with 750–10,250 procedures worth of surgical experience tend to have fewer procedure-related complications than patients who were treated by less experienced doctors [31]. Haptic technology provides an immersive, realistic, cost-effective and reusable route to prepare medical students for surgical procedures. Haptics have shown promise for reducing cognitive load and improving accuracy during minimally invasive surgery training, and have been determined to improve training results [34]. Laparoscopic procedures are minimally invasive procedures of the abdomen that require small incisions for surgical instruments to enter through. During a laparoscopic surgery, surgeons cut small incisions in the abdominal wall through which surgical tools and the laparoscope (which is a small fiber-optic camera used to view internal organs and the procedure) can be inserted. Since laparoscopic surgical tools are long-stemmed, surgeons often receive distorted force feedback while performing the procedure [27]. A study conducted by Zhou et al. [27] looked at the role haptic augmentation played in a virtual reality laparoscopic surgery simulator. Zhou et al. investigated the efficacy of surgeons while inducing cognitive load via mental math questions in three different virtual reality surgery simulations: one without haptic feedback (the MIST-VR), one with haptic feedback (the ProMIS-AR), and one with exaggerated haptic feedback (the ProMIS-AR). The MIST-VR, developed by Wolfson Centre for Minimally Invasive Therapy (Manchester, UK) is a minimally invasive surgery trainer exclusively for virtual reality and is composed of two position-sensing gimbals that detect the hand movement of the user in order to control the position of two laparoscopic instruments on a computer screen. The ProMIS-AR (Haptica, Dublin, Ireland), a commercial laparoscopic simulation platform, is an augmented reality simulator and includes a life-size torso with a light source and laparoscopic tools. The ProMIS-AR, equipped with three cameras, can track and display on a computer screen the three-dimensional position of the laparoscopic tools inside the torso. Three performance measures were collected from each simulator: time to task completion, errors, and number of math problemsattempted. The subjects preformed the virtual procedure 21% faster when they were not cognitively loaded, which indicated that the mental math problems were competing with the laparoscopic task for cognitive resources. In general, even while cognitively loaded, the subjects preformed faster (36% with exaggerated haptics and 33% with haptics) and more accurately (97% with exaggerated haptics and 91% with haptics) with haptics than without. This data suggests that haptics has the potential to not only enhance surgical performance, but possibly mitigate the amount of cognitive load experienced [27]. These findings held true in the study regardless of the magnitude of haptic feedback, and it was determined that haptic feedback enhances performance and negates cognitive load during virtual surgical simulations compared to non-haptic simulations. Haptic augmentation has also proven beneficial during real laparoscopic surgery as well. The tools utilized during laparoscopic surgery can significantly reduce sensitivity and consequently may impede the transmission of important tactile feedback [1]. Sensory loss from long-stemmed endoscopic instruments has been found, with the main cause of this disturbance being frictional forces. It was found that when haptic feedback was absent, there was increased tissue slippage and damage that occurred during grasping actions [35]. The reduced level of force feedback can be combated by incorporating haptic feedback into laparoscopic surgical tools, thus providing the surgeon with increased sensitivity and ability to more accurately perform the surgical procedure, while potentially freeing up cognitive load [27].
A study completed by Vapenstad et al. [36] investigated the skill transfer from a VR haptic simulator to a porcine organ model in a box trainer, which is a physical surgical training environment. Students in the haptic group first underwent training on a haptic training simulation where they were trained on five common surgical tasks: coordination, clip applying, lifting and grasping, fine dissection, and pattern cutting. The control group had no training experience prior to performing surgical tasks in the porcine organ model box trainer. Both groups of participants were then evaluated on their ability to perform a cholecystectomy in a porcine organ model box trainer. Video evaluations were rated by two surgeons unaware of the subject’s training condition (haptic training or no prior training). It was found that the haptic simulation was not directly transferrable to performance in the porcine organ model box trainer; this negative result was speculated to be a result of poor mechanical performance of the simulated haptic feed. Further studies must be conducted between groups of haptic devices to determine the educational results various haptic devices may provide.
A study conducted by Strom et al. [37] investigated whether incorporating haptic training early-on (prior to other training) during surgical simulation would improve resident performance. 38 surgical residents were randomly assigned to two groups: early-haptic and late-haptic. The former group was designated the “early-haptic” group because in the first hour of training they received haptic feedback. The latter started the first hour of their training with the Procedius abdomen simulator and were termed the “late-haptic” group. After the completion first session, the groups switched training methods (haptic or non-haptic) types for the second session of training. The simulation tasks being completed were manipulation and diathermy for 30 min and point diathermy for 30 min, with the surgical context being a virtual upper abdomen. They found that the experimental group receiving the “early” haptic intervention preformed significantly better on the two diathermy tasks and was the only group to improve their skills on the second session. From these results, Strom et al. concluded “haptic feedback could be important in the early training phase of skill acquisition during a surgical simulator” [37].

3.2.3. Veterinary Medicine

To keep up with modern concerns and requirements, canonical methods of teaching anatomy to veterinary students must adapt [38]. Ethical, resource, and time constraints are a few problems with traditional dissection and anatomy classes, which typically use live animals or cadavers [38]. Haptic technology allows for realistic training with a virtual model, thus providing a reusable method of instruction while reducing ethical and welfare concerns [39]. A study conducted by Kinnison et al. [38]., investigated the practicality and usefulness of a haptic simulator for teaching basic bovine abdominal anatomy. A total of 184 students provided feedback about their experience by answering a questionnaire. The questionnaire results were markedly positive; the students felt that the simulator was “useful”, “helped with their understanding of bovine abdominal anatomy”, and was “enjoyable” [38]. The hardware used in the study was the PHANToM Premium (3D Systems, Inc., Rock Hill, SC, USA), which had a thimble attachment at the end of the mechanical arm for the student to place their middle finger in to experience the force feedback with. While the majority of the subjective feedback received from the students was positive, some students expressed the desire for the ability to use the whole hand to experience the haptic simulation as to experience a more immersive haptic simulation. Palpation is a widely used technique in veterinary medicine, especially in relatively smaller animals such as cats or dogs; in small animals, abdominal contents can be assessed directly by palpation [39]. Plenty of hands-on experience is necessary for developing palpation skills, however certain small animals, such as cats, exhibit a low tolerance for such examinations. This can make it difficult for novices to gain necessary skill [39]. To address this problem, seven veterinarians consulted on the design of this mixed-reality simulator which was comprised of a stuffed cat modified with PHANToM premium haptic devices. The consulting veterinarians were confident in the ability of the mixed-reality simulator to teach feline abdominal palpation and specified it could have an effective role in instruction. The feedback was majorly positive and hopeful that the simulator could address training issues, such as insufficient opportunities for hands-on training [39].

3.2.4. Venipuncture

Intravenous (IV) catheter insertion is one of the most commonly performed medical procedures [26]. While it is one of the most common, it is also deemed one of the most complex procedures learned by nurses [26]. Serious complications in patients such as infiltration, phlebitis, or pain can be avoided when the IV is inserted with proficiency [26]. Traditional methods of teaching IV insertion include didactic instruction followed by hands-on practice on either a real or simulated arm. These educational methods can be time consuming, cost ineffective, limited in variability, faculty intensive, and may be painful for the training procedure recipient. A literature review conducted by McWilliams and Malecha [26] compared traditional IV insertion instructional methods with haptic IV simulators in terms of their outcomes. he authors found that haptic IV simulators were at least equally as effective as traditional methods of teaching IV insertion and provide a cost-effective alternative to previous instructional methods [26].

3.3. Visual Disability Instruction

It is estimated that there are approximately 43.3 million people suffering from blindness globally [40]. Students with visual disabilities are at a disadvantage when it comes to learning traditionally in a classroom [40,41]. Inadequate math literacy has been found to be a prevalent problem among the visually impaired and is exacerbated by a limited availability of assistive technology [42]. Mathematics and science often rely on visual presentation, such as graphs; conveying this information in a non-visual way can be an especially problematic tasks for instructors who may have based their own understanding of these topics on visual representations [40,41,42,43,44]. Current computer education programs heavily rely on graphics to convey subject material, excluding students who suffer from visual impairments. Haptic incorporation in educational settings may provide students with visual disabilities with another means of access to the educational material via touch. A preliminary study conducted by Darrah [41] sought to determine if haptic-based apps could promote positive learning outcomes when teaching science and mathematical content to visually impaired learners. Four computer applications were developed to be tested: Three Dimensional Shapes, Exploring the Atom, Gravity on the Planets, and Surface Area of a Cube. The four haptic applications were first tested by a group of experts, which included blind adults, science and math teachers, teachers of the visually impaired, and computer developers. After being tested by the experts, the applications were then tested by four visually impaired learners. During their engagement with the computer applications, the students were observed by the developers. Afterward, the students were interviewed about their experience. Their feedback was accounted for and the haptic design was adjusted accordingly prior to entering educational settings to determine classroom feasibility. Once deemed ready for testing in the classroom setting, the haptic apps were tested at the West Virginia School for the Blind and at Space Camp for Interested Visually Impaired Students (SCI-VIS) In both settings, student understanding of content was measured by pre- and post-test analysis. Students self-reported the haptic apps being fun, easy to use, and helpful for understanding the content [41]. Pre- and post-test score differences indicated that the haptic-based apps show promise in teaching classroom materials to visually impaired students. The ‘Exploring an Atom’ haptic-based application was tested at the West Virginia School for the Blind and was found to increase average test scores from 4.75 points to 9 points out of a total of 14 points. At SCI-V IS, ‘Exploring an Atom’, ‘Gravity on Planets’, and ‘Surface Area of a Cube’ haptic-based apps were tested and found to increase student scores significantly (2.21 to 5.36 out of 14; 5.67 to 8.22 out of 16; and 1.36 to 3.43 out of 11, respectively). These preliminary results indicate that learning gains can be achieved via haptic-based applications for students who exhibit blindness or visual impairments.
A subsequent study conducted investigated the efficacy of haptics-based computer applications in teaching a broad range of science concepts, such as gravity, atoms, surface area, cells, and anatomy, to students afflicted with a range of visual impairments [44]. A pre- and post-test were administered to determine the students’ learning gains acquired from using the haptic-based computer applications. Murphy and Darrah found significant increase in score, on average 41%, between the pre- and post-test after incorporating the haptic-based computer lesson. After the lesson, qualitative reports were also collected that assessed student enjoyment. It was found that students felt exceptionally positive about the haptic incorporation into the classroom, with an average rating of 3.41 (4.00 being the best).
A study conducted by Avila-Soto et al. investigated the initial use of a tangible user interface (TUI) math application, TanMath, to teach arithmetic operations to a visually impaired student. In a TUI, physical objects, instruments, surfaces, and spaces work as physical interfaces for manipulating digital information [43]. Users interact with TUI systems by manipulating physical objects, referred to as physical icons or “phicons”, which are tracked by the computer systems and represented digitally.
It is thought that limited access and ability to do mathematics may be one of the biggest challenges for students with visual impairments [43]. Traditional methods of teaching visually impaired students involve having the students recognize the shape of a number, and teaching them to perform basic addition and subtraction operations with the physical number shapes. Concurrently, the students begin to learn numbers in braille. Once the students have mastered those mathematical techniques and their concept of numbers is concrete, they begin to work with a Japanese abacus. A Japanese abacus is a calculating tool that was used to perform calculations and represent numerical values prior to the written numerical system [45]. The researchers tested their concept program on a blind 5th grader who attended public school. The findings were positive and promising, as the subject completed the mathematical tasks well and navigated the program with ease [43]. The study conducted by Avila-Soto et al. illuminated space for innovation in teaching mathematics to visually impaired students and suggests the need for digital augmentation [43,44]. Haptics and TUIs offer a digital, yet embodied, learning experience that can greatly benefit the visually impaired.
Bateman et al. [40] conducted a user centered design study that analyzed visually impaired participants’ interactions with an electrostatic haptic touchscreen system. Electrostatics is a subfield of surface haptics and produces haptic effects via the application of voltages to a conductive surface, which created friction on one’s fingers. Bateman et al. first interviewed 6 individuals as to identify the technological needs of visually impaired students and their educators: a technology expert from the International Braille and Technology Center for the Blind (IBTC), the president of Maryland chapter of the National Federation of the Blind (NFB), and the principal, vice-principal, and two elementary math teachers from the Maryland School for the Blind (MSB). Questions such as “What categories of technologies are most widely used?” and “Which technologies receive the most criticism from educators and users? Why?” were asked during the interview. Common interviewee responses addressed several problems, including: the size and cost of high-tech devices, cross-compatibility problems between assistive devices and their various operating systems, and a lack of portability. A usability study was then conducted that measured participant accuracy and efficiency as the primary metrics while using Bateman et al.’s haptic device. The participants were recruited from the National Federation of the Blind and the Maryland School for the Blind. Participants were asked to locate a haptic dot on the device. The study included 30 different displays with the dot being located at one of the 30 predetermined locations on the screen. Response accuracy and time to dot location (indicated by a verbal response) were recorded by video. If a participant located five correct dots consecutively, the test was concluded. The videos were analyzed afterward to confirm responses, and to also investigate user strategy for exploring the haptic screen. Of the 12 participants, 11 gained mastery of the simple haptic task of dot location within the 30 trials, with the average participant only needed 8.27 trials to do so [40]. The improvement of the participants’ accuracy in locating haptic dots demonstrates the usability of the device, and the ability of the participants to develop effective screen exploration strategies indicted that the device is intuitive for users [40].
Navigating through unfamiliar spaces can be a difficult and daunting task for those who are visually impaired [46] To achieve orientation and mobility (O&M) visually impaired individuals rely entirely on tactile or auditory feedback from the environment. Sanchez [46] describes the development of a haptic and audio augmented virtual environment videogame designed with the goal of teaching orientation and mobility skills to people who are visually impaired [46]. Sanchez conducts studies investigating usability and impact on orientation and mobility skills in blind students. The video game was referred to as the “Audio Haptic Maze (AHM)” [46] The objective of AHM was to escape from a maze. In order to meet that objective, players of the game were required to locate jewelry boxes, which contained keys and treasures. These jewelry boxes were dispersed throughout the maze. The keys that were located within the jewelry boxes had geometric shapes on them that corresponded to a certain door. A participant’s score was determined by how many treasures they discovered and how fast they finished the maze. AHM used a “spatialized sound to represent the ambience of the corridors” and sound cues to denote certain events, such as finding a key [46]. Haptic feedback was employed to denote change in textures, different shapes, and chosen direction of avatar movement. To test the usability of the device, an initial study was conducted on ten blind learners ages 9–15. The usability study results revealed two key points for design improvement: pure sounds and simple shapes. Designers initially created the AHM with metallic sounding audio feedback, which was found to confuse some users. Also, complex shapes were deemed harder to identify by users via touch. To evaluate the cognitive impact the AHM video game had on the blind users, an O&M skills checklist was employed. 7 blind learners between the ages of 10–15 were selected from the Santa Lucia School for the Blind. The metrics measured by the O&M checklist were: (1) Sensory perception (2) Tempo-spatial development (3) Orientation and mobility skills [46]. The O&M checklist served as a pretest. After the pretest was administered, the AHM videogame was completed by the participants. Upon conclusion of the AHM videogame, participants were asked to form a graphic representation of the virtual environment they had traversed during the game. This representation served as a measurement to determine the efficacy of the game in helping the user build a mental model of the virtual environment. Additionally, the orientation and mobility pre-test were re-administered to the participants as a post-test. Evaluation of the participants demonstrated an increase in pretest/posttest means in all dimensions and that training with the audio-haptic-maze simulator improved the development of orientation and mobility skills in visually impaired users [46]. The usability study found that choosing appropriate audio icons is crucial to user understanding and the transmission information; certain sound effects may be confusing for the participant. This study of the AHM determined that use of audio and haptic icons enable the learners to effectively navigate the virtual environment in a way that is translatable to real-world navigation [46].
Certec, the Division of Rehabilitation Engineering Research, Department of Design Sciences at Lund University, has been working on haptic interfaces since 1995 for visually impaired people. A paper authored by Sjostrom [47] presents a brief overview of tests that have been conducted on these haptic devices and games, as well as the detailing the resulting design implications for blind users. Researchers at Certec found that their haptic programs generally worked better for the visually impaired when they incorporated certain design elements, resulting in the following design guidelines (Table 1):
Preliminary findings by Bussell align with Sjostrom’s design principles. A paper by Bussell offers a report on the progress of a project to create a geometry software program for students with visual impairment. The geometry software was called Touch Tiles, and employed both haptic and auditory feedback. This software enabled visually impaired learners to complete “pattern creation tasks” using geometric shapes [42]. Four users were studied during the preliminary tests. Like Sjostrom’s research, it was found that sharp corners can be disorienting to visually impaired users. Bussell [42] also found some textures to be more confusing than others, for example a grid texture was misinterpreted as a moving square by one user. Bussell found that, “… recessed grid lines were easier to follow than raised ones, however both led to confusion at the intersections” [42]. Another key finding determined that circles inscribed inside of squares also led to user confusion at the corresponding convergences. The strength of the force feedback experienced by the user was also found to be influential on shape identification. One boy during the preliminary study stated that, “The strongest effect was the best”. When the force feedback was set to maximum strength, it was observed that all users appeared to have an easier time following the shapes [42]. The highly visual nature of science simulations can present accessibility barriers to students who are blind or visually impaired [48]. The accessibility of a haptic simulation program was investigated in a study conducted by Hansen et al. The study explored four different modalities of teaching a science concept (water evaporation) to students grades 7–9. The first modality was the “base condition” which involved only the use of a screen reader as it was meant to simulate a “less-than-adequate” method of presenting information to visually impaired students. The second condition used the Novint Falcon haptic game controller. The third condition utilized vibrotactile haptics on an Android tablet. The final condition was the tactile graphic condition (embossed on thick paper). Hansen et al. utilized mainly qualitative methods of data analysis. Each of the 3 students in the study experienced the conditions in the same order—base, Falcon, Android, and tactile graphics. It was found that students had difficulties determining the spatial arrangement of the water particles via the conditions, so researchers switched the task to something they deemed simpler for the students, which was counting particles. For counting the particles, it was found that all students had some difficulty counting particles with the Novint Falcon. It was believed that the 3-D nature of the Falcon may have presented more opportunities to miss the particles versus the flat nature of the Android or tactile graphics. Despite of having the lowest accuracy rate in counting water particles, users of the Falcon had the highest level of student agreement with the statement “I would recommend the access method for science simulations” [48]. Students also experienced some difficulty finding particles via the vibrotactile Android simulation as well. It was hypothesized by Hansen et al. that the small size of the particles on the Android (0.9 cm in height, in contrast to the 1.4 cm in height of the tactile graphics display). A background vibration that was employed to alert students of when they were touching the graph area on the display was also thought to have caused some confusion among the blind and visually impaired users. The tactile graphics method was found to produce the best signs of usability, with the highest counting accuracy among the students and positive subjective reports [48].

3.4. Language and Handwriting Acquisition

At first glance, language learning may seem like an unlikely candidate to benefit from the incorporation of haptic technology. However, there is increasing evidence that linguistic cognition is rooted in sensorimotor engagement [16,49,50] and that linguistic expression is directed toward action [16]. It is even possible to communicate language though touch [51].
Achievement data in early literacy learning indicate two concerning trends: overall, general slippage in early literacy development outcomes and, secondly, a persistent and growing achievement gap over time in the shift to academic literacy development among linguistically vulnerable populations of children including those raised in poverty or of language minority background. Emergent research findings suggest the etiological underpinnings to these trends may be found in children’s control over fine motor skills and their influence on language learning as these are the major drivers of literacy development over time [49]. Hands-on exploration is an innate method of learning for children [49]. Bara et al., sought to understand the impact of “multi-sensory training” on children’s understanding of alphabetic principles. To explore this effect, kindergarten children from low socio-economic status families were studied in two different conditions: a visual-haptic exploration of letters (referred to as HVAM or Haptic-Visual-Auditory-Metaphonological) and a solely visual exploration of letters (referred to as VAM or Visual-Auditory-Metaphonological). When the participants received HVAM training, higher scores of performances in a phenome identification task and a letter recognition task were reported. This finding suggests the embodied interaction with letters led to better learning outcomes, possibly by creating a link between the biologically primary interactions (haptic exploration of letters) and the biologically secondary knowledge (letter recognition). In addition to assisting with child literacy learning, it has been found that haptic systems provide a viable route for language interpretation.
In addition to assisting in the collection of knowledge, haptics has been found to effectively convey language. Jiao et al. [51] conducted a study investigating two different haptic learning approaches to teaching a language via tactile stimulation. This haptic translator turns English phonemes into haptic patterns termed “haptic symbols” which can be detected by skin on the forearm. The haptic device is strapped onto the non-dominant arm and produces tactile stimulation via haptic vibrotactile patterns to represent phenomes or words. The study compares word-based and phenome-based approaches to teaching a haptic language. It was determined that both language learning methods were effective, however the phenome-based approach was determined to be the most reliable choice [51]. In a similar vein, Dunkelberger et al. [52] proposed a multi-sensory haptic device called MISSIVE that communicates the full English phenome set via haptics. This novel approach to conveying language may be beneficial for those with impairments, or those with auditory and visual channels already in use. Dukelberger et al. [52] evaluated their approach by teaching subjects a set of 23 phenomes and participants demonstrated an 86% accuracy in a 50-word identification task after 100 min of training [52]. These study results show promise for learning, conveying, and interpreting language via haptics, and corroborate the ability of haptic technology to conceivably provide additional routes of communication and access to information for hearing impaired individuals.
Handwriting of individual letters has been associated with proficiency in both reading and spelling [53]; according to Germain et al. [53], “Handwriting is a fairly complex sensorimotor task that requires perceptual, motor, cognitive and linguistic skills”. The successful acquisition of handwriting skill ensures that the cognitive processes of producing text do not interfere with the cognitive mechanisms of composing and spelling. When handwriting skill is adequately learned the motor skills of producing writing become more or less automatic; subsequently, working memory resources are liberated for devotion to “higher-level aspects of composing” [53]. Handwriting is best learned through practice and refinement [54]. A haptic-visual interface was created by researchers Xiong et al. [55] to investigate the optimal method of teaching Chinese handwriting to students. Participants in the study were divvied up into three training conditions: haptic, visual, and a combined haptic-visual. The subjects were asked to write a given character followed by two characters with common morphology before and after the administered training. The haptic-visual interface recorded teacher’s writing skill and applied the forces to the students during the handwriting training. Their performance was measured in terms of mean speed, in-air time (pausing time in air during writing), size, order, and shape. It was found that the group trained via the haptic-visual instruction performed the best and had the least amount of air time [55]. It was found that the different sensory modalities (visual and haptic) were more effective for the learning different writing properties; for example: the visual modality was most important for learning writing shape, while the haptic modality was more important for the skill transfer of shape learning. Xiong et al.’s findings suggest that vision is most useful for extracting the shape properties of an object as a whole, whereas it was speculated that haptic information was more effective in creating an internal mental model of the shape and each stroke separately, resulting in better transfer of skills [55]. Another study regarding haptic guidance for hand writing skills development was conducted by Teranishi et al. [54]. In this paper, full and partial haptic guidance methods were compared in their ability to teach Arabic writing skills. Haptic guidance methods are developed to support the learning of handwriting skills by physically moving the learner’s hand along the desired trajectory [54]. In full haptic guidance, the haptic interface leads the handwriting task while the learner passively follows the movements along the trajectory. In the case of partial haptic guidance, the learner actively moves the interface, but is redirected by corrective force feedback when the user strays too far from the reference trajectory. Upon analysis of the handwriting quizzes and questionnaires, it was found that the most effective method of teaching handwriting through haptic simulation occurs when partial haptic mediated guidance is followed by full haptic mediated guidance. In children, acquiring handwriting is typically a slow and arduous process. To attain mastery of handwriting, several years of formal education are often required [53]. A visuo-haptic device, the Telemaque, was investigated by Germain et al. [53] on its ability to help develop handwriting skill learning among pre-handwriting Kindergarten students. Forty-two Kindergarten students (ages 4–5) without disabilities or special educational needs participated in the study. To measure their handwriting performances, the children were asked to copy six cursive letters. This performance measurement served as both the pre- and post-test analysis. The outputs being measured were movement velocity, number of velocity peaks, and the number of breaks during execution. In the visuo-haptic intervention, two exercises were presented by the Telemaque: The circuit game and the dynamic tracing of letters. In the circuit game, the pen used for writing was attached to a force feedback arm which attracted the pen to the nearest point on the correct trajectory if the child deviated from the intended course. During the exercise, force was progressively decreased over the course of 10 trials per each letter, with the last two trials displaying no force feedback. During the dynamic tracing of letters, children were instructed to hold the pen as an individual letter was presented on the screen and the pen moved “alone” via haptic force feedback. During dynamic tracing, ten trials were performed for each letter. The control group was first asked to copy a target letter ten times on a sheet of paper. The group was then asked to type a sentence in which the target letter appeared frequently. It was found that use of the visuo-haptic system improved the fluency of handwriting production for each letter that was learned when compared to the control group. The visuo-haptic group also exhibited a faster than average writing velocity (a roughly 50% increase in average speed), fewer velocity peaks, and less pausing during the post-test task.
Built to simulate the role of occupational therapists, the Haptics Assisted Training (HAT) System is a force feedback workstation that aims to improve handwriting skill by guiding the user’s hand along the sequence of strokes of the reference handwriting recorded by a teacher [56]. HAT training tasks are presented as a 3D game and it is designed for practical use in actual classrooms. Kim et al. [56] conducted a comparative study on handwriting performance of twenty participants grouped into four categories by education (special vs. typical) and haptic devices (Phantom Omni, a high-end device that now called as Geomagic TouchTM from 3D Systems, Inc., Rock Hill, SC, USA vs. Novint Falcon, an affordable device from Novint Technologies, Albuquerque, NM, USA). Twenty children were recruited, ages 6–11. Each participant carried out the following procedure: (i) an introductory session with explanation on the game (ii) pre-test with no haptics in the system (iii) training session with haptics (iv) post-test with no haptics, and (v) interview session with questionnaires [56]. Each participant underwent four sessions within a two-week period. The main quantitative measurement the researchers recorded was ‘tracing precision’ in regard to the expert reference handwriting (error measured in millimeters). Participant behavior was also monitored during the game and while completing the tracing tasks. Kim et al. [56] found that children’s tracing performance on the HAT system improved for all four groups in a relatively short time of the study sessions [56], however the children in the typical class using the Omni, a high-end haptic device, outperformed for all measures. However, children using the Falcon, a more affordable haptic device, were able to adapt and may have been at a slight disadvantage of mapping a smaller haptic workspace to the visual display. The authors suggest that the less-expensive Falcon can be used in introductory training that requires robustness for playful children, and that the Phantom Omni can be used for advanced training lessons using the more delicate haptic feedback [56]. A pilot study conducted by Ogawa et al. [57] explores the effect of haptic gloves on user vocabulary learning on a computer keyboard. The haptic elicited a vibrotactile signal to the fingers associated with a specific letter on the keyboard (according to QWERTY mapping) just before letter appeared on the screen. Each subject in the study learned 20 words using the haptic gloves and 20 words without haptics in an only visual condition. It was found that the words learned with haptic cues lead to more effective knowledge retention measured one week after training.

3.5. Additional Applications in Training

Outside of the traditional learning environment, haptic augmentation can provide substantial benefits in the form of skill acquisition and physical training. A recent study conducted by Kasahara et al. [58] explored the ability for an electrical muscle stimulation (EMS) haptic device to speed up a user’s response. Kasahara et al. found that the users’ reaction time significantly decreased not only while wearing the haptic device which elicited a response, but the faster response time persisted even when the haptic device was removed. This has substantial implications for many fields, such as sports or professional gaming, that require highly precise timing and quick reactions. Haptic intervention has also been found to lead to robust training benefits for sports training via real-time methods of physical feedback, such as posture correction and movement coordination. Erp et al. [59] investigated a haptic system to test a rowing machine’s ability to instruct the learning of coordinated movement (e.g., whether during a stroke their back was too early or late in the “pull phase”). Three experimental groups were created. Group I received direct real-time positional feedback via vibrators located on the knees and back. These simultaneously vibrated when a certain correct angle was met between the back and knees. Group II received visual positional feedback where circles on a screen corresponded to their posture. Group III received feedback similar to traditional coaching, which was provided once every 25 strokes. Each subject had eight sessions in one day (pre-test, six trainings, and a post-test). While there were not significant differences in the resulting speed or other measures of improvement, the group that had received haptic feedback had an overall lower heart rate during the post-test, which the authors contribute to decreased mental effort. This may indicate that the users who received haptic feedback had a reduced mental workload in line with what we would expect from the off-loading concepts of Embodied Cognition and Cognitive Load Theory.

4. Discussion

With haptic technology becoming increasingly more available for commercial and personal use, there is an increased ability to address pressing research questions related to educational design. Haptic technology offers the versatility and flexibility of a virtual system, while retaining the embodied quality of tangible equipment. This combination allows the user to reap the benefits and affordances of both modalities, thus providing the learner with an effective learning environment. The effectiveness of the learning environment, however, seem to show differences based on the applied domain. Our observations and findings of reviewed papers based on the learning domain and discipline are shown in Table 2. As the epitome of our review, it is seen that more sensitive and immersed haptic experiences lead to better learning outcomes especially in the STEM field and actions lead to better learning outcomes when they are congruent with conceptualization of the learning material. We see that the realism of the haptics simulation, however, is not as imperative in the STEM field as it is for medical training simulations using haptics. As required by most medical diagnosis and treatments that utilize palpation, force feedback with relatively higher forces becomes an important aspect of a haptic simulation for medical training as far as user engagement and simulation of real-life examinations are considered. Therefore, in order to provide the required high fidelity for medical training applications, high-resolution haptic devices with a large workspace are usually preferred, which in return increases the cost of these simulations significantly. The timing of incorporating haptic simulations into the training regimen of professionals affects the learning outcomes; implementation of haptic simulations early in the curriculum proves to be more effective especially in medical training and learning languages. For training of visually impaired persons, incorporation of auditory guidance becomes as important as haptic feedback and the forces reflected to these users tend to remain on the higher range for a more effective learning experience. Our review also revealed that haptic augmentation offers distinct benefits depending on the context it is used in and the educational goal the learner is trying to meet. Due to the abstract nature of STEM subjects, haptic technology may help the learner connect with the learning material in a way that feels concrete and understandable. Haptic technology may also facilitate the acquisition of biologically secondary knowledge through biologically primary methods of interaction; this embodied way of understanding subject material through haptics has been referred to as a “Haptic Bridge” by Davis et al. [24].
We know that the success of a haptic simulation for learning is dependent on several factors including but not limited to its context, user population, quality of the software design (user-friendliness, clarity of instructions, graphics etc.), hardware (haptic interface), engaging features, and effective implementation of pedagogical principles. Therefore, including haptics in any simulation does not guarantee that it will be a successful learning tool. In that direction, we collected the best practices to aid the developers of haptic simulations. The detailed results are presented in Table 2 for the interested reader, but the best design practices and features can be summarized as:
  • Early implementation of haptic feedback during the learning process creates positive learning outcomes.
  • The more sensitive the haptic device, the better the overall learning outcome and skill gained from the simulation.
  • The designed actions in a simulation must be congruent with desired learning outcome, and realism secondary to a well-interpreted action/simulation (higher sensitivity and immersion for a more embodied experience).
  • Incorporating simple sounds, emphasizing haptic feedback, and incorporating audio guidance help improve the effectiveness of the simulation.
  • Medical simulations benefit from tactile fidelity, higher degrees of realism, and force feedback with relatively larger forces. Incorporation of “Haptic Mirrors” [24] in which the goal of the simulation is to accurately mimic the real situation as closely as possible is important.
  • Visually disabled users benefit from having another avenue of access to subject material through the phenomenon of embodied cognition.
  • Language and handwriting students benefit from the embodied nature of design and motor skills training. Partial haptic guidance followed by full haptic guidance is most effective in teaching handwriting skills; haptic guidance was more effective in teaching the overall shape of the handwriting skills, whereas the full haptic guidance was more effective in teaching the fine details [54].
Substantial research has been conducted on visual and auditory working memory, in the forms of the visuo-spatial sketchpad and the phonological loop. However, literature on tactile working memory is scarce. Modular (or sensory-specific) working memory should be further investigated from a neurobiological and/or neurocomputational perspective to further understand the distinct modalities human attentional processes may adopt. It has been proven that touch can off-load cognition from working memory during task performance [8,12,13,27,60], but the neurological basis behind that off-loading remains to be unclear. Previous related hypotheses have been explored under the ideologies of “Cognitive Theory of Multimedia Learning” and “Dual Coding Theory”, however there is no current neuropsychological model that altogether explains the phenomena of off-loading cognition via touch. Somatosensory or tactile, working memory has been linked to activity in an area of the brain called the inferior frontal gyrus, or IFG [61]. Disruption of this area with repeated transcranial magnetic stimulation (rTMS) has been found to disrupt human participants’ working memory capacity during tactile discrimination tasks without interfering with their tactile perception. This study provides evidence for a causal involvement of the IFG in somatosensory working memory function. Determining the basis and limits of a somatosensory processing system, as well as how it may interact with other attentional processes to create cognitive models will have significant implications for educational design research, because pedagogical material will be able to follow distinct guidelines on how humans process tactile information going forward, as well as decrease cognitive load. Haptic technology design will especially benefit from further understanding of somatosensory working memory because it a primarily tactile technology that has expansive affordances in the educational domain.
Moving forward, the actions that best support a desired cognitive outcome must be clarified to inform future haptic design. Ayelet [62] determined that actions positively affect cognition in a mathematical context when they are congruent with thinking (i.e., tapping for discrete counting, sliding finger for continuous number line estimation). Moyer-Packham identified “Linked Physical Actions” which are actions that were found to best support the student learning outcome. During their preliminary study, Davis et al. [24] emphasized that the symbolic haptic feedback was only as effective as how it was interpreted and comprehended by the user. Determining which actions are most easily or innately interpreted into a cognitive model by the user is crucial for haptic design moving forward as to promote the best learning outcomes.
The innate processing mechanisms that process biologically primary information may be employed while the user is engaged in haptic simulations due to the embodied nature of haptics. This can lead to lower cognitive load constraints, thus facilitating better learning outcomes through the implementation of haptic technology. This cognitive offloading through haptic interaction has been documented in a multitude of studies, but the mechanism by which cognition is “off-loaded” via the sensorimotor route remains to be elucidated. As it stands, there are predominately two main schools of thought. Each sensory system has its working memory and therefore some working memory can be off-loaded from the visual or auditory channel and redirected to the touch channel. Due to evolutionary pressures, humans have evolved to process certain information more efficiently and innately. This type of information is referred to as biologically primary knowledge and is speculated to have lower working memory restrictions than cultural, or biologically secondary, knowledge. While these theories are not mutually exclusive, further research must be done to precisely address the neuropsychological mechanism by which the off-loading of cognition occurs. In order to achieve this, however, more effective and empirical methods of measuring cognitive load must be standardized. As Skulmowski and Rey [13] accurately put it, “the measurement of cognitive load is generally regarded as a difficult task”. Currently, there are three ways of measuring cognitive load: subjective, behavioral, and physiological. Collecting these metrics of data pose some issues. For example, when collecting subjective measurements of cognitive load, they must be appropriately phrased to be useful.
Furthermore, it is important to understand subjects’ understanding of cognitive load question items in different contexts [13]. Subjective measurements are also usually attained after the experimental task has been completed, so real time data may not be collected. Behavioral measures of cognitive load provide a real time alternative to subjective measurements. A variety of metrics are used to determine cognitive load via participant behavior, such as response latency, eye movements, and dual-task performance. The last method of cognitive load measurement is physiological measures. These include electroencephalography and pupil dilation. A potential problem with both behavioral and physiological measurements is that they may be affected by variables unrelated to the experimental task, such as daydreaming, shifts in attention, etc. However, both measurements of cognitive load offer objective alternatives to subjective questionnaires and as technology improves in mobility, so will the potential for measuring cognitive load in a variety of settings. Additional research with aims of delineating clear guidelines for methods of assessing cognitive load must be conducted [13].

5. Conclusions

Haptic technology provides a virtual, yet embodied, avenue of interaction with subject material to students. It has wide-reaching applicability in fields including: STEM, Medicine, Disability Education, and Language/Handwriting acquisition. In the STEM field, haptic technology has been found to provide an embodied and concrete way of interaction with the abstract concepts found in the STEM field. Haptic technology can provide this outcome by facilitating the acquisition of biologically secondary knowledge through biologically primary methods of embodied interaction, according to Pass and Sweller’s account of Cognitive Load Theory. This embodied way of understanding subject material through haptics has been referred to as a “Haptic Bridge” by Davis et al. [24] and is thought to create a bridge between an action completed and conceptual understanding of the user, by-passing normal cognitive working memory processing constraints via capitalizing on innate processing mechanisms refined through evolution. Haptic technology has also been found to free up cognitive load during medical simulation [27], thus leading to faster and more accurate completion of the surgical simulation. In the realm of medical and veterinary education, haptic technology provides a realistic and reusable avenue of training. It can reduce the use of cadavers or live subjects, thus reducing concerns for subject welfare and also increasing the number of resources available to the student. Haptic technology has also proven to be a useful tool for those who are visually disabled, as it provides an additional avenue of interaction with subject material and can implement lasting, transferrable results [46]. Language learning and handwriting acquisition has also been found to benefit from haptics due to the embodied nature of design and motor skills training. Studies suggest an embodied role in language and literacy learning [49].
It was found that across disciplines, haptic technology provided the most learning benefit when it was implemented early-on in the learner’s training progression. This suggests haptic technology may serve a role in creating an embodied foundation. It was also found that exaggerated and more sensitive haptic devices delivered better learning outcomes to the users. For abstract subjects, such as those found in the STEM field, more immersion and the higher degree of haptic feedback experienced led to better learning outcomes. Increased realism of the haptic simulation was indicated to be beneficial to the user, however it wasn’t found to be necessary in certain contexts. For example, surgical simulations were found to benefit from increased realism, but haptic-augmented notebooks were effective in creating positive learning outcomes even without realistic feedback [6]. Haptic augmentation is in part interpreted by the user, emphasizing the need to determine which actions best benefit learning outcomes.

Author Contributions

Conceptualization, E.K.; methodology, R.C. and E.K.; resources, E.K.; data curation, R.C.; writing—original draft preparation, R.C.; writing—review and editing, E.K. and R.C.; supervision, E.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Acknowledgments

The authors would like to thank K. Nusaiba Hafiz, a graduate student at the Robotics and Haptics Lab at the School of Engineering & Technology at Central Michigan University for her help in reviewing and proofreading the final draft of the manuscript.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Kapoor, S.; Arora, P.; Kapoor, V.; Jayachandran, M.; Tiwaria, M. Haptics—Touch feedback Technology Widening the Horizon of Medicine. J. Clin. Diagn. Res. JCDR 2014, 8, 294. [Google Scholar] [PubMed]
  2. Minogue, J.; Jones, M.G. Haptics in Education: Exploring an Untapped Sensory Modality. Rev. Educ. Res. 2006, 76, 317–348. [Google Scholar] [CrossRef] [Green Version]
  3. Zara Stone. Controllers Bring Real Pain to VR Games. Available online: https://www.wired.com/story/haptic-controllers-for-vr-bring-real-pain-to-games/ (accessed on 9 September 2019).
  4. Han, I.; Black, J.B. Incorporating haptic feedback in simulation for learning physics. ScienceDirect 2011, 57, 2281–2290. [Google Scholar] [CrossRef]
  5. Skulmowski, A.; Pradel, S.; Kuhnert, T.; Brunnett, G.; Rey, G.D. Embodied Learning using a tangible user interface: The effects of haptic perception and selective pointing on a spatial learning task. Comput. Educ. 2015, 92–93, 64–75. [Google Scholar] [CrossRef]
  6. Hightower, B.; Lovato, S.; Davison, J.; Wartella, E.; Piper, A. Haptic explorers: Supporting science journaling through mobile haptic feedback displays. Int. J. Hum. Comput. Stud. 2019, 122, 103–112. [Google Scholar] [CrossRef]
  7. Williams, R.; Meng-Yun, C.; Seaton, J. Haptics-Augmented Simple-Machine Educational Tools. J. Sci. Educ. Technol. 2003, 12, 1–12. [Google Scholar] [CrossRef]
  8. Manches, A.; O’malley, C. Tangibles for learning: A representational analysis of physical manipulation. Pers. Ubiquitous Comput. 2012, 16, 405–419. [Google Scholar] [CrossRef]
  9. Hayes, J.C.; Kraemer, D.J. Grounded understanding of abstract concepts: The case of STEM learning. Cogn. Res. Princ. Implic. 2017, 2. [Google Scholar] [CrossRef] [Green Version]
  10. Han, I.S. Feel, Imagine and Learn!—Haptic Augmented Simulation and Embodied Instruction in Physics Learning; Teachers College, Columbia University: ProQuest LLC.: Ann Arbor, MI, USA, 2010. [Google Scholar]
  11. Paas, F.; Sweller, J. An Evolutionary Upgrade of Cognitive Load Theory: Using the Human Motor System and Collaboration to Support the Learning of Complex Cognitive Tasks. Educ. Psychol. Rev. 2012, 24, 27–45. [Google Scholar] [CrossRef] [Green Version]
  12. Wilson, M. Six Views of Embodied Cognition. Psychon. Bull. Rev. 2002, 9, 625–636. [Google Scholar] [CrossRef]
  13. Skulmowski, A.; Rey, G.D. Measuring Cognitive Load in Embodied Learning Settings. Front. Psychol. 2017, 8, 1191. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Mitra, R.; McNeal, K.S.; Bondell, H.D. Pupillary response to complex interdependent tasks: A cognitive-load theory perspective. Behav. Res. Methods 2017, 49, 1905–1919. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  15. Jong, T. Cognitive load theory, educational research, and instructional design: Some food for thought. Instr. Sci. 2010, 38, 105–134. [Google Scholar] [CrossRef] [Green Version]
  16. Glenberg, A.M.; Gallese, V. Action-based language: A theory of language acquisition, comprehension, and production. Cortex 2012, 48, 905–922. [Google Scholar] [CrossRef]
  17. Weisberg, S.M.; Newcombe, N.S. Embodied Cognition and STEM learning: Overview of a topical collection in CR: PI. Cogn. Res. Princ. Implic. 2017. [Google Scholar] [CrossRef] [Green Version]
  18. Kontra, C.; Lyons, D.; Fischer, S.; Beilock, S. Physical experience enhances science learning. Psychol. Sci. 2015, 26, 737–749. [Google Scholar] [CrossRef] [Green Version]
  19. Jones, G.; Minogue, J.; Tretter, T.; Negishi, A.; Taylor, R. Haptic Augmentation of Science Instruction: Does Touch Matter? Sci. Educ. 2006, 90, 111–123. [Google Scholar] [CrossRef]
  20. Jones, G.; Andre, T.; Superfine, R.; Taylor, R. Learning at the Nanoscale: The Impact of Students’ Use of Remote Microscopy of Concepts of Viruses, Scale, and Microscopy. J. Res. Sci. Teach. Off. J. Natl. Assoc. Res. Sci. Teach. 2003, 40, 303–322. [Google Scholar] [CrossRef] [Green Version]
  21. Schonborn, K.J.; Bivall, P.; Tibell, L.A. Exploring relationships between students’ interaction and learning with a haptic virtual biomolecular model. Comput. Educ. 2011, 57, 2095–2105. [Google Scholar] [CrossRef] [Green Version]
  22. Nathan, M.J.; Walkington, C. Grounded and embodied mathematical cognition: Promoting mathematical insight and proof using action and language. Cogn. Res. Princ. Implic. 2017, 2, 9. [Google Scholar] [CrossRef] [Green Version]
  23. Moyer-Packenham, P.; Lammatsch, C.; Litster, K.; Ashby, J.; Bullock, E.; Roxburgh, A.; Shumway, J.; Speed, E.; Covington, E.; Hartmann, C.; et al. How design features in digital math games support learning and mathematics connections. Comput. Hum. Behav. 2019, 91, 316–332. [Google Scholar] [CrossRef]
  24. Davis, R.; Martinez, M.; Schneider, O.; MacLean, K.; Okamrua, A.; Blikstein, P. The Haptic Bridge: Towards a Theory for Haptic-Supported Learning; Association for Computing Machinery: New York, NY, USA, 2017. [Google Scholar]
  25. Yiannoutsou, N.; Johnson, R.; Price, S. Exploring How Children Interact with 3D Shapes Using Haptic Technologies; Association for Computing Machinery: New York, NY, USA, 2018. [Google Scholar]
  26. McWilliams and Malecha. Comparing Intravenous Insertion Instructional Methods with Haptic Simulators. Nurs. Res. Pract. 2017, 2017, 4685157. [Google Scholar]
  27. Zhou, M.; Jones, D.B.; Schwaitzberg, S.D.; Cao, C.G.L. Role of Haptic Feedback and Cognitive Load in Surgical Skill Acquisition. Hum. Factors Ergon. Soc. 2007, 51, 631–635. [Google Scholar] [CrossRef]
  28. Dunkin, B.; Adrales, G.L.; Apelgren, K.; Mellinger, J.D. Surgical simulation: A current review. Surg. Endosc. 2006, 21, 357–366. [Google Scholar] [CrossRef]
  29. Sidarta, A.; van Vugt, F.T.; Ostry, D.J. Somatosensory working memory in human reinforcement-based motor learning. J. Neurophysiol. 2018, 120, 3275–3286. [Google Scholar] [CrossRef] [Green Version]
  30. Karadogan, E.; Williams, R.L. Haptic modules for palpatory diagnosis training of medical students. Virtual Real. 2013, 17, 45–58. [Google Scholar] [CrossRef]
  31. Escobar-Castillejos, D.; Noguez, J.; Neri, L.; Magana, A.; Benes, B. A review of simulators with haptic devices for medical training. PubMed 2016, 40, 104. [Google Scholar] [CrossRef] [PubMed]
  32. Mahoney, L.; Csima, A. Efficiency of palpation in clinical detection of breast cancer. Can. Med. Assoc. J. 1982, 127, 729–730. [Google Scholar] [PubMed]
  33. Li, M.; Konstantinova, J.; Secco, E.; Jiang, A.; Liu, H.; Nanayakkara, T.; Senviratne, L.; Dasgupta, P.; Althoefer, K.; Wurdemann, H. Using visual cues to enhance haptic feedback for palpation on virtual model of soft tissue. Med. Biol. Eng. Comput. 2015, 53, 1177–1186. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Overtoom, E.; Horeman, T.; Jansen, F.; Dankelman, J.; Schreuder, H. Haptic Feedback, Force Feedback, and Force-Sensing in Simulation Training for Laparoscopy: A Systematic Overview. J. Surg. Educ. 2018, 76, 242–261. [Google Scholar] [CrossRef]
  35. Van der Putten Westebring, E.P. A Sense of Touch in Laparoscopy: Using Augmented Haptic Feedback to Improve Grasp Control. 2011. Available online: http://resolver.tudelft.nl/uuid:beebbcde-0129-4e10-94b3-241609069445 (accessed on 24 January 2011).
  36. Vapenstad, C.; Hofstad, E.; Bo, L.; Kuhry, E.; Johnsen, G.; Marvik, R.; Lango, T.; Hernes, T. Lack of transfer of skills after virtual reality simulator training with haptic feedback. Minim. Invasive Ther. Allied Technol. 2017, 26, 346–354. [Google Scholar] [CrossRef] [Green Version]
  37. Strom, P.; Hedman, L.; Sarna, L.; Kjellin, A.; Wredmark, T.; Fellander-Tsai, L. Early exposure to haptic feedback enhances performance in surgical simulator training: A prospective randomized crossover study in surgical residents. Surg. Endosc. Other Interv. Tech. 2006, 20, 1383–1388. [Google Scholar] [CrossRef] [PubMed]
  38. Kinnison, T.; Forrest, N.; Frean, S.; Baillie, S. Teaching Bovine Abdominal Anatomy: Use of a Haptic Simulator. Anat. Sci. Educ. 2009, 2, 280–285. [Google Scholar] [CrossRef] [PubMed]
  39. Parkes, R.; Forrest, N.; Baillie, S. A Mixed Reality Simulator for Feline Abdominal Palpation Training in Veterinary Medicine. 2009. Stud. Health Technol. Inform. 2009, 142, 244–246. [Google Scholar] [PubMed]
  40. Bateman, A.; Zhao, O.; Bajcsy, A.; Jennings, M.; Toth, B.; Cohen, A.; Horton, E.; Khattat, A.; Kuo, R.; Lee, F.; et al. A user-centered design and analysis of an electrostatic haptic touchscreen system for students with visual impairments. Int. J. Hum. Comput. Stud. 2018, 109, 102–111. [Google Scholar] [CrossRef]
  41. Darrah, M. Computer haptics: A new way of increasing access and understanding of math and science for students who are blind and visually impaired. J. Blind. Innov. Res. 2013, 3, 3–47. [Google Scholar] [CrossRef]
  42. Bussell, L. Touch Tiles: Elementary Geometry Software with a Haptic and Auditory Interface for Visually Impaired Children. 2003. Available online: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.3.4699&rep=rep1&type=pdf (accessed on 7 February 2020).
  43. Avila-Soto, M.; Bahamondez, E.; Schmidt, A. TanMath: A Tangible Math Application to Support Children with Visual Impairment to Learn Basic Arithmetic; Association for Computing Machinery: New York, NY, USA, 2017. [Google Scholar]
  44. Murphy, K.; Darrah, M. Haptics-based apps for middle school students with visual impairments. IEEE Trans. Haptics 2015, 8, 318–326. [Google Scholar] [CrossRef]
  45. Fernandes, L. The Abacus: A Brief History. Available online: https://www.ee.ryerson.ca/~elf/abacus/history.html (accessed on 31 July 2014).
  46. Sanchez, J. Development of navigation skills through audio haptic videogaming in learners who are blind. Procedia Comput. Sci. 2012, 14, 102–110. [Google Scholar] [CrossRef] [Green Version]
  47. Sjostrom, C. Using Haptics in Computer Interfaces for Blind People; Association for Computing Machinery: New York, NY, USA, 2001. [Google Scholar]
  48. Hansen, E.; Liu, L.; Rogat, A.; Hakkinen, M.; Darrah, M. Designing Innovative Science Assessments That Are Accessible for Students Who Are Blind. J. Blind. Innov. Res. 2016, 6, 1. [Google Scholar] [CrossRef]
  49. Roessingh and Bence. Embodied Cognition: Laying the Foundation for Early Language and Literacy Learning. Lang. Lit. 2018, 20, 23–39. [Google Scholar]
  50. Bara, F.; Gentaz, E.; Cole, P. Haptics in learning to read with children from low socio-economic status families. Br. J. Dev. Psychol. 2007, 25, 643–663. [Google Scholar] [CrossRef]
  51. Jiao, Y.; Severgnini, F.; Martinez, J.; Jung, J.; Tan, H.; Reed, C.; Wilson, E.; Lau, F.; Israr, A.; Turcott, R.; et al. A Comparative Study of Phoneme- and Word-Based Learning of English Words Presented to the Skin; Springer: Berlin/Heidelberg, Germany, 2018. [Google Scholar]
  52. Dunkelberger, N.; Bradley, J.; Sullivan, J.; Israr, A.; Lau, F.; Klumb, K.; Abnousi, F.; O’Malley, M. Improving Perception Accuracy with Multi-Sensory Haptic Cue Delivery; Springer: Berlin/Heidelberg, Germany, 2018. [Google Scholar]
  53. Palluel-Germain, R.; Bara, F.; Boisferon, A.; Hennion, B.; Gouagout, P.; Gentaz, E. A visuo-haptic device-telemaque-increases kindergarten children’s handwriting acquisition. In Proceedings of the Second Joint EuroHaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (WHC’07), Tsukuba, Japan, 22–24 March 2007. [Google Scholar]
  54. Teranishi, A.; Korres, G.; Park, W.; Eid, M. Combining Full and Partial Haptic Guidance Improves Handwriting Skills Development. IEEE Trans. Haptics 2018, 11, 509–517. [Google Scholar] [CrossRef] [PubMed]
  55. Xiong, M.; Milleville-Pennel, I.; Dumas, C.; Palluel-Germain, R. Comparing Haptic and Visual Training Method of Learning Chinese Handwriting with a Haptic Guidance. JCP 2013, 8, 1815–1820. [Google Scholar] [CrossRef]
  56. Kim, Y.S.; Collins, M.; Bulmer, W.; Sharma, S.; Mayrose, J. Haptics Assisted Training (HAT) System for Children’s Handwriting. In Proceedings of the IEEE World Haptics Conference (WHC’13), Daejeon, Korea, 14–17 April 2013. [Google Scholar]
  57. Ogawa, D.; Ikeno, S.; Okazaki, R.; Hachisu, T.; Kajimoto, H. Tactile Cue Presentation for Vocabulary Learning with Keyboard; Association for Computing Machinery: New York, NY, USA, 2014. [Google Scholar]
  58. Kasahara, S.; Takada, K.; Nishida, J.; Shibata, K.; Shimojo, S.; Lopes, P. Preserving Agency during Electrical Muscle Stimulation Training Speeds up Reaction Time Directly after Removing EMS; Association for Computing Machinery: New York, NY, USA, 2021. [Google Scholar]
  59. Erp, J.; Saturday, I.; Jansens, C. Application of Tactile Displays in Sports: Where to, How and When to Move. Available online: https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.505.5924&rep=rep1&type=pdf (accessed on 6 July 2021).
  60. Mayer, R.E.; Moreno, R. Nine Ways to Reduce Cognitive Load in Multimedia Learning. Educ. Psychol. 2003, 38, 43–52. [Google Scholar] [CrossRef] [Green Version]
  61. Auksztulewicz, R.; Spitzer, B.; Goltz, D.; Blankenburg, F. Impairing somatosensory working memory using rTMS. Eur. J. Neurosci. 2011, 34, 839–844. [Google Scholar] [CrossRef]
  62. Ayelet, S. Do Gestural Interfaces Promote Thinking? Embodied Interaction: Congruent Gestures and Direct-Touch Promote Performance in Math. Ph.D. Thesis, Columbia University, New York, NY, USA, 2011. [Google Scholar]
  63. Olympiou, G.; Zacharia, Z.C. Blending Physical and Virtual Manipulatives: An Effort to Improve Students’ Conceptual Understanding Through Science Laboratory Experimentation. Sci. Educ. 2011, 96, 21–47. [Google Scholar] [CrossRef]
Figure 1. Theoretical framework for Cognitive Load Theory (CLT) and Embodied Cognition (EC).
Figure 1. Theoretical framework for Cognitive Load Theory (CLT) and Embodied Cognition (EC).
Applsci 11 06245 g001
Table 1. Factors that influence the effectiveness of a learning environment (Adapted from [47]).
Table 1. Factors that influence the effectiveness of a learning environment (Adapted from [47]).
NavigationFinding ObjectsUnderstanding ObjectsHaptic WidgetsPhysical Interaction
(1) Include easily found, well-defined reference points in the virtual environment.(1) To reduce instances of the user missing objects, use salient notifiers, such as “enlarged interaction points, magnetic objects, or different surface characteristics”.(1) When realism is not paramount, allowing the user to feel just the outline of an object is beneficial.(1) The finger tends to accelerate when passing a wall or edge.(1) The haptic tool/device (stylus, thimble, joystick, etc.) utilized affects the perceived sensation significantly and should be carefully selected.
(2) If possible, avoid changing the reference system.(2) Provide paths to objects, such as grooves, magnetic lines, or ridges.(2) Rounded shapes without sharp corners or edges are typically easier to understand.(2) Avoid placing adjacent walls/edges too close in proximity as they may be missed.-
-(3) One may also consider including a “virtual search tool”.---
Table 2. Observation and best design practices based on learning domain and discipline.
Table 2. Observation and best design practices based on learning domain and discipline.
Learning DomainDisciplineSourceOutcome of Haptics
Augmentation
Best Design Practices
STEMPhysics[4]Haptic augmented simulations promoted better learning outcomes.The force feedback experienced during haptic simulation was beneficial in physics learning scenarios.
[18]Students tactically interacting with physics concept reported better learning outcomes.Increased embodiment (and related neural activity) is correlated with better learning outcomes.
[63]The blending of physical and virtual manipulatives provides beneficial learning affordances.Virtual manipulatives can prevent natural error (thus reducing distraction when learning a novel task), alter time, simplify real-world models, provide immediate feedback, focus attention on key concepts, and cut down on experimental time and cost.
Physical manipulatives can help users deal with measurement and ambiguity, invoke the senses for observation, bolster the acquisition of psychomotor skills, allow user to experience certain characteristics of a concrete object, and allow for real-world data analysis.
Biology[6]Haptic augmentation of a mobile science journal promoted more “on-task” conversation than the control condition.Specific texture fidelity was secondary if the goal of haptic augmentation was to guide attention toward certain parts of an artifact or prompt particular interactions.
The haptic system performed well when children had the option to add each modality in sequence, rather than all at once.
[18]Students in haptic group had better learning experiences and outcomes.Higher fidelity/higher sensitivity haptic devices promoted the use of more haptic terms to describe the learning material.
[19]Haptic augmentation is useful for teaching microscale morphology.Full haptic feedback (versus partial) promoted better understanding of the learning material.
[21]Haptic feedback helps students locate feasible biomolecular docking positions more efficiently.The realistic awareness that haptic augmentation imparts can induce learning benefit when learning about microscale concepts.
Mathematics[24]Symbolic haptic feedback can be useful in facilitating conceptual understanding.Symbolic haptics are as effective as how they are interpreted and understood by the user.
[23]Mathematical proficiency can be improved though digital games.Linking physical actions with mathematical concepts (e.g., tracing shapes with fingers) promotes positive learning outcomes.
Helpful digital game design features include: providing user with unlimited/multiple attempts, information tutorials and hints, focused constraint, progressive levels, game efficiency, linked representation, and linked physical actions.
[59]Direct touch interfaces better support visual-motor coordination and promote student use of advanced strategies for problem solving.Actions were found to positively affect cognition when they were congruent with participant thinking (e.g., tapping for discrete counting or sliding for continuous number line estimation).
[25]Learning potential for haptic augmentation was identified.Haptic feedback provided simultaneously with visual feedback tended to be overlooked (Colavita visual dominance effect).
MedicineOsteopathic[30]Subjective reports indicate majorly positive reports across metrics for usability, effectiveness, and clarity. Haptics were favorable due to lower time constraints and fewer consequences.According to subjective reports, the least favorable model was the “Pitting Edema” haptic module. This module was the only one to provide simultaneous haptic and visual feedback. Visual dominance was observed.
Veterinary[38]Haptic simulation was proven to help students in understanding of bovine abdominal anatomy and 3D visualization.Subjective feedback determined that there was a student desire to be able to feel the haptic simulation with the whole hand as opposed to one finger.
[39]Positive feedback from veterinarians was encouraging, suggesting that the feline abdominal haptic simulator has a potential role in veterinary student training.The veterinarians suggested that there would be benefits in using the simulator before, and as a complement to, examining live animals, particularly as the instructor can follow the movements “inside” the cat on the computer monitor while directing the trainee and identifying structures palpated.
[27]Haptic feedback enhances performance during a laparoscopic surgery simulation and is thought to reduce cognitive load.Exaggerated haptics led to faster and more accurate completion of the surgery simulation.
General[31]Virtual environments provide a viable alternative to traditional methods of medical training.Current haptic technologies are thought to be hindered by their lack of realism.
Different levels of technology (DoF, DoFF, immersion, visual realism) may suit various simulations better; likewise, required level of visual realism varies significantly between different medical simulations.
Haptic simulations may benefit from providing progressive levels.
[33]Combining haptic and visual feedback led to the best simulation results.Addition of visual modality to the haptic palpation simulation increased user sensitivity by 5% and the positive predictive value by 4%, and decreased tumor detection time by 48.7%
[26]Haptic simulators were at least equal to the traditional method of teaching IV insertion.The most effective method was concluded to be haptic simulation followed by a physical practice.
[34]Haptic feedback can lead to improved learning gains for laparoscopic procedures.Haptic feedback is more important for learning complex tasks.
Skill improvement was minor and less pronounced for novice surgeons.
[37]Haptic augmentation improved skill acquisition.Haptic augmentation experienced in the early training phase of skill acquisition improved learner outcomes optimally.
[36]Skills acquired through VR haptic simulator did not transfer to a clinical setting.Haptic realism and fidelity were determined to be key factors for positive learning outcomes.
[35]Haptic augmentation can improve laparoscopic surgical outcomes.Haptic sensors that measure exerted forces should be located in the tip of the grasper tool.
Immediate real-time feedback is required for best results.
[1]Haptics have proven to be a beneficial tool for improving clinical proficiency, while decreasing medical costs and errors.There is room for improvement on haptic realism.
[41]Haptic augmentation can be a useful tool for teaching math and science to visually impaired students.Students appreciated the implementation of “self-checks” that gave them feedback on their understanding.
Students expressed desire for changes in screen readers’ voices and pronunciation, button use and placement on the device, and additional instructions.
Visually disabled [43]Preliminary usability testing and subjective reports were positive.Size of magnetic numbers used as physical icons in the study were thought to be too small to be suitable for tangible perception, thereby requiring prior knowledge of the physical icons.
The user required better guidance during the learning task to properly align the magnetic numbers on the work surface.
[40]The haptic touchscreen meets the desired accessibility needs of the visually impaired.The best methods of haptic touchscreen exploration were systemic and rapid exploration of the screen. Haptic elements placed near the corners of the screen were more easily located.
[42]Haptics provide the visually impaired with an additional way to interact with subject material.Best design: sharp corners can be disorienting; some textures are more confusing than others (e.g., grid texture was mistaken for moving square); strength of force feedback was influential for shape identification, with stronger feedback being preferable.
[48]Subjective student reports indicated student preference for haptic augmentation over other methods of instruction.The 3D experience of Novint Falcon presented difficulties during a particle counting task. Users of the vibrotactile screen also struggled to locate particles during task. Background vibration was thought to impair task success.
[46]The Audio Haptic Maze (AHM) virtual game was found to improve orientation and mobility skills in visually impaired users.Pure sounds and tones were most effective, as opposed to complex sounds. Simpler shapes were easier to identify via haptic exploration.
[47]Haptics offer visually impaired learners an additional way of interacting with learning material.Best design: provide well-defined and consistent reference points; connect haptic objects and provide haptic path to objects; provide a virtual search tool; sharp edges and corners are more difficult to feel and understand than rounded shapes; if realism is not necessary, the outline of an object may be more easily understood; replacing boundaries with a magnetic line that pulls user to the center can be helpful; manipulandum can affect haptic sensation.
[44]Haptic augmentation can be a useful tool for visually impaired students.Inclusion of audio information was a helpful addition to the haptic simulation.
[50]Performance in letter recognition, initial phenome identification improved more after haptic intervention versus just visual intervention.Not available.
Language and handwriting acquisition [52]Learning and decoding language via haptic symbols on skin is feasible through a wearable device.Not available.
[53]Fluency of handwriting was higher after visuo-haptic training than for control training.Not available.
[51]Learning and decoding language via haptic symbols on skin is feasible.Phenome-based haptic approach provides the most consistent approach for learning.
[56]Children’s tracing performance on the HAT system was improved for all groups receiving haptic feedback.Children using the high-end device (Phantom Omni) in the standard class outperformed all other groups across all measures.
[49]Embodied role in language and literacy learning is emphasized. Roessignh and Bence offer a “Play-Based Pedagogy” framework that emphasizes sensorimotor engagement.Not available.
[54]Haptic handwriting guidance is found to be effective and pleasant for participants.A combination of full and partial haptic guidance resulted in statistically significant improvements in the quality of handwriting. Implementing partial haptic guidance in the early stages of learning and then using full haptic guidance during later stages was found to be the most effective training method. Partial haptic guidance was thought to be effective in teaching the gross shape of handwriting skills, whereas full haptic guidance was thought to be more effective in teaching fine details.
[55]Implementation of haptic information showed significant improvement for the transfer of shape learning.The combination of visual and haptic information provided the best learning results, as opposed to only visual. It was concluded that visual information may benefit gross shape learning, while haptic information may benefit transfer of shape learning by creating an internal model of the shape of each stroke separately.
[57]It was found that the words learned with haptic cues lead to more effective knowledge retention measured one week after training.Providing vibrotactile feedback to fingers according to the QWERTY keyboard layout led to better retention of new vocabulary learned.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Crandall, R.; Karadoğan, E. Designing Pedagogically Effective Haptic Systems for Learning: A Review. Appl. Sci. 2021, 11, 6245. https://0-doi-org.brum.beds.ac.uk/10.3390/app11146245

AMA Style

Crandall R, Karadoğan E. Designing Pedagogically Effective Haptic Systems for Learning: A Review. Applied Sciences. 2021; 11(14):6245. https://0-doi-org.brum.beds.ac.uk/10.3390/app11146245

Chicago/Turabian Style

Crandall, Riley, and Ernur Karadoğan. 2021. "Designing Pedagogically Effective Haptic Systems for Learning: A Review" Applied Sciences 11, no. 14: 6245. https://0-doi-org.brum.beds.ac.uk/10.3390/app11146245

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop