Next Article in Journal
Geoethics, a Branding for Sustainable Practices
Next Article in Special Issue
Contradictory Aspects of Job Searching in the COVID-19 Pandemic: Relationships between Perceived Socioeconomic Constraints, Work Volition, and the Meaning of Work
Previous Article in Journal
LSTM-Based Model Predictive Control for Optimal Temperature Set-Point Planning
Previous Article in Special Issue
The Impact of Isolation Due to COVID-19 on Physical Activity Levels in Adult Students
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Sustainability
Volume 13
Issue 2
10.3390/su13020896
sustainability-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Understanding Student Acceptance of Online Learning Systems in Higher Education: Application of Social Psychology Theories with Consideration of User Innovativeness

by
Eun-Jung Kim
,
Jinkyung Jenny Kim
* and
Sang-Ho Han
*
School of Hotel and Tourism Management, Youngsan University, Busan 48015, Korea
*
Authors to whom correspondence should be addressed.
Sustainability 2021, 13(2), 896; https://0-doi-org.brum.beds.ac.uk/10.3390/su13020896
Submission received: 22 December 2020 / Revised: 14 January 2021 / Accepted: 15 January 2021 / Published: 17 January 2021
(This article belongs to the Special Issue University Education in the Age of COVID-19: Transformations and Challenges for Students and Teachers)
Browse Figures
Versions Notes

Abstract

:
The COVID-19 pandemic has created the opportunity to conduct a huge experiment with online classes in the university setting. The objectives of this study are (1) to provide obvious insight into determining factors driving greater intention of students to use online learning systems based on an integrated technology acceptance model (TAM) and theory of planned behavior (TPB), and (2) to provide the moderating role of innovativeness as a key factor. An online survey was conducted with 216 university students taking hospitality and tourism studies in Busan, Korea. The results of the study are as follows. First, perceived ease of use affects perceived usefulness, perceived usefulness affects attitude, whereas perceived ease of use does not directly affect attitude. Second, attitude and subjective norms positively influence behavioral intention, while perceived behavioral control does not. Third, user innovativeness plays a moderating role in the relationship between subjective norms and behavioral intention. As part of the lessons learned from COVID-19, it is meaningful to provide insightful implications to academia, specifically to the college of hospitality and tourism.

1. Introduction

The recent outbreak of coronavirus disease (COVID-19) has reshaped the way we live in many aspects. According to current evidence, the novel coronavirus is primarily transmitted between people through contact routes and respiratory droplets [1]. Due to this nature of the coronavirus, social distancing has been the common practices around the world. In other words, authorities around the world began to restrict the movement of people and enforce physical distancing to stem COVID-19 [2]. Such rules and regulations were not limited to universities [3,4,5]. Many universities across different regions sent out notices that all classes would shift from traditional in-person classes to online classes until such time when students can safely resume face-to-face lectures on campus [6,7]. Indeed, the COVID-19 pandemic has created the opportunity to conduct a huge experiment with online classes in the university setting.
Many attempts have been made to predict individual acceptance of technology-based products/services based on existing theories. Of various theories, the technology acceptance model (hereafter TAM) is one of the most cited theoretical frameworks. TAM posits perceived ease of use and perceived usefulness which aid in building a favorable attitude and subsequently increase behavioral intention toward to a particular technology [8]. In the meantime, the theory of planned behavior (hereafter TPB) is another conceptual model that has been frequently employed in numerous studies to determine the driving forces of individual behavior. In TPB, individual behavioral intentions are formulated by attitude, subjective norms, and perceived behavioral control [9]. Both TAM and TPB were adapted from the theory of reasoned action (hereafter TRA) which postulates that individual behavioral intention is completely under volitional control [10]; these two models have been widely used to examine the acceptance of technology-powered learning in higher education [11,12,13]. Furthermore, there have been other worthy endeavors that have extended existing theories by adopting extra facilitators and incorporating theories to better comprehend the formation of individual behavioral intention in the university sector [14,15,16,17].
User innovativeness has been illustrated as an important construct that influences a person’s intention to use a system [18,19,20]. User innovativeness was conceptualized as the likelihood of an individual to pursue novel products or services in early stages [21]. Particularly, the moderating impact of individual innovativeness was identified in the relationship among key study variables of TAM and TPB in various sectors [22,23,24]. However, limited findings of roles of user innovativeness exist in the field of higher education, and thus it would be meaningful to examine its impact in the formation of students’ adoption of online learning systems.
New forms of learning through online system such as a remote learning and virtual classrooms were forced on to universities after the outbreak of COVID-19 [4,25]. This may not be extremely new to all; however, it was considered a sudden change for many students in universities where learning environments used to be face-to-face. In addition, online methods require not only a high level of self-directed learning in terms of students’ volition and skills but also a high level of readiness for technological capacity of digital learning platforms [25]. Nonetheless, there was no sufficient preparation for this transition of learning style for students during COVID-19. Likewise, students at university have exhibited discontent about recent online education during the pandemic. For example, according to a survey at 203 universities in Korea conducted by the National University Student Council Network, more than 99 percent complained about online lectures [26]. Moreover, there are quite substantial amounts of practical and craft-based training in the hospitality sector that require more hands-on learning to be exercised throughout the course [5,27]. Thus, online learning in the hospitality context is to some extent even more challenging. Despite the aforementioned numerous studies on predicting student adoption of online learning systems in higher education based on the existing theories, no attempts were made to test the level of students’ acceptance in the area of hospitality and tourism.
The ultimate way of preventing the coronavirus is still unknown and some educators expect online classes will be the new normal even after COVID-19 [7]. Likewise, universities such as Cambridge University have announced that all lectures will be online only until summer of 2021 [28]. Meanwhile, to utilize systems to their full potential, it is first necessary to investigate individual intention to use such systems [29]. Therefore, it is of great importance to assess current student acceptance level of online learning systems in the college of hospitality and tourism for the coming semesters and years. Given this, the purposes of this research are to provide a clear understanding of the determining factors in inducing greater intention for students to use online learning systems on the basis of incorporated TAM and TPB, with a consideration of innovativeness as a key moderator. Accordingly, the present study will contribute to providing insightful implications to academia, specifically to colleges of hospitality and tourism, as part of the lessons from COVID-19.

2. Literature Review

2.1. Online Learning Systems in Higher Education

The application of online learning system as the part of edutech has continued to grow in higher education across the globe [30]. The same phenomenon has occurred in South Korea and online education, which includes learning through the medium of computers and mobile phones has gained general acceptance over the last few years [13,31]. Likewise, the concept of online classes through a designated platform might not be absolutely new to instructors and students. However, a large proportion of lectures have generally been managed in the university classroom, and thus, it is still a novel concept in the university setting due to the suddenly enforced physical distancing after the outbreak of COVID-19 [3,5]. During the lockdown and social-distancing period, instructors were required to record lectures through an online platform in advance, and students tool courses by watching recorded programs aired on the system. Alternatively, there were online video classrooms allowing real-time connection between students and instructors through a cloud platform.
Online learning systems can supplement or replace traditional classrooms enabling students to engage in learning through various tools and web technologies [31,32]. Concannon et al. [33] asserted that the new generation of students would accelerate the demand of integrating smart technologies into higher education. Moreover, they stated that universities, through a wide range of implementation of innovative technologies into learning model, could attract and engage many more students. However, there is still significant resistances to technology in the education sector [34,35]. Wang et al. [5] differentiated operational and practical courses from knowledge-based courses and asserted that technological innovation in the domain of education should be treated differently. They accordingly suggested that respective curricula in online learning technologies for operational courses should be modified. Therefore, it is expected that the acceptance behavior of students toward online learning systems depends on the course they take if it is theoretical or practical learning course.

2.2. Technology Acceptance Model (TAM)

TAM is a derivative of the TRA [10,36]. TAM determines that two fundamental elements, namely perceived ease of use and perceived usefulness, are primary triggers of individual attitude toward technology-powered products/services which eventually induces behavioral intention [8]. Perceived ease of use denotes the degree to which individuals expect that using a specific applications will be free of effort, whereas perceived usefulness describes individuals’ subjective probability to believe that using a specific technology will increase their work performance. According to TAM, these two determinants aid in building attitude, which refers to “the degree to which a person has a favorable or unfavorable evaluation or appraisal of the behavior in question” [9]. Attitude consequently influences individual behavioral intention, reflecting the level to which an individual formulates conscious plans to engage in a particular behavior [37].
The application of TAM has long been widely popular to test individual acceptance of various smart technologies in the education context [18,34,38,39]. Saade et al. [29]. conducted an empirical study based on 362 responses from students in higher education institutions; they provided evidence that TAM is a solid theoretical model in the e-learning context. Al-Adwan et al. [32] examined indicators of the acceptance of e-learning from students’ standpoint based on TAM; their results supported the application of TAM to predict user intention to accept e-learning systems. Abdullah and Ward [14] developed GETAMEL, a general extended technology acceptance model for e-learning grounded in TAM; they validated the theory as a fundamental theoretical framework for explicating student intention to use e-learning systems. More recently, Granić and Marangunić [39] reviewed a respectable number of studies built on TAM between 2003 and 2018 in the domain of technology-based learning. They established a firm foundation of TAM in predicting individual intention in the educational context.

2.3. Theory of Planned Behavior (TPB)

TPB is an extension of TRA; it suggests attitude and subjective norms as major driving forces behind behavioral intention [10,36]. The core mechanism of TPB is that individual behavioral intention is determined by not only volitional factors, namely, attitude and subjective norms but also the non-volitional factor of perceived behavioral control. As explained in the previous section, attitude represents a person’s overall evaluation of a specific behavior in reference to the perceived outcomes of an act. Subjective norms refer to the extent to which an individual believes that significant numbers of others think he or she should engage in the certain behavior [40]. Perceived behavioral control involves the perception of internal and external constraints on a specific behavior [9]. That is, individual behavioral intention depends on individual assessment of a particular behavior, individual propensity to comply with the opinions of people surrounding themselves, and the level of perceptions of inner and outer constraints on behavior.
A number of studies in the field of technology-powered education have built on TPB to comprehend student acceptance of learning through online tools. For example, Chiang et al. [41] explored factors that propel student adoption of technology-integrated learning; they found that attitude as well as subjective norms are the salient drivers of intention to use such systems. Yu and Yu [42] developed a theoretical framework based on TPB to investigate factors inducing student online learning utilization. They surveyed 870 students in Taiwan; their analysis results successfully identified critical constructs that facilitate greater intention to use online learning systems. Chu and Chen [11] applied TPB to examine student acceptance of e-learning technology; they confirmed that attitude, subjective norms, and perceived behavioral control showed significant and positive correlations to e-learning intentions. Yeap et al. [43] built a mobile learning readiness framework on the basis of TPB; their results demonstrated that all three variables stemming from TPB exerted meaningful influence on student intention to adopt mobile learning. Lung-Guang [15] incorporated a self-regulated learning model into TPB to investigate intention to accept online courses at universities. Their analysis results using 222 responses showed that social norms and perceived behavioral controls are factors inducing better participation in online courses.

2.4. Incorporating TAM and TPB

Many scholars have extensively applied TAM and TPB to test individual behavioral intention in the field of education. Nonetheless, a substantial amount of criticism of TAM has been addressed to its parsimony [44,45]. In other words, the essential determinants of behavior may be overlooked and thus many efforts have been made to broaden or deepen the theory, and incorporate TAM into other theories [14,17,35]. Similarly, even though the strong predicting power of TPB has been supported in many studies, there have also been many attempts to extend TPB or merge the theory with another framework to respond to the shortcomings of the original theory [15,46,47]. In this respect, numerous scholars have endeavored to improve the predictive power of individual behavioral intention by combining TAM and TPB in the context of user behavior toward novel technology in various contexts [6,12,48,49].
Aboelmaged and Gebba [50] examined the adoption level of mobile banking by integrating TAM and TPB. Their analysis results indicate a positive association between subjective norms and system adoption and the critical impact of perceived usefulness on attitude which, in turn, influences adoption. Cheung and Vogel [48] explored the formation of student intention to participate in collaborative e-learning platform based on a combined model of TAM and TPB. Their analysis results, based on data collected from students at the Hong Kong Polytechnic University, successfully indicate that antecedent variables rooted in these two frameworks are essential determinants of student usage intention. Yu et al. [51] developed a holistic model by integrating TAM and TPB to explain the intentions to use a sharing system. Their results showed that attitude is built by perceived usefulness as well as perceived ease of use of the system, and that intention to use is positively influenced by perceived usefulness, attitude, and perceived behavioral control. Hua and Wang [49] built a combined framework of TAM and TPB to examine the drivers of consumers’ acceptance of energy-efficient applications. Based on an analysis of 280 sets of data, they found overall significant relationships among study variables rooted in these two theories. More recently, Nadlifatin et al. [16] investigated university student intention to use a blended learning system, which is a mixture of online and offline education. Their results attested to the suitability of the combined TAM and TPB model in explaining student behavioral intention. These endeavors at merging TAM and TPB demonstrate strong predictive power, helping in comprehensive understanding of user behavior. Thus, in order to predict student intention toward online learning system through an integration of TAM and TPB, we posited following hypotheses.
Hypothesis 1a (H1a).
Perceived ease of use significantly and positively affects perceived usefulness.
Hypothesis 1b (H1b).
Perceived ease of use significantly and positively affects attitude.
Hypothesis 2 (H2).
Perceived usefulness significantly and positively affects attitude.
Hypothesis 3 (H3).
Attitude significantly and positively affects behavioral intention.
Hypothesis 4 (H4).
Subjective norms significantly and positively affect behavioral intention.
Hypothesis 5 (H5).
Perceived behavioral control significantly and positively affects behavioral intention.

2.5. User Innovativeness and Its Moderating Role

User innovativeness has been described as individual propensity to accept novel or different products or services at a relatively early stage [21]. That is, a person with a greater degree of innovativeness is more likely to adopt new products/services rather than remain with previous and current selections. Likewise, Manning et al. [52] showed that individual innovativeness mirrors the self-direction value to embrace and adapt to change and novelty seeking which is the tendency to pursue novel products/services. In a similar vein, Yilmaz and Bayraktar [20] asserted that individual openness to change and to adopt an innovation depends on that person’s level of innovativeness. Hence, user innovativeness has been constantly validated as an important indicator of successful diffusion of newly introduced products or services, particularly novel technology-based ones. For example, Mahat et al. [19] focused on mobile learning, which became a new paradigm of higher education in Malaysia and confirmed that students’ personal innovativeness influenced their intention to participate in mobile learning. Arpaci [18] examined the adoption of cloud computing in the education and the author demonstrated that users’ innovativeness plays a vital role in adopting new technology-powered system.
In addition, user innovativeness has been identified as an important moderator in the acceptance of technology-powered products/systems [22,23,53]. Lee, Qu, and Kim [54] postulated that individual behavior of adopting online shopping varies according to individual innovativeness; they examined their hypotheses using 208 sets of data. Their results indicated that a group of people with a high level of innovativeness will be largely influenced by their attitudes when they embrace online shopping, whereas individuals who are less innovative tend to count on attitude and subjective norms to mitigate the uncertainty inherent in online transactions. Crespo and del Bosque [55] analyzed the factors that induce individuals to adopt electronic commerce in consideration of personal innovativeness. Their results show the significant moderating impact of personal innovativeness in the link between attitude toward the system and behavioral intention. Similarly, Ahmed et al. [22] confirmed that individual innovativeness moderated the link between attitude and future intention to use online shopping. Leicht et al. [53] provided evidence that individual innovativeness moderates the association between social influence and behavioral intention toward autonomous car adoption. In an educational context, Matute-Vallejo and Melero-Polo [24] predicted student acceptance of an online simulation and confirmed the moderating role of personal innovativeness. More specifically, their results revealed that the relationship between perceived ease of use and attitude was moderated by student innovativeness. This existing evidence supports user innovativeness as a robust moderator in the formation of individual usage intention toward technology driven products/systems. Given this, we hypothesized followings to identify the moderating role of user innovativeness in the development of student intention toward online learning system.
Hypothesis 6a (H6a).
User innovativeness moderates the link between perceived ease of use and attitude.
Hypothesis 6b (H6b).
User innovativeness moderates the link between perceived usefulness and attitude.
Hypothesis 6c (H6c).
User innovativeness moderates the link between attitude and behavioral intention.
Hypothesis 6d (H6d).
User innovativeness moderates the link between subjective norms and behavioral intention.
Hypothesis 6e (H6e).
User innovativeness moderates the link between perceived behavioral control and behavioral intention.
Figure 1 displays our research model, which involves a total of seven latent constructs and contains a total of eleven hypotheses. In the theoretical framework, the initial constructs and links of TAM are outlined within the green highlighted lines; the original variables and paths of TPB are indicated in the blue box.

3. Methodology

3.1. Data Collection and Profile of Survey Respondents

The survey was conducted to provide a clear understanding of determining factors in inducing greater intention for students to use online learning systems. An online survey set up on Google.
The data were collected in the first of half of June 2020, prior to the end of the semester, for college students taking hospitality courses in B area. A total of 220 responses were collected; 216 were retained for analysis after excluding four responses that were identified as outliers. The sample size of this study was greater than 187 samples at the significance level of 5% [56]. Utilizing SPSS 24.0 and SmartPLS 3.0 programs, this study performed frequency analysis, confirmatory factor analysis, correlation analysis, and reliability analysis to assess the reliability and validity of the measurement tools, and structural modeling analysis to examine the proposed research model and test the hypotheses.
Men accounted for 56.9 percent of the respondents. 41.7% of all respondents were freshman, followed by 20.8% for sophomore, 20.4% for junior, and 16.2% for senior. By majors, social science was the most numerous at 74.2%, arts and science students made up 13%, and humanities students were 7%. Table 1 provides a summary of the demographic characteristics of respondents.

3.2. Measures

All items were measured using a seven-point Likert scale ranging from “strongly disagree” to “strongly agree”.
The perceived ease of use was defined as the degree to which students expected to have no difficulty in using the e-learning system; the usefulness was defined as the individual’s subjective probability that the e-learning system would increase the effectiveness of the class. Attitudes were defined to the extent that students provided a favorable assessment of the behavior of using online learning systems. The subjective norm was defined to the extent that students believed other students should participate in online learning; perceived behavioral control was defined as the perception of internal and external restrictions on participation in the e-learning system. Individual behavioral intention was defined as student intention to use the e-learning system. User innovation has been defined as the individual propensity for university students to embrace e-learning systems.
Each of the five items were used to evaluate perceived ease of use and perceived usefulness. These measurement items were borrowed from Fishbein and Ajzen [10]; they were modified to fit the context of online education. Attitude, subjective norms, and perceived behavioral control were measured with four items, each adapted from Fishbein and Ajzen [10], Lung-Guang [15], and Yeap et al. [43]. Individual behavioral intention was assessed with three measurement items cited from Lung-Guang [15] and Yeap et al. [43]. Furthermore, user innovativeness was measured using five items borrowed from Matute-Vallejo and Melero-Polo [24] and modified.

4. Results

4.1. Analysis of Validity and Reliability

To evaluate the measurement model and the structural model [57], we used SmartPLS 3.0, a two-step procedure including a bootstrapping technique [58]. First, reliability analysis was performed using Cronbach’s α and Research Unit Reliability (CR) to measure the internal consistency of the variables used in the study. As Table 2 displays, Cronbach’s α values of each study variable ranged from 0.887 to 0.938, rho_A values were ranged from 0.887 to 0.987, and C.R. values were found to be between 0.924 and 0.967. It was found that the reliability between the measurement items exceeded the standard of 0.70. Next, to measure the validity of the concept, the validity of convergence and the validity of discrimination were determined and verified.
Convergence validity is indicating as a high correlation between the same concepts. The factor loading values were all 0.796 or more, and the AVE value was 0.732 or more, confirming the convergence validity of each research unit in Table 2.
In addition, among the latent variables, the square root of AVE in each construct was greater than the other correlation values (see Table 3). Thus, discriminant validity is well established.

4.2. Assessment of Structural Model

For this study, SmartPLS 3.0 was used and structural models were evaluated according to the following criteria [56,59].
First, as a result of evaluation using VIF (the variance inflation factor), VIF was found to range from 1.926 to 4.971, indicating that there was no problem with multicollinearity [56,60].
Second, the predictive power of the model was assessed using variance explained (R2) in endogenous constructs. As shown in Table 4, the R2 of the outcome variable was 0.610 to 0.772; higher than the reference value of 0.10 [61,62].
Third, as can be seen in Table 4, Q2 (the Stone–Gesser test) value was higher than 0, indicating that there was no problem with the predictive relevance of the reflection measurement model and the endogenous structure.
Finally, the value of SRMR (the standardized root mean square residual) was 0.084, which is regarded as indicating a good fit [63].

4.3. Hypotheses Testing

Hypothesis 1 states that perceived ease of use will directly influence affective perceived usefulness and attitude. As shown in Table 4, perceived ease of use has significant positive impact on perceived usefulness (β = 0.782, t = 28.958, p < 0.01). Thus, H1a was supported. However, perceived ease of use does not have a positive effect on attitude (β = 0.083, t = 1.215, n.s), and thus H1b is not supported.
Hypothesis 2 postulates that affective perceived usefulness will directly affect cognitive attitude. The results of analyses were that affective perceived usefulness has a significant and positive effect on cognitive attitude (β = 0.809, t = 14.024, p < 0.01), and so H2 was supported.
Hypothesis 3 and Hypothesis 4 were supported, because attitude (β = 0.579, t = 7.897, p < 0.01) and subjective norms (β = 0.093, t = 2.004, p < 0.05) have significant effects on behavioral intention.
Finally, Hypothesis 5 was not supported, because perceived behavioral control does not have a significant effect on behavioral intention (β = −0.044, t = 0.975, ns).

4.4. Moderating Analysis

The results of the analysis to test the moderating effect of user innovativeness are exhibited in Table 5. First, user innovativeness has been shown to have no moderate effect between attitude and perceived ease of use (e.g., t = −0.031, t = 0.449, p = 0.453), and perceived usefulness (e.g., t = 0.741; p = 0.459). Therefore, H6a and H6b were not supported. User innovativeness, also, does not moderate the association between attitude and behavioral intention (ß = 0.085; t = 1.529, p = 0.126), hence H6c is not supported. User innovation regulates the relationship between subjective norms and behavioral intention (ß = −0.119; t = 2.840, p = 0.005), so H6d is supported. This is consistent with our postulation that user innovativeness will play a moderating role in subjective norms and behavioral intention, as depicted in Figure 2. The results show that the higher the subjective norms for lower innovativeness group, the greater the change in behavioral intention to accommodate online learning systems than the higher innovativeness group. Finally, user innovativeness also does not moderate the association between perceived behavioral control and behavioral intention (ß = −0.020; t = 0.455, p = 0.649), and thereby H6e is not supported.

5. Implications and Future Research

This study embraced a research model that incorporated TAM and TPB into one comprehensive theoretical framework to examine behavioral intention to use online learning systems in higher education. Further, this study investigated the moderating role of individual innovativeness. Data analyses were conducted using SmartPLS 3.0 [56]. Results of empirical analyses have the following meaningful theoretical and practical implications.

5.1. Theoretical Implications

Universities have been affected in the aftermath of the COVID-19. Concretely, they were forced to move their education programs to online space. In preparation for prolonged COVID-19, a considerable body of new studies attempts to examine students’ perspective of online learning system [64,65,66]. These studies provided evidences regarding students’ experience in using online education platforms and how it could be enhanced. Nonetheless, they did not consider if the online courses are more focused on lecture on theory or practical trainings. Unlike these studies, the present research is among the first to predict the intention toward online learning system of students majoring in hospitality and tourism which involve more practical learning.
The results show that, in higher education, student adoption of online learning systems is not only explained by TAM, but also demonstrated by TPB. This finding suggests that TAM, as well as TPB jointly affect individual behavioral intention in e-learning environments. This result strongly confirms the credibility of the TAM model in facilitating assessment criteria for acceptance of diverse types of technology. We employed perceived ease of use and perceived usefulness to raise the predictive validity of TAM in this study. In the TPB model, individual behavioral intention is formulated by attitude, subjective norms, and perceived behavioral control.
Specifically, this study examined the moderating effect of user personal innovativeness in the relationship among key study variables of TAM and TPB in the adoption of online learning systems. However, these relationships were not observed in the original model by Aboelmaged and Gebba [50], or in the model recently modified by Nadlifatin et al. [16]. In this regard, this study proposes a research model incorporating the moderating effect of user innovativeness into existing TAM and TPB studies in order to examine the intention to use the online learning system that emerged recent during the COVID-19 pandemic.

5.2. Practical Implications

Considering the data analysis results, there is the potential for practical implications in the development and management of online learning systems in universities.
First, it is not surprising that the development of positive attitudes among perceived ease of use, perceived usefulness, and behavioral intention are important for the acceptance of most new technologies; we have reaffirmed this to be true even in the higher education environment. University managers and faculty should make efforts to boost students’ positive attitude toward online learning, because attitude has the largest direct effect on behavioral intention to use. To boost students’ positive attitude toward online learning, universities should provide a high quality LMS environment, including Wi-Fi zones, m-learning (using smart phones), and an online mentor system. In addition, university should develop and activate massive open online courses (MOOCs) that allow students to participate in an unlimited number through the web to improve their online learning effects [67]. In particular, in the field of hospitality, there are many curriculums that require practical skills, such as hotels and tourism. In the current COVID-19 pandemic situation, it is difficult to provide students with face-to-face practical skill classes, but if faculty provide well-organized LMS, such as online practice classes that can be easily accepted by small groups, students’ positive attitude and willingness to accept online learning systems will increase. Ultimately, this type of small group practical skill class will be an important indicator of student acceptance level of online learning systems in colleges of hospitality and tourism for coming semesters and years.
Second, this study confirmed that social influence is an important construct in building student behavioral intention toward online learning systems in higher education. This finding is similar to Nadlifatin et al.’s [16] study that subjective norms have a positive relationship to the behavior intention to use blended learning system. This result can be interpreted as showing that students are highly impacted by recommendations of family and friends. Therefore, subjective norms have a significant impact on student behavioral intention to use e-learning systems in situations in which new forms of learning through online systems such as distance learning and virtual classrooms are necessarily required.
Finally, the significant moderating role of innovativeness was identified in the link between subjective norms and behavioral intention (see Figure 2). This result can be explained as showing that the higher the subjective norms are for the lower innovativeness group compared to the high innovativeness group, the higher the behavioral intention will be to accept the online learning system. Thus, university and faculty first classify students according to their innovativeness [20] and should operate an integrated online learning system so that students with low innovativeness can be influenced by students with high innovativeness. Additionally, it is commonly accepted that user segmentation is a useful management strategy to enhance effectiveness [68]. For instance, in the ongoing COVID-19 pandemic, it is necessary to apply more focused norms or systems for the low user innovativeness group to increase positive behavioral intention for student acceptance of online learning systems. Meanwhile, for highly innovative students to better accept and utilize online learning systems in the college of hospitality and tourism, it is necessary to develop diverse and systematic online practical curriculums.

5.3. Limitations and Future Study

The findings of this study present not only a baseline for clearer understanding of the psychological dynamics of online learning system users, but also provide insight for future studies that can address certain limitations in this research. First, this research was conducted with university students in Busan, the second largest city in Korea. Respondents may vary by region, so further studies of cultural orientations (e.g., individualistic or group cultures) across different regions as well as regions are needed to better understand and generalize the results presented in this study. Second, this study focused on the higher education environment, so it is somewhat difficult to apply the results of this research to other industries. Third, this study focuses on practical skills-oriented departments such as colleges of hospitality and tourism, so there is a limitation that this study cannot be applied to theory-oriented departments. Fourth, this study failed to assess the roles of prior experience and year of students in the development of student intentions. These factors may influence perceived ease of use and perceived usefulness towards online learning system, and thus future studies are recommended to take them into consideration in order to provide the more detailed implications. Finally, this study was conducted only with student respondents. To generalize the acceptance of online learning systems, a comparative study to determine differences between the two groups of students and faculty is needed.

Author Contributions

Conceptualization, E.-J.K., J.J.K. and S.-H.H.; formal analysis, E.-J.K. and S.-H.H.; methodology, E.-J.K., J.J.K. and S.-H.H.; project administration, J.J.K.; writing—original draft, E.-J.K., J.J.K., and S.-H.H.; writing—review and editing, E.-J.K., J.J.K. and S.-H.H. All authors have read and agreed to the published version of the manuscript.

Funding

This work was supported by Youngsan University Research Fund of 2020.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy concerns.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. CDC. How COVID-19 Spreads. Available online: https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/how-covid-spreads.html (accessed on 28 October 2020).
  2. Painter, M.; Qiu, T. Political beliefs affect compliance with covid-19 social distancing orders. SSRN 2020. [Google Scholar] [CrossRef]
  3. Cao, W.; Fang, Z.; Hou, G.; Han, M.; Xu, X.; Dong, J.; Zheng, J. The psychological impact of the COVID-19 epidemic on college students in China. Psychiatry Res. 2020, 287, 112934. [Google Scholar] [CrossRef] [PubMed]
  4. Viner, R.M.; Russell, S.J.; Croker, H.; Packer, J.; Ward, J.; Stansfield, C.; Mytton, O.; Bonell, C.; Booy, R. School closure and management practices during coronavirus outbreaks including COVID-19: A rapid systematic review. Lancet Child. Adoles. Health 2020, 4, 397–404. [Google Scholar] [CrossRef]
  5. Wang, C.; Cheng, Z.; Yue, X.-G.; McAleer, M. Risk management of COVID-19 by universities in China. J. Risk Financ. Manag. 2020, 13, 36. [Google Scholar] [CrossRef] [Green Version]
  6. Korea Times. Online Learning Goes Mainstream as New Normal. Available online: https://www.koreatimes.co.kr/www/nation/2020/04/181_288366.html (accessed on 20 April 2020).
  7. Nature. Universities Will Never Be the Same after the Coronavirus Crisis. Available online: https://0-www-nature-com.brum.beds.ac.uk/articles/d41586-020-01518-y (accessed on 1 June 2020).
  8. Davis, F.D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989, 13, 319–340. [Google Scholar] [CrossRef] [Green Version]
  9. Ajzen, I. The theory of planned behavior. Organ. Behav. Hum. Decis. Process. 1991, 50, 179–211. [Google Scholar] [CrossRef]
  10. Fishbein, M.; Ajzen, I. Belief, attitude, intention, and behavior: An introduction to theory and research. Philos. Rhetor. 1977, 10, 130–132. [Google Scholar]
  11. Chu, T.-H.; Chen, Y.-Y. With good we become good: Understanding e-learning adoption by theory of planned behavior and group influences. Comput. Educ. 2016, 92, 37–52. [Google Scholar] [CrossRef]
  12. Gao, H.L. A Systematic Literature Review of Technology Acceptance Model and Theory of Planned Behaviour towards Online Learning Perspective. J. Arts Humanit. 2019, 8, 75–82. [Google Scholar]
  13. Park, S.Y.; Nam, M.W.; Cha, S.B. University students’ behavioral intention to use mobile learning: Evaluating the technology acceptance model. Br. J. Edu. Technol. 2012, 43, 592–605. [Google Scholar] [CrossRef]
  14. Abdullah, F.; Ward, R. Developing a General Extended Technology Acceptance Model for E-Learning (GETAMEL) by analysing commonly used external factors. Comput. Hum. Behav. 2016, 56, 238–256. [Google Scholar] [CrossRef]
  15. Lung-Guang, N. Decision-making determinants of students participating in MOOCs: Merging the theory of planned behavior and self-regulated learning model. Comput. Educ. 2019, 134, 50–62. [Google Scholar] [CrossRef]
  16. Nadlifatin, R.; Miraja, B.; Persada, S.; Belgiawan, P.; Redi, A.; Lin, S.-C. The measurement of University students’ intention to use blended learning system through technology acceptance model (TAM) and theory of planned behavior (TPB) at developed and developing regions: Lessons learned from Taiwan and Indonesia. Int. J. Emerg. Technol. Learn. 2020, 15, 219–230. [Google Scholar] [CrossRef]
  17. Wu, B.; Chen, X. Continuance intention to use MOOCs: Integrating the technology acceptance model (TAM) and task technology fit (TTF) model. Comput. Hum. Behav. 2017, 67, 221. [Google Scholar] [CrossRef]
  18. Arpaci, I. Antecedents and consequences of cloud computing adoption in education to achieve knowledge management. Comput. Hum. Behav. 2017, 70, 382–390. [Google Scholar] [CrossRef]
  19. Mahat, J.; Ayub, A.F.M.; Luan, S. An assessment of students’ mobile self-efficacy, readiness and personal innovativeness towards mobile learning in higher education in Malaysia. Procedia Soc. Behav. Sci. 2012, 64, 284–290. [Google Scholar] [CrossRef] [Green Version]
  20. Yilmaz, O.; Bayraktar, D.M. Teachers’ attitudes towards the use of educational technologies and their individual innovativeness categories. Procedia Soc. Behav. Sci. 2014, 116, 3458. [Google Scholar] [CrossRef] [Green Version]
  21. Midgley, D.F.; Dowling, G.R. Innovativeness: The concept and its measurement. J. Consum. Res. 1978, 4, 229–242. [Google Scholar] [CrossRef]
  22. Ahmed, A.; Rehman, U.; Rizwan, M.; Rafiq, M.Q.; Nawaz, M.; Mumtaz, A. Moderating role of perceived risk and innovativeness between online shopping attitude and intention. J. Basic Appl. Sci. Res. 2013, 3, 310–323. [Google Scholar]
  23. Fatima, J.K.; Ghandforoush, P.; Khan, M.; Di Masico, R. Role of innovativeness and self-efficacy in tourism m-learning. Tour. Rev. 2017, 72, 344–355. [Google Scholar] [CrossRef]
  24. Matute-Vallejo, J.; Melero-Polo, I. Understanding online business simulation games: The role of flow experience, perceived enjoyment and personal innovativeness. Aust. J. Educ. Technol. 2019, 35, 71–85. [Google Scholar] [CrossRef]
  25. UNESCO. COVID-19 Education Response. Education Sector Issue Notes. Available online: https://unesdoc.unesco.org/ark:/48223/pf0000373305 (accessed on 16 April 2020).
  26. Yonhap News. Almost All University Students Want Tuition Refunds over COVID-19: Poll. Available online: https://en.yna.co.kr/view/AEN20200421007300315 (accessed on 16 April 2020).
  27. Sigala, M.; Baum, T. Trends and issues in tourism and hospitality higher education: Visioning the future. Tour. Hosp. Res. 2003, 4, 367–376. [Google Scholar] [CrossRef]
  28. BBC. Cambridge University: All Lectures to be Online-Only until Summer of 2021. Available online: https://www.bbc.com/news/education-52732814 (accessed on 19 May 2020).
  29. Saadé, R.G.; Nebebe, F.; Tan, W. Viability of the “Technology Acceptance Model” in multimedia learning environments: A comparative study. Interdiscip. J. Knowl. Learn. Objects. 2007, 3, 175. [Google Scholar]
  30. Allen, I.E.; Seaman, J. Online Nation: Five Years of Growth in Online Learning; ERIC: Newburyport, MA, USA, 2007. [Google Scholar]
  31. Joo, Y.J.; Lee, H.W.; Ham, Y. Integrating user interface and personal innovativeness into the TAM for mobile learning in Cyber University. J. Comput. High. Educ. 2014, 26, 143–158. [Google Scholar] [CrossRef]
  32. Al-Adwan, A.; Al-Adwan, A.; Smedley, J. Exploring students acceptance of e-learning using Technology Acceptance Model in Jordanian universities. Int. J. Educ. Dev. Using Inf. Commun. Technol. 2013, 9, 4–18. [Google Scholar]
  33. Concannon, F.; Flynn, A.; Campbell, M. What campus-based students think about the quality and benefits of e-learning. Br. J. Educ. Technol. 2005, 36, 501–512. [Google Scholar] [CrossRef]
  34. Gong, M.; Xu, Y.; Yu, Y. An enhanced technology acceptance model for web-based learning. J. Inf. Syst. Educ. 2004, 15, 365–374. [Google Scholar]
  35. Tarhini, A.; Elyas, T.; Akour, M.A.; Al-Salti, Z. Technology, demographic characteristics and e-Learning acceptance: A conceptual model based on extended technology acceptance model. High. Educ. Stud. 2016, 6, 72. [Google Scholar] [CrossRef] [Green Version]
  36. Azjen, I. Understanding Attitudes and Predicting Social Behavior; Pearson: Englewood Cliffs, NJ, USA, 1980. [Google Scholar]
  37. Oliver, C. Sustainable competitive advantage: Combining institutional and resource-based views. Strateg. Manag. J. 1997, 18, 697–713. [Google Scholar] [CrossRef] [Green Version]
  38. Farahat, T. Applying the technology acceptance model to online learning in the Egyptian universities. Procedia Soc. Behav. Sci. 2012, 64, 95–104. [Google Scholar] [CrossRef] [Green Version]
  39. Granić, A.; Marangunić, N. Technology acceptance model in educational context: A systematic literature review. Br. J. Educ. Technol. 2019, 50, 2572–2593. [Google Scholar] [CrossRef]
  40. De Groot, J.; Steg, L. General beliefs and the theory of planned behavior: The role of environmental concerns in the TPB. J. Appl. Soci. Psychol. 2007, 37, 1817–1836. [Google Scholar] [CrossRef] [Green Version]
  41. Chiang, F.; Wuttke, H.; Knauf, R.; Sun, C.; Tso, T. Students’ attitudes toward using innovative information technology for learning based on theory of planned behavior. Int. J. Adv. Corp. Learn. 2009, 2, 9–14. [Google Scholar] [CrossRef] [Green Version]
  42. Yu, T.K.; Yu, T.Y. Modelling the factors that affect individuals’ utilisation of online learning systems: An empirical study combining the task technology fit model with the theory of planned behaviour. Br. J. Educ. Technol. 2010, 41, 1003–1017. [Google Scholar] [CrossRef]
  43. Yeap, J.A.L.; Ramayah, T.; Soto-Acosta, P. Factors propelling the adoption of m-learning among students in higher education. Elect. Mark. 2016, 26, 323–338. [Google Scholar] [CrossRef]
  44. Bagozzi, R.P. The legacy of the technology acceptance model and a proposal for a paradigm shift. J. Assoc. Inf. Syst. 2007, 8, 244–254. [Google Scholar] [CrossRef]
  45. Venkatesh, V.; Davis, F.D. A theoretical extension of the technology acceptance model: Four longitudinal field studies. Manag. Sci. 2000, 46, 186–204. [Google Scholar] [CrossRef] [Green Version]
  46. Jun, J.; Arendt, S.W. Understanding healthy eating behaviors at casual dining restaurants using the extended theory of planned behavior. Int. J. Hosp. Manag. 2016, 53, 106–115. [Google Scholar] [CrossRef]
  47. Kim, J.J.; Hwang, J. Merging the norm activation model and the theory of planned behavior in the context of drone food delivery services: Does the level of product knowledge really matter? J. Hosp. Tour. Manag. 2020, 42, 1–11. [Google Scholar] [CrossRef]
  48. Cheung, R.; Vogel, D. Predicting user acceptance of collaborative technologies: An extension of the technology acceptance model for e-learning. Comput. Educ. 2013, 63, 160–175. [Google Scholar] [CrossRef]
  49. Hua, L.; Wang, S. Antecedents of consumers’ intention to purchase energy-efficient appliances: An empirical study based on the technology acceptance model and theory of planned behavior. Sustainability 2019, 11, 2994. [Google Scholar] [CrossRef] [Green Version]
  50. Aboelmaged, M.; Gebba, T.R. Mobile banking adoption: An examination of technology acceptance model and theory of planned behavior. Int. J. Bus. Res. Dev. 2013, 2, 35–50. [Google Scholar] [CrossRef]
  51. Yu, Y.; Yi, W.; Feng, Y.; Liu, J. Understanding the Intention to Use Commercial Bike-Sharing Systems: An Integration of TAM and TPB. In Proceedings of the 51st Hawaii International Conference on System Sciences, Kohala Coast, HI, USA, 3–6 January 2018. [Google Scholar]
  52. Manning, K.C.; Bearden, W.O.; Madden, T.J. Consumer innovativeness and the adoption process. J. Consum. Psychol. 1995, 4, 329–345. [Google Scholar] [CrossRef]
  53. Leicht, T.; Chtourou, A.; Youssef, K.B. Consumer innovativeness and intentioned autonomous car adoption. J. High. Technol. Manag. Res. 2018, 29, 1–11. [Google Scholar] [CrossRef]
  54. Lee, H.Y.; Qu, H.; Kim, Y.S. A study of the impact of personal innovativeness on online travel shopping behavior—A case study of Korean travelers. Tour. Manag. 2007, 28, 886–897. [Google Scholar] [CrossRef]
  55. Crespo, Á.H.; del Bosque, I.R. The effect of innovativeness on the adoption of B2C e-commerce: A model based on the Theory of Planned Behaviour. Comput. Hum. Behav. 2008, 24, 2830–2847. [Google Scholar] [CrossRef]
  56. Hair, J.F., Jr.; Hult, G.T.M.; Ringle, C.; Sarstedt, M. A Primer on Partial Least Squares Structural Equation Modeling (PLS-SEM); Sage Publications: Newbury Park, CA, USA, 2016. [Google Scholar]
  57. Anderson, J.C.; Gerbing, D.W. Structural equation modeling in practice: A review and recommended two-step approach. Psychol. Bull. 1988, 103, 411–423. [Google Scholar] [CrossRef]
  58. Hair, J.F.; Sarstedt, M.; Ringle, C.M.; Mena, J.A. An assessment of the use of partial least squares structural equation modeling in marketing research. J. Acad. Mark. Sci. 2012, 40, 414–433. [Google Scholar] [CrossRef]
  59. Han, S.; Lee, Y.; Lee, J. Effects of organizational justice on emotions, job satisfaction, and turnover intention in franchise industry. Kor. J. Fr. Manag. 2018, 9, 7–16. [Google Scholar]
  60. Vinzi, V.E.; Trinchera, L.; Amato, S. PLS Path Modeling: From Foundations to Recent Developments and Open Issues for Model Assessment and Improvemen. In Handbook of Partial Least Squares; Springer: Berlin/Heidelberg, Germany, 2010; pp. 47–82. [Google Scholar]
  61. Chin, W.W. The partial least squares approach to structural equation modeling. Mod. Methods Bus. Res. 1998, 295, 295–336. [Google Scholar]
  62. Falk, R.F.; Miller, N.B. A primer for Soft Modeling; University of Akron Press: Akron, OH, USA, 1992. [Google Scholar]
  63. Hu, L.-T.; Bentler, P.M. Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychol. Methods 1998, 3, 424–453. [Google Scholar] [CrossRef]
  64. Chen, T.; Peng, L.; Jing, B.; Wu, C.; Yang, J.; Cong, G. The impact of the COVID-19 pandemic on user experience with online education platforms in China. Sustainability 2020, 12, 7329. [Google Scholar] [CrossRef]
  65. Coman, C.; Țîru, L.G.; Meseșan-Schmitz, L.; Stanciu, C.; Bularca, M.C. Online teaching and learning in higher education during the coronavirus pandemic: Students’ perspective. Sustainability 2020, 12, 10367. [Google Scholar] [CrossRef]
  66. Khan, M.A.; Nabi, M.K.; Khojah, M.; Tahir, M. Students’ perception towards e-learning during COVID-19 pandemic in India: An empirical study. Sustainability 2021, 13, 57. [Google Scholar] [CrossRef]
  67. Cervi, L.; Pérez Tornero, J.M.; Tejedor, S. The Challenge of Teaching Mobile Journalism through MOOCs: A Case Study. Sustainability 2020, 12, 5307. [Google Scholar] [CrossRef]
  68. Wu, J.; Lin, Z. Research on customer segmentation model by clustering. ACM Int. Conf. Proc. Ser. 2005, 113, 316–318. [Google Scholar]
Sustainability 13 00896 g001 550
Figure 1. Proposed conceptual model.
Figure 1. Proposed conceptual model.
Sustainability 13 00896 g001
Sustainability 13 00896 g002 550
Figure 2. Moderating effect of user innovativeness.
Figure 2. Moderating effect of user innovativeness.
Sustainability 13 00896 g002
Table
Table 1. Profile of survey respondents (n = 216).
Table 1. Profile of survey respondents (n = 216).
VariableFrequencyPercentage (%)
GenderMale12356.9
Female9343.1
Year of StudentsFreshman9041.7
Sophomore4520.8
Junior4420.4
Senior3516.2
MajorsSocial science16074.2
Arts and sciences2813
humanities157
Online Lecture ExperienceYes11955.1
No9744.9
Table
Table 2. Measurement Model.
Table 2. Measurement Model.
Final ItemsFactor LoadingsαRho_AC.R.AVE
Attitude30.932–0.9430.9380.9380.9600.889
Perceived behavioral control40.822–0.8230.9180.9350.9420.804
Perceived ease of use50.796–0.8900.9000.9260.9240.709
Individual behavioral intention20.968–0.9670.9320.9320.9670.936
Subjective norms30.911–0.9300.9230.9280.9510.867
Perceived usefulness30.878–0.9100.8870.8890.9300.816
User innovativeness50.843–0.8860.9090.9230.9320.732
α = Cronbach’s α; C.R. = composite reliability; AVE = average variance extracted.
Table
Table 3. Fornell-Larcker criterion.
Table 3. Fornell-Larcker criterion.
1234567
1. Attitude0.943
2. Perceived behavioral control0.4970.897
3. Perceived ease of use0.7160.8350.842
4. Individual behavioral intention0.8680.450.6730.968
5. Subjective norms0.6140.5380.640.6110.931
6. Perceived usefulness0.8740.5410.7820.8330.6410.904
7. User innovativeness0.7280.5520.7010.7040.530.7570.855
Diagonal elements, which are marked in bold, are the square root of the variance shared between the variables and their measures (AVE).
Table
Table 4. Standardized structural estimates (PLS).
Table 4. Standardized structural estimates (PLS).
PathEstimatetpResult
H1aPerceived ease of use→Perceived usefulness0.78228.9580.000 **Supported
H1bPerceived ease of use→Attitude0.0831.2150.225Not supported
H2Perceived usefulness→Attitude0.80914.0240.000 **Supported
H3Attitude→Behavioral intention0.5797.8970.000 **Supported
H4Subjective norms→Behavioral intention0.0932.0040.045 *Supported
H5Perceived behavioral control→Behavioral intention−0.0440.9750.330Not supported
R2 Q2
Perceived usefulness0.765 0.493
Attitude0.778 0.675
Behavioral intention0.610 0.720
** p < 0.01. * p < 0.05.
Table
Table 5. Moderating Analysis Results (PLS).
Table 5. Moderating Analysis Results (PLS).
PathEstimatetpResult
H6aPerceived ease of use x User innovativeness→Attitude−0.0310.4490.653Not supported
H6bPerceived usefulness x User innovativeness→Attitude0.0610.7410.459Not supported
H6cAttitude x User innovativeness→Behavioral intention0.0851.5290.126Not supported
H6dSubjective norms x User innovativeness→Behavioral intention−0.1192.8400.005 **Supported
H6ePerceived behavioral control x User innovativeness→Behavioral intention0.0330.8080.419Not supported
** p < 0.01.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Kim, E.-J.; Kim, J.J.; Han, S.-H. Understanding Student Acceptance of Online Learning Systems in Higher Education: Application of Social Psychology Theories with Consideration of User Innovativeness. Sustainability 2021, 13, 896. https://0-doi-org.brum.beds.ac.uk/10.3390/su13020896

AMA Style

Kim E-J, Kim JJ, Han S-H. Understanding Student Acceptance of Online Learning Systems in Higher Education: Application of Social Psychology Theories with Consideration of User Innovativeness. Sustainability. 2021; 13(2):896. https://0-doi-org.brum.beds.ac.uk/10.3390/su13020896

Chicago/Turabian Style

Kim, Eun-Jung, Jinkyung Jenny Kim, and Sang-Ho Han. 2021. "Understanding Student Acceptance of Online Learning Systems in Higher Education: Application of Social Psychology Theories with Consideration of User Innovativeness" Sustainability 13, no. 2: 896. https://0-doi-org.brum.beds.ac.uk/10.3390/su13020896

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop