Zoom In, Zoom Out: The Impact of the COVID-19 Pandemic in the Classroom

Abstract

On 11 March 2020, the World Health Organization declared COVID-19 to be a pandemic to be confronted by humanity. As a result, social isolation has become the norm in most countries, with the consequent replacement of face-to-face classes by classes mediated by information and communication technology. Within this context, this work sets out to investigate the factors necessary for courses mediated by technology to attain their pedagogical objectives. Additionally, the study examines how subjects that develop hard and soft skills differ in a technology-mediated setting. The results show that the teacher’s digital competence on the technological platform and the metacognitive support available in the digital environment are significant factors for a course to attain its pedagogical objectives successfully. Lastly, the study revealed that hard skill disciplines, when they migrate to technology-mediated environments, are more likely to fail to achieve their educational goals than soft skill disciplines subject to the same migration.

1. Introduction

The evolution of a pandemic is one of the most dangerous and complex problems for society, and its management and mitigation by the government are, for this very reason, challenging [1]. This can be confirmed by the severe pandemics that humanity has faced throughout its history, such as the Spanish flu in 1917, the Hong Kong flu (H3N2) in 1968, and the swine flu (H1N1) in 2009.

In December 2019, the city of Wuhan, the capital of China’s Hubei province, became the center of a pneumonia outbreak of unknown cause. On 7 January 2020, Chinese scientists isolated a new virus—severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2)—among patients with symptoms of pneumonia. This virus was called coronavirus disease 2019 (COVID-19) in February 2020 by the World Health Organization (WHO), who decreed COVID-19 a pandemic to be confronted by the whole of humankind on 11 March 2020.

In view of this pandemic, social isolation has become the norm in most nations, with the consequent suspension of face-to-face classes and their replacement by classes mediated by information and communication technology, which has altered the routine of thousands of students worldwide.

However, information and communication technology (ICT), although crucial to the viability of education in the COVID-19 pandemic period, does not offer a guarantee of success to those who adopt it, as it must be associated with pedagogical and didactic aspects related to the subject taught [2].

Thus, the objective of this article is to identify the key success factors in the introduction of ICT in courses that were formerly face-to-face and that changed—from one moment to the next and due to the pandemic of COVID-19—to being mediated by ICT by means of the Zoom platform. In addition to this, it seeks to verify the impact of the introduction of ICT on hard skill courses and soft skill courses, in order to verify whether the type of subject taught has a significant influence upon the attainment of the objectives of the courses mediated by technology. Thus, the research question of this work is: What are the key success factors in courses mediated by information and communication technology of hard skill and soft skill contents?

This article is structured as follows: after this introduction, the state-of-the-art of the topic under investigation is described. Then, the methodological procedures adopted in this work are outlined, followed by the results obtained. Next, the results are discussed, and in the last section, the contributions of this research are presented, as well as their limitations.

1.1. State-of-the-Art

In order to answer the research question of this article, it is necessary to address some important theoretical issues, namely the main pedagogical approaches associated with the utilization of ICT in the classroom, the technology-mediated learning (TML) concept, the evaluation models of ICT-mediated courses, and the influence of the characteristics of TML courses contents in the attainment of the learning objectives of said courses.

1.1.1. ICT in the Classroom: Pedagogical Approaches

Two paradigms associated with pedagogical approaches in the classroom became very influential from the 20th century onwards. In addition to including an overview of how the learning process is achieved, these paradigms offer an insight into the nature of knowledge itself—in other words, if knowledge exists in an absolute form or if it is something that is constructed and relative. These two paradigms are traditionally referred to as “instructivism/behaviorism” and “constructivism/cognitivism” [3,4,5].

The core distinction between instructivism/behaviorism and constructivism/cognitivism lies in the concept of knowledge per se. In the former, knowledge is passive—automatic responses to external factors—whereas in the latter, knowledge is perceived as an entity constructed by each student over the learning process. Knowledge seen from the constructivist/cognitivist standpoint does not have absolute characteristics as it does in the instructivist/behaviorist standpoint and cannot therefore be simply handed down from one person to another [3,4,5,6,7,8].

In the context of this work, the central aspects of instructivism/behaviorism are the concepts that the student needs to adapt to the environment, and that learning is a passive process in which no explicit treatment or interest is ascribed to the mental processes. In the case in question, the student simply responds to the demands of the environment by means of stimuli. Knowledge is therefore understood as an absolute and immutable concept [3,9,10]. Therefore, the instructivist/behaviorist paradigm is suited to a given training program, as long as it has clearly defined objectives and the ensuing results can be clearly measured. A good example of this is the training videos that the U.S. army used in World War Two for repetitive tasks such as assembling a rifle [11].

On the other hand, constructivist/cognitivist theorists are of the opinion that learning is an active process. Indeed, constructivism/cognitivism is based on the concept that students develop their own views of knowledge, as opposed to the notion that teachers hand down information and knowledge to their students [12,13,14]. Constructivist/cognitivist proponents are of the opinion that the learning plan should always place emphasis on the student, and not on the content and format of the program or the ideas of the instructor [4,5,15]. In this manner, a transition is made from a model in which the instructor is the focus of the teaching program (instructivism/behaviorism) to a model in which the student is an active participant in the course (constructivism/cognitivism).

1.1.2. Technology-Mediated Learning

Although the digital revolution has the potential to completely digitize services previously provided face-to-face, there is a tendency for hybrid solutions to prevail, such that the real and digital worlds coexist peacefully. This occurred, for example, with e-commerce. From the initial concept that the Internet would eliminate traditional commerce completely, a situation was established in which both physical and online stores co-existed peacefully, by means of a hybrid commerce model [16].

The same phenomenon occurred in education. From totally face-to-face education, it was considered that through ICT—and especially the Internet—a fully online teaching–learning relationship would arise [17,18], in what was traditionally referred to as e-learning [19]. However, in recent years, hybrid teaching–learning models—generically referred to as technology-mediated learning (TML)—have begun to emerge [20,21,22].

TML is defined as an environment in which students’ interactions with learning material, teachers, and/or their classmates are mediated through advanced ICT [23]. In addition, TML courses can be fully supported by the two educational paradigms described above (instructivism/behaviorism and constructivism/cognitivism), allowing the teacher to create an appropriate instructional project for the course to be taught [21]. TML offers the opportunity to integrate the strengths of synchronous (in-person) and asynchronous (IT-based) learning activities [24]. Therefore, the importance of TML tends to increase even more, as it allows the design of innovative and more customized forms of learning, such as micro-learning in the workplace or cloud-based learning regardless of location [25].

On the other hand, one of the most revolutionary aspects of the technological change that has been occurring in education is the emergence of massive open online courses (MOOCs), aimed at unlimited participation and open access via the web [26]. MOOCs were first introduced in 2006, becoming popular in 2012 [27]. However, the characteristics of MOOCs differ from those of traditional and small-scale courses, due to their flexible schedules and large number of enrollments, issues that hinder the personal relationship between teachers and students of MOOCs. In fact, in addition to (i) designing and implementing course contents and activities (for example, recording videos, setting up activities); (ii) promoting the interaction of the participants; (iii) solving unexpected technical problems; and (iv) obtaining feedback for future versions of the course, MOOC teachers are also responsible for (v) assisting participants through private messages and forum posts during the course. Therefore, their heavy workload, together with the massive character of MOOCs, makes it difficult for teachers to provide individual attention and feedback to each student [28].

Based on that and the very need schools had to quickly implement TML courses due to the COVID-19 pandemic, Zoom software was the choice of many government agencies, universities, non-profit organizations, and individuals to teach online classes as an alternative to face-to-face ones [29]. Indeed, despite competing with other software, such as Skype, Google Meet, Webex, MS Teams, and even Whatsapp, Zoom was the most used TML platform to teach online classes during the COVID-19 pandemic [30]. Thus, in this work, one analyzed the impact of the introduction of Zoom in a classroom previously conducted in person.

Zoom is a web-based collaborative video conferencing tool that provides quality audio, video, and screen sharing. It has been used for virtual conferences, online lectures and meetings, webinars, etc. Professors can use the different features of Zoom to create interactive learning environments, which include a virtual white board with annotation capacity to explain concepts, breakout rooms to create small collaborative group work, polls for students feedback, and chat to facilitate class discussions, to name just a few. In addition, zoom meetings can be recorded and made available for future reference [29].

1.1.3. Evaluation of ICT-Mediated Courses

Investigation into the success of ICT implementation in learning environments remains a popular topic [31,32,33,34], although no clear conclusion has been reached on this research question [25,32,35]. Online teachers are conscious of the fact that strategies used in face-to-face classrooms cannot be fully replicated in an online environment [36]. Nonetheless, professors still have doubts about how to promote student engagement in online environments as well as how ICT-mediated courses in higher education impact student commitment to them [37].

Indeed, there is no pivotal theoretical foundation for the evaluation of ICT-mediated courses, as the most common way of developing a measurement model has been conducting a literature review that analyzes a similar problem and the derivation of a modified model [38]. Thus, there is now a broad diversity of approaches and models for assessing the outcome of ICT-mediated courses [38]. However, there is a tendency for research in this area to follow basically one of two different research paths [38]. The first strand is based on models that analyze the factors of acceptance of TML courses by students, which are supported mainly by extant information systems acceptance and adoption frameworks, such as the Technology Acceptance Models (TAM and TAM2), the Unified Theory of Acceptance and Use of Technology (UTAUT), the Electronic TAM Model (e-TAM), the Website Quality Model (WEBQUAL), and the DeLone and McLean Model of Information Systems Success, to name just a few [39,40,41,42,43], as well as variations of them [37,38]. In this regard, the acceptance/adoption of TML courses by students is considered similar to the acceptance/adoption of information systems by users.

The second strand, on the other hand, evaluates a TML course from the standpoint of its pedagogical approach and instructional design [25,44,45,46]—in other words, in relation to its pedagogical option, namely instructivism/behaviorism or constructivism/cognitivism, the Reeves’ Model being a reference in this case [44].

In this work, the second way was opted for, since most of the models developed for the evaluation of TML courses in the first way totally disregard the pedagogical philosophy of the course in question [38,47], which is the focus of this work. Thus, an adaptation of the Reeves’ model is adopted for the identification and analysis of the impact of the different pedagogical dimensions involved in the teaching–learning process [44]. This approach has been applied consistently and successfully in the evaluation of courses mediated by ICT [4,5].

The adaptation of Reeves’ model was made on the basis of the evaluation framework proposed by Siragusa, Dixon, and Dixon [46], seeking to include dimensions associated with student and teacher dexterity in relation to the use of the TML platform adopted, since these dimensions are absent in the original model. In the end, a model with twelve dimensions was decided upon, which relates the pedagogical characteristics of the TML course taught with the students’ perception regarding the attainment of the objective proposed in the syllabus of the course, as detailed below.

Each of the twelve dimensions of the final model is represented by a continuum, which ranges from 0 to 10, having, at each end, opposite philosophical orientations for each dimension—ranging from “fully aligned with the instructivist/behaviorist paradigm”, rank 0, to “fully aligned with the constructivist/cognitivist paradigm”, rank 10. The dimensions are: (1) Pedagogical Philosophy; (2) Learning Theory; (3) Goal Orientation; (4) Task Orientation; (5) Source of Motivation; (6) Teacher’s Role; (7) Teacher’s Digital Competence; (8) Student’s Digital Competence; (9) Metacognitive Support; (10) Collaborative Learning; (11) Cultural Sensitivity; and (12) Structural Flexibility.

Table 1. Parameters for assessment of TML courses.

0←	Dimension	→10
Instructivist Knowledge is handed down by the instructor	Pedagogical Philosophy (PP) 0–10	Constructivist Knowledge is constructed in the mind of the student
Behaviorist Emphasis on discernible behavior	Learning Theory (LT) 0–10	Cognitivist Emphasis on individual mental perception
Specific Direct instruction focusing on expected behavior	Goal Orientation (GO) 0–10	Generic Simulations with various solutions for a given problem
Academic Emphasis on standard academic exercises	Task Orientation (TO) 0–10	Authentic Emphasis on exercises that go beyond the parameters of the student in authentic environments
Extrinsic Motivation extrinsic to the student and the learning environment	Source of Motivation (SM) 0–10	Intrinsic Motivation centered on the student and the learning environment
Didactic The teacher is considered to be the repository of knowledge	Teacher’s Role (TR) 0–10	Facilitative The teacher is a study facilitator and guide for the students
None The teacher does not have the ability to work in a TML environment	Teacher’s Digital Competence (TC) 0–10	Expert The teacher is an expert in working in a TML environment
None The student does not have the ability to work in a TML environment	Student’s Digital Competence (SC) 0–10	Expert The student is an expert in working in a TML environment
Unsupported No student progress tracking mechanisms or adjustments to individual needs are made	Metacognitive Support (MS) 0–10	Integrated Student progress tracking mechanisms, as well as adjustments to individual needs, are implemented.
Unsupported Students work independently	Collaborative Learning (CL) 0–10	Integrated Students work in pairs or in small groups
Unsupported Training is prepared without heeding the culture and diversity of the students it sets out to address	Cultural Sensitivity (CS) 0–10	Integrated The program is adapted to the cultural differences among the students
Fixed Program restricted to specific places and specific times	Structural Flexibility (SF) 0–10	Open Program independent of constraints of time and/or location

Note: adapted from [4,5,47].

below represents the twelve dimensions used to evaluate TML courses. For each dimension, the opposite poles of the continuum are described, and their significance explained.

1.1.4. The Role of Content in TML Courses

Some authors argue that due to the novelty of TML, considerable uncertainty prevails in relation to the subject and type of content most suitable for delivery in the digital environment [48,49].

TML is generally thought to be an effective means for transferring factual and procedural knowledge when the instructivist/behaviorist paradigm is employed [50,51]. On the other hand, technologies that promote participant communication and interaction can be effectively used in TML courses following a constructivist/cognitivist approach to emphasize discussion, brainstorming, problem-solving, collaboration, and reflection [46], aiming to develop higher order thinking skills and build conceptual knowledge.

Although some academics have argued that the type of course content is relevant for the outcomes of TML courses [48,49], the type of course to be taught—namely soft skill course or hard skill course—has not been taken into consideration when evaluating a TML course by the class [35,52], this being a research gap that this study intends to address.

According to some researchers, hard skills are linked to technical competence associated with working with equipment, data, software, numbers, equations, graphs, etc. [53,54,55]. On the other hand, soft skills deal with intrapersonal skills, such as one’s ability to manage oneself, as well as interpersonal skills such as how one handles one’s interactions with others [54,55,56,57].

In line with previous studies [56,57], making the distinction between hard- and soft-skill TML courses can be considered to have a significant impact on training transfer and will add significantly to the understanding of the success of a TML class. Thus, the type of course delivered by TML is expected to be taken into consideration in the evaluation of same, as also pointed out by [48,58,59].

2. Materials and Method

The objective of this article is to identify the antecedents for the successful introduction of ICT in courses that were previously in-person and moved—from one moment to the next and due to the COVID-19 pandemic—to a TML environment, namely Zoom. In addition, the research seeks to verify the impact of the introduction of ICT in courses of hard skills and soft skills, in order to verify if the type of subject has significant influence in the achievement of the objectives of courses mediated by ICT. Based on that, the work intends to answer: What are the key success factors in courses mediated by information and communication technology of hard skill and soft skill contents?

Thus, an undergraduate class in Business Administration was investigated, which, due to the COVID-19 pandemic, approached the subjects of the first academic semester of 2020 via TML supported by the Zoom tool. Two subjects studied by this class were analyzed in this work: Differential Calculus (a hard skill discipline) and Rhetoric and Argumentation (a soft skill discipline), such that the impact of the TML environment in subjects that develop opposite skills could be identified.

2.1. Sample

All 46 freshman year students enrolled in the Business Administration undergraduate course of a university served as the sample for this work. Most of the students in the sample are in the 18 to 21 age group, and 41% of the sample consists of women. The online classes mediated by the Zoom platform were established due to the COVID-19 pandemic, which led to the implementation of social distancing, making it impossible for the classes to be taught in-person. In total, 65% of students stated that they had never used the Zoom platform before.

2.2. Data Collection

The data collection questionnaire was made available to all 46 students in the first-year undergraduate course in Business Administration, in June 2020, at the end of the last day of class of the first semester. Two disciplines were investigated in this research: “Differential Calculus” and “Rhetoric and Argumentation”, so that it was possible to evaluate the potential differences between disciplines that develop hard skills (Differential Calculus) and soft skills (Rhetoric and Argumentation) resulting from the introduction and use of ICT.

By means of the developed questionnaire (see the Appendix A), the students evaluated the twelve dimensions of the model according to their perceptions regarding the pedagogical approach associated with each one. That is, the students filled out the interval scale (0–10) associated with each dimension of the model. If they perceived the dimension as fully aligned with the instructivist/ behaviorist paradigm, they were expected to score 0 on the interval scale. On the other hand, if they perceived the dimension as fully aligned with the constructivist/cognitivist paradigm, they were expected to score 10 on the interval scale associated with this dimension. Finally, if they perceived the dimension aligned with a combination of the two paradigms, they should score an integer value between 0 to 10 (0, 1, 2, 3, …, 9, 10), according to their perception of the weights of each paradigm in the composition of the pedagogical dimension.

After that, the students outlined their perceptions in relation to the degree of attainment of the objectives foreseen for both courses. First, the data related to the Differential Calculus course were collected and then those linked to the Rhetoric and Argumentation course. At the end, the questionnaire presented open-ended questions regarding the perception of the students about the courses and the teachers in general, the TML platform, and the impact of the COVID-19 pandemic on their learnings.

In addition to the questionnaire, online interviews were conducted with the two teachers of the courses (Ti, i = 1,2) and five students chosen at random (Si, i = 1,5), seeking to ensure an enhanced understanding of the results obtained.

2.3. Data Analysis

For analysis of the data collected, it was verified first if one might support the hypothesis that the data obtained came from a normal distribution. For doing that, the Shapiro–Wilk goodness of fit test was applied [60], all p-values obtained being higher than 0.05. Thus, one cannot reject at a 5% level of significance the null hypothesis about the normal distribution of the sample, and by consequence, one may use parametric statistical tests. Additionally, the Central Limit Theorem ensures that samples larger than 30 tend to comply with a normal distribution, regardless of the format of the distribution of probability of the population from which the sample is being taken [61].

Then, in order to analyze the psychometric characteristics of the instrument used to collect data, its reliability and content and construct validities were analyzed [62]. The reliability of the tool was tested via the alpha of Cronbach test [63], while the content validity of the questionnaire was assessed via a pre-test [62] involving five students of other courses as well as two teachers.

The construct validity of the instrument was assessed via multiple linear regressions conducted by means of the Ordinary Least Squares (OLS) method. The dependent variable of the regressions was the Attainment of Objectives (AO) foreseen for the courses under analysis. In turn, the independent variables were the twelve dimensions of the adopted model, which range from 0 to 10, and where each end of the scale is fully associated with an educational paradigm (0 to the instructivist/behaviorist paradigm and 10 to the constructivist/cognitivist paradigm). The dimensions considered were: Pedagogical Philosophy (PP), Learning Theory (LT), Goal Orientation (GO), Task Orientation (TO), Source of Motivation (SM), Teacher’s Role (TR), Teacher’s Digital Competence (TC), Student’s Digital Competence (SC), Metacognitive Support (MS), Collaborative Learning (CL), Cultural Sensitivity (CS), and Structural Flexibility (SF).

In order to ascertain the impact of the courses’ content type on the values ascribed to the accomplishment of objectives of the courses (AO), a dummy variable (DM) was created depending on the type of subject taught (hard skill or soft skill), which assumes a value equal to 1 when the subject is Differential Calculus (hard skill) and zero when it is Rhetoric and Argumentation (soft skill).

Statistical average comparison tests (t-tests) between the respective dimensions were also processed for each of the two subjects analyzed (hard skills and soft skills), to better understand the results of the multiple regressions conducted.

Lastly, content analysis of the interviews with teachers and students was conducted, in order to triangulate the data obtained in search of a better interpretation and discussion of the results [64].

This study thus triangulates a quantitative and qualitative approach to better investigate the research problem, as this allows the limitations of each approach to be overcome, comparing the results from different perspectives [20,64,65,66]. In effect, multiple regressions reveal only numbers, without capturing the perceptions and feelings of students or teachers about the evaluation of TML courses. On the other hand, the interviews collected and analyzed do not allow the measurement of the level of (in)satisfaction of the students with the courses or the understanding of the importance of some dimensions over others in the fulfillment of the objectives of the courses.

3. Results

As stated earlier, this research seeks to investigate the significant dimensions related to the attainment of objectives of two undergraduate courses in Business Administration, which, due to the COVID-19 pandemic, were mediated by technology via the Zoom platform.

The reliability of the data collected via the instrument used was supported, since all Cronbach’s alphas were greater than 0.85 [63]. In addition, the content validity of the questionnaire was also guaranteed by the pre-test carried out with five students and two teachers [62].

After doing that, the Shapiro–Wilk goodness of fit test was applied to ascertain whether the sample adopted had come from a normal distribution [60]. The students evaluated the twelve dimensions of the model presented and outlined their perception in relation to the degree of attainment of the objectives foreseen for both courses. All the p-values found were higher than 0.05. That way, we cannot reject the null hypothesis about the normal distribution of the sample, and by consequence one may use parametric statistical tests [60].

Regarding the variable attainment of objectives, the general average (AVG) of both courses was 6.73, on a scale of 0 to 10, with a standard deviation (SD) of 2.66. This average was significantly higher for the Rhetoric and Argumentation course (AVG = 8.13; SD = 1.59) than for the Differential Calculus one (AVG = 5.33; SD = 2.79). Thus, by applying a t-test, it emerges that the course associated with the development of soft skills—Rhetoric and Argumentation—was evaluated as having a better attainment of objectives than that which develops hard skills—Differential Calculus (p-value = 0.000).

Table 2. Means and correlations between variables.

Variables	Mean	(1)	(2)	(3)	(4)	(5)	(6)	(7)	(8)	(9)	(10)	(11)	(12)	(13)	(14)
(1) Pedagogical Philosophy (PP)	6.185	1.000
	(2.777)
(2) Learning Theory (LT)	5.402	−0.275	1.000
	(2.746)	(0.008) **
(3) Goal Orientation (GO)	5.454	0.536	−0.394	1.000
	(3.255)	(0.000) **	(0.000) **
(4) Task Orientation (TO)	5.815	0.595	−0.307	0.682	1.000
	(3.610)	(0.000) **	(0.003) **	(0.000) **
(5) Source of Motivation (SM)	5.685	0.047	0.074	0.232	0.126	1.000
	(2.669)	(0.660)	(0.480)	(0.026) *	(0.231)
(6) Teacher’s Role (TR)	4.815	0.490	−0.100	0.408	0.630	0.110	1.000
	(3.200)	(0.000) **	(0.342)	(0.000) **	(0.000) **	(0.296)
(7) Teacher’s Digital Competence (TC)	6.989	−0.182	0.117	−0.249	−0.316	−0.108	−0.174	1.000
	(2.452)	(0.082)	(0.269)	(0.017) *	(0.002) **	(0.305)	(0.097)
(8) Student’s Digital Competence (SC)	7.348	−0.140	0.207	−0.036	−0.069	−0.098	0.195	0.364	1.000
	(2.308)	(0.182)	(0.048) *	(0.734)	(0.515)	(0.353)	(0.063)	(0.000) **
(9) Metacognitive Support (MS)	5.369	0.270	−0.145	0.227	0.253	0.108	0.298	0.011	0.130	1.000
	(2.167)	(0.009) **	(0.167)	(0.030) *	(0.015) *	(0.306)	(0.004) **	(0.916)	(0.217)
(10) Collaborative Learning (CL)	6.967	0.411	−0.004	0.230	0.446	0.151	0.462	0.032	0.151	0.281	1.000
	(2.925)	(0.000) **	(0.971)	(0.027) *	(0.000) **	(0.152)	(0.000) **	(0.761)	(0.150)	(0.007) **
(11) Cultural Sensitivity (CS)	4.826	0.406	−0.018	0.376	0.426	0.071	0.444	−0.104	0.149	0.296	0.417	1.000
	(2.839)	(0.000) **	(0.867)	(0.000) **	(0.000) **	(0.501)	(0.000) **	(0.322)	(0.158)	(0.004) **	(0.000) **
(12) Structural Flexibility (SF)	4.783	0.156	−0.089	0.225	0.181	0.082	0.298	−0.153	−0.033	0.092	0.280	0.394	1.000
	(2.736)	(0.138)	(0.398)	(0.031) *	(0.085)	(0.435)	(0.004) **	(0.146)	(0.754)	(0.386)	(0.007) **	(0.000) **
(13) Attainment of the Objectives (AO)	6.728	0.379	−0.126	0.364	0.319	0.054	0.383	0.180	0.320	0.414	0.431	0.488	0.202	1.000
	(2.661)	(0.000) **	(0.230)	(0.000) **	(0.002) **	(0.607)	(0.000) **	(0.086)	(0.002) **	(0.000) **	(0.000) **	(0.000) **	(0.054)
(14) Dummy Hard Skill (DM)	0.5	−0.657	0.203	−0.628	−0.760	−0.086	−0.625	0.236	−0.028	−0.212	−0.527	−0.562	−0.216	−0.530	1.000
	(0.503)	(0.000) **	(0.052)	(0.000) **	(0.000) **	(0.415)	(0.000) **	(0.023) *	(0.788)	(0.043) *	(0.000) **	(0.000) **	(0.039) *	(0.000) **

Notes: Standard deviations in brackets for the first column and p-values in brackets for the others. ** p < 0.01, * p < 0.05.

presents the descriptive statistics and correlations between the variables of the evaluation model adopted.

One can observe in Table 2 that the hypothesis Ho (the correlation between the variables is null) cannot be supported for some correlations at a significance level of 5% or 1%. Therefore, a multicollinearity test should be performed for the independent variables of the model to calculate the variance inflation factor (VIF).

Table 3. Variance inflation factor (VIF).

Variable	VIF	1/VIF
Dummy Hard Skill (DM)	3.780	0.265
Task Orientation (TO)	3.580	0.280
Goal Orientation (GO)	2.640	0.378
Teacher’s Role (TR)	2.270	0.440
Pedagogical Philosophy (PP)	2.150	0.465
Cultural Sensitivity (CS)	1.830	0.547
Collaborative Learning (CL)	1.810	0.552
Student’s Digital Competence (SC)	1.520	0.657
Learning Theory (LT)	1.370	0.729
Structural Flexibility (SF)	1.370	0.730
Teacher’s Digital Competence (TC)	1.370	0.732
Metacognitive Support (MS)	1.250	0.799
Source of Motivation (SM)	1.190	0.839
Mean VIF	2.010	-

shows the VIF for each dimension. A maximum VIF above 10 indicates that multicollinearity may be influencing minimum square estimates. As the maximum VIF of the regression is 3.78 in relation to the dummy variable and the average VIF is 2.01, one can support that the regression does not present multicollinearity problems.

That done,

Table 4. Regression results.

Variables	Attainment of the Objectives (AO)
	Model 1	Model 2
Pedagogical Philosophy (PP)	0.129 (0.110)	0.0421 (0.108)
Learning Theory (LT)	−0.0620 (0.0920)	−0.0721 (0.0872)
Goal Orientation (GO)	0.157 (0.105)	0.0801 (0.102)
Task Orientation (TO)	−0.0678 (0.108)	−0.171 (0.107)
Source of Motivation (SM)	−0.00892 (0.0880)	0.0152 (0.0836)
Teacher’s Role (TR)	0.0361 (0.1000)	−0.0198 (0.0963)
Teacher’s Digital Competence (TC)	0.204 ** (0.103)	0.226 ** (0.0974)
Student’s Digital Competence (SC)	0.219 * (0.115)	0.211 * (0.109)
Metacognitive Support (MS)	0.234 ** (0.110)	0.291 *** (0.106)
Collaborative Learning (CL)	0.145 (0.0960)	0.0691 (0.0940)
Cultural Sensitivity (CS)	0.229 ** (0.0983)	0.138 (0.0973)
Structural Flexibility (SF)	0.0123 (0.0916)	0.0468 (0.0874)
Dummy Hard Skill (DM)		−2.514 *** (0.790)
Constant	−0.786 (1.402)	2.618 (1.705)
Observations	92	92
R-squared	0.480	0.540

Notes: Standard errors in brackets; *** p < 0.01, ** p < 0.05, * p < 0.1.

presents the results of the two multiple regressions processed (Models 1 and 2). The only difference between Model 1 and Model 2 is that in Model 2, the aforementioned dummy variable associated with the type of subject (hard skill or soft skill) was added. The R² of Model 2 is equal to 0.54 (as opposed to 0.48 of Model 1), which means that approximately 54% of the variance of the Attainment of the Objectives variable can be explained by the dimensions included in the regression.

Observing the results of Model 2 (Table 4), one realizes that the dimensions Teacher’s Digital Competence, Metacognitive Support, and the dummy variable associated with the type of subject are significant antecedents to the attainment of the objectives of the courses under analysis. Teacher’s Digital Competence has a positive (β = 0.23) and significant effect (p < 0.05) in attaining the objectives of the subject; Metacognitive support also has a positive (β = 0.29) and significant effect (p < 0.01). In addition, the fact that the subject is associated with the development of hard skills has a significant negative effect (β = −2.51) and a statistically significant effect (p < 0.01).

Consolidating the results obtained, the following relationship was found:

$$AO= \beta 0+ \beta 1\ast TC+ \beta 2\ast MS+ \beta 3\ast DM$$

where AO = Attainment of Objectives; TC = Teacher’s Digital Competence; MS = Metacognitive Support; and DM = dummy variable. The variables AO, TC, and MS vary from 0 to 10, while DM has a value of 1 when the subject is Differential Calculus (associated with hard skills) and zero when the subject is Rhetoric and Argumentation (associated with soft skills).

Replacing the coefficients with the calculated values, the following equation was obtained:

$$AO= 2.618+ 0.226\ast TC+ 0.291\ast MS-2.514\ast DM$$

Next, the results obtained are analyzed and the findings presented.

Findings

Teacher’s digital competence (p < 0.05), metacognitive support (p < 0.01), and the dummy variable (p < 0.01) were considered significant dimensions for the courses to reach their objectives. Thus, irrespective of the type of subject, the digital competence of the teacher on the TML platform and the metacognitive support are significant antecedents for the TML courses to attain the desired objectives.

This finding can be supported by the following statements from the students (Si) and teachers interviewed (Ti):

-

Teacher’s Digital Competence

S1: It is essential that teachers have full mastery of the tool. I have the impression that hard skill teachers have an easier time with the tool. The greatest difficulty for teachers was to divide the class into groups, which is essential to take full advantage of the TML tool.

Indeed, a t-test performed showed that the hard skill teacher really had greater competence on the platform than the soft skill teacher at a p-value of 0.0234 (AVG Soft Skill = 6.413; S.D. = 2.400; and AVG Hard Skill = 7.565; S.D. = 2.391).

-

Metacognitive Support

S1: In general, I would say that all things considered and in the final analysis it was positive. We had access to all notes and recordings of the classes, and from that point of view it even added value to the course.

S2: To my mind, having the recording of the classes on file was especially useful for me to be able to track my progress in the subject. In addition, the use of an auxiliary system to TML, to provide other tools to enhance the course, helped us greatly to understand where we were in the subject, enabling us to adapt the course to our individual needs.

T1 (Hard Skill Teacher): I used One Note a great deal, where I saved the notes and posted them on the e-class system. In addition, I recorded the lessons on Zoom and made them available to students. In this way, they could review the classes and track their progress on the course.

T2 (Soft Skill Teacher): I sought to adapt my course to the online reality, so that students could monitor their progress in the area of rhetoric and argumentation.

Thus, teachers’ digital competence on the platform and metacognitive support (the student’s ability to track his/her progress and potential progress on the course) are fundamental to the success of TML courses.

In addition, there is statistical significance in the dummy variable associated with the type of subject taught. This is made visible by the difference in assessment of 2.514 points (coefficient β3 in the final equation found), on a scale of 0 to 10, when the subject is a hard skill, compared to the soft skill subject.

The phrases below support this result:

S2: There were losses in the learning process, especially in Differential Calculus, which requires a great deal of concentration. It’s much worse!

S4: Learning was jeopardized in general, but Differential Calculus was the one that suffered the most—you need to be fully concentrated. By the end of class, you are exhausted. During the last 20 min of class, I cannot even absorb any further information.

S5: Differential Calculus is far more tiring than Rhetoric and Argumentation in online teaching. After class I have zero desire and willingness to exercise. I get far more tired than in the classroom because of the effort to remain focused, and I end up not studying as much as before.

To understand the reasons for the difference between the hard skill and soft skill courses, a t-test of comparison of averages between all dimensions of the model was conducted, with the dimensions in

Table 5. t-test of Comparison of Averages between Subjects.

Dimensions	Average
	Soft Skill	Hard Skill	p-Value
Structural Flexibility (SF)	5.370	4.196	0.039 (**)
	(0.396)	(0.396)
Cultural Sensitivity (CS)	6.413	3.239	0.000 (***)
	(0.340)	(0.356)
Attainment of the Objectives (AO)	8.130	5.326	0.000 (***)
	(0.234)	(0.411)

Standard errors in brackets (**) p < 0.05; (***) p < 0.01.

found to be significant.

Thus, Structural Flexibility (SF) and Cultural Sensitivity (CS) were especially important for the soft skill subject to obtain a better evaluation than the hard skill subject, given the possibility of the subject content and pedagogical practice to be flexible and customized to the reality of each student, unlike the hard skill subject—a fact attested to by the statements below by students and teachers.

T1 (Hard Skill Teacher): It is very difficult to adapt a subject like Calculus to a participatory online environment. The course is extremely high in content. If the intention is to continue with online classes, it would be necessary to think about a hybrid model in the future, with part of the course being face-to-face. And for the computing part, it is necessary to have different tools. Merely sharing a screen does not work.

T2 (Soft Skill Teacher): I was surprised at how well suited all my subjects are for online teaching. I teach Geopolitics, Law, Rhetoric and Argumentation, and I provide soft skill training for executives. I was surprised at how soft skill activities are appropriate in online communication and what can be achieved in the digital environment. The students showed me that there is a demand for this type of content taught online.

S4: In Calculus the subject lacked flexibility. There were people in the class with coronavirus; people lost family members; we are not at home because we want to be. The teacher has to understand that our stress level is greater, as we have no environment in which to study and in this respect, we expected him to adapt the subject and conduct the assessments differently.

S1: The Rhetoric teacher is sensational. He persuaded everyone and enthralled the whole group. Everyone was delighted. He managed to adapt and understand the group and was highly supportive. On the other hand, the Calculus teacher took an uncompromising stance. For example, in the exam, a student from a favela explained that the Internet was crashing all the time and that she could not be in Zoom with the camera connected. His answer was “find a way around it.”

S3: The Differential Calculus teacher was highly intransigent and did not take into consideration the nuances of the moment we were going through, unlike the Rhetoric teacher, who managed to put himself clearly in our place.

Thus, the lack of structural flexibility and cultural sensitivity associated with the type of content were decisive factors for the evaluation of the hard skill subject to be far lower than the evaluation of the soft skill subject, as seen in Table 5.

As the digital competence of the teacher on the TML platform and the existence of adequate metacognitive support are significant antecedents for the TML courses to attain their objectives, it is mandatory for the teachers to master the features available on the Zoom platform, especially the creation of workgroups for the dynamics of the subject. Moreover, the recording of the classes by Zoom for later consultation and viewing at any time and in any place, the appropriate use of other systems to provide didactic material to promote the course, as well as the availability of activities, tasks, and templates for the proposed exercises allow the students to track their own development and growth throughout the TML course, thereby increasing the metacognitive support.

In addition, it is noted that soft skills disciplines are more likely to achieve their goals in TML environments than those of hard skills from the students’ perspective. This fact is mainly because the potential soft skills disciplines in TML environments have to accommodate greater structural flexibility and cultural sensitivity, when compared to the hard skills ones.

4. Discussion

This research provided insights into the impact of the COVID-19 pandemic in the classroom, investigating the introduction of ICT tools in the teaching-learning process.

According to some authors [67,68,69], the instructor’s attitude towards interactive learning is a key factor for the success of TML initiatives. However, for these authors, this feature covers not only the digital competence of teachers and their mastery of TML tools, but also their ability to interact with students inside or outside the class and their ability to stimulate student–student interaction. As can be seen, these characteristics are strongly related to the dimension of metacognitive support, also found in this study as a critical antecedent for the success of a course based on TML.

Additionally, one realizes that soft skill courses are more likely to be successful in TML environments than hard skill ones. Regarding the influence of the content of the discipline in the success of a TML-based course, one can state that this is an under-researched topic. Most of the time, articles analyze the key success factors of TML initiatives without taking into consideration the content of the discipline being delivered. As an exception, Purarjomandlangrudi and Chen [70] investigated the role of the content for the success of a TML-based discipline. However, the focus of them was on how students perceived the importance of the discipline rather than if the discipline developed hard or soft skills in the students.

Despite the findings related to the influence of the discipline content type on the success of TML initiatives, Moore and Pearson [71] argue that the content development of hard skills courses for online environments is relatively simple and straightforward. On the other hand, the authors argue that improving students’ soft skills in online classrooms is considerably more complex and demanding.

Moreover, Fan et al. [72] support that, since hard skills are mainly related to technical knowledge, they can be more easily developed and measured. Soft skills, however, have a slower development, as they require first a change of attitude and then the mastery of methods for their development [73].

That way, although the results obtained have indicated that soft skills courses are better evaluated than hard skill ones by the students in a TML environment, some authors [71,73] argue that the migration of hard skill courses from the traditional classroom to a TML setting is easier than for soft skill courses, as the former seem to be independent of students’ idiosyncrasies, values, and beliefs. In sum, it seems to be easier to move a hard skill course to a TML environment than a soft skill one, although soft skills courses seem to be more enjoyable in a TML setting for the students than hard skill ones.

4.1. Research Contributions

In view of the results and findings presented above, the contributions for practice listed below seem to be important for attaining the objectives proposed in TML courses:

• Train teachers adequately in all features of the TML platform adopted. In the case of Zoom, the proper use of workgroup rooms, the sharing of files in real time, the staging of quizzes, among other aspects, were features cited by the students as decidedly important for TML courses to attain their goals.
• Make auxiliary systems available that enable the supply of support material for the course, broadsheets to students, exercises, recordings of previous classes, etc., such that the students can have an understanding of the degree of cognitive transformation that the course is providing them—in other words, offer adequate metacognitive support to the student by means of the TML platform and the instructional design of the course.
• Make the structure of the TML course highly flexible, avoiding merely transforming it from an instructional design based on the face-to-face approach to an instructional design supported by an ICT-mediated approach—which involves the development of actions that lead to greater student participation—and use auxiliary systems to support the teaching–learning process. In addition, it is necessary to observe the context and climate of the class, seeking to obtain adequate cultural sensitivity to the reality faced by the students, in order to be able to customize the course adequately to that context.
• Make teachers aware that although hard skills courses are easier to transfer to a TML environment than soft skills courses, the latter are more appreciated by students in a TML environment than the former.

According to Alavi [23], there is a need for greater depth of research into the question of how technology enhances learning. As such, this work intended to diminish this gap by making some academic contributions to the computer-based higher education scientific literature as explained below.

First, this work, by exploring the relationship between pedagogical approaches, instructional strategy, and contextual factors involved in learning, revealed crucial factors for the success of TML courses.

Second, the study examined the differences between hard skills and soft skills disciplines in online instructional environments. Thus, an original contribution of this study is the analysis of the impact of the course’s content type on the success of TML initiatives.

4.2. Research Limitations and Further Steps

This research, like all research, has its limitations, as set forth below.

First, the teacher’s relationship with students was not considered in the evaluation model. The good relationship between the teacher and the class may have influenced the results of the TML courses, as suggested by Marks et al. [74]. In the case under analysis, there was a greater empathy between the teacher of the soft skills discipline and the class than between the teacher of the hard skills discipline and the class. Thus, in future research, the impact of teachers’ social and interpersonal skills on the results obtained must be taken into account in order to investigate whether it is not so much the subject of the course as the teacher’s personal skills that influences how well the course is received online.

Another limitation is related to the acuity of perception of the students who took part in this research. Scandura and Williams [75] and Bertucci [76] contend that this issue is related to a restriction of the information available when the respondents filled out the survey, to the epistemological profile of the students, and to the plethora of possible interpretations of the questionnaire they filled out in their attempt—not necessarily conscious—to present a good/bad impression of the TML courses under analysis.

In addition to this, data were collected at a very complicated time in Brazil—the peak of COVID-19 pandemic. Therefore, the number of respondents in the sample used in this study is limited and, thus, some care is needed when generalizing the results obtained. In other words, the external validity of the study cannot be fully guaranteed [77]. Thus, further research is proposed involving not only larger samples of students, but also different contexts, with regard to the disciplines associated with soft skills and hard skills, in order to verify whether the results are replicable, aiming to strengthen the external validity of this research.

Moreover, a specific TML platform was investigated, namely Zoom. Thus, it is necessary that other TML platforms available on the market (MS Teams, Google Meet, etc.), applied to both hard and soft skill subjects, be investigated as well [78]. In this way, it could be assured, with greater security, that the results presented here remain valid in other computational environments.

By way of conclusion, it is hoped that this work—the interest in which arose due to the sudden growth of technology-mediated courses due to the COVID-19 pandemic—can contribute to a better understanding of the key success factors in TML courses, as well as their suitability to subjects that predominantly develop either hard or soft skills.