text new page (beta)
English (pdf)
Article in xml format
Automatic translation
Cited by SciELO 
Cited by Google 
Similars in
SciELO 
Similars in Google 
Permalink
1 Introduction
Advancement of information technology has made crowd intelligence possible as a new source of innovation and promoted the popularity of crowdsourcing contest platforms. When facing a problem or task, an organization or an individual, generally called a seeker, can post an open call on these platforms to solicit solutions from a diverse range of individuals (solvers) [1]. A reward is usually attached to attract solvers, but only the solver who generates the best solution wins the reward [35]. Crowdsourcing contest platforms are now widely used for a variety of tasks, including product innovation (e.g., Site 1), logo design (e.g., Site 2), software development (e.g., Site 3), and minor jobs (e.g., Site 4). Having the potential of accessing extensive knowledge and skills, these platforms are now enjoying fast development. For example, over 4 million tasks are posted on Epwk.com, a famous crowdsourcing contest platform in China.
Although crowdsourcing contest platforms have become popular in a variety of industries, they suffer from some challenges, such as how to maintain the attractiveness to solvers. Attracting solvers’ initial participation is an important first step toward realizing the success of a platform, however, its sustainability and eventual success heavily depend on solvers’ continuance participation. In fact, unfortunately many solvers discontinue or become inactive in crowdsourcing activities over time [49]. For example, [26] revealed that the number of active solvers declined over time on IdeaStorm.com, a crowdsourcing community operated by Dell. Our analysis of Epwk.com shows that among all registered solvers approximately only 11.7% participate in crowdsourcing tasks more than once. Crowdsourcing scholars recently are realizing the significance of solvers’ continuance intention and are paying increasing attention to this topic [8], [48], [49]. But related studies are still limited and scarce, calling for more effort in this stream. Therefore, investigating factors that influence solvers’ continuance intention constitutes the objective of our study.
In the area of continuance intention research, expectation-confirmation model (ECM) has long been proved to be a useful framework [7] and has been widely modified to study technology adoption and continuance use [11], [27], [34]. Under the setting of crowdsourcing contest, solvers are generally initially motivated by utilitarian value (e.g., task reward) and hedonic value (e.g., enjoyment, self-challenge, skill improvement, etc.) to contribute their ideas or solutions for the task [31], [48]. However, solvers’ initial expectation can be changed or reshaped with first-hand experience to form post-perception which will substantially determine their future intention. The similar rationale between solvers’ continuance intention and users’ technology continuance intention suggests the great potential ability of ECM to predict solvers’ continuance intention. However, the characteristics of crowdsourcing contest are different from that of a technology product, which present some challenges to the potential predictive power of ECM framework.
First, ECM conceptualizes perceived usefulness to represent the ex post expectation about the levels of attributes possessed by a technology product [7]. In the crowdsourcing contest context, solvers are generally viewed as participators (not consumers) of an internet-mediated contest, who expect to receive the utilitarian value or hedonic value from their participation behavior [49]. Thus, solvers’ overall assessment of their behavior is much more likely based on the perception of what is received, which can be reflected by the term perceived benefits [34]. Second, ex post expectation in the original ECM framework only focuses on the product material attributes or quality. However, individuals’ perceptions are not necessarily restricted to the material or performance aspect [52]. Under the unrestricted, anonymous environment of crowdsourcing contest, solvers additionally have a great demand on psychological factors, such as trust belief [2]. In particular, solvers are indeed facing some uncertainties or risks, such as fraud information and labor squeeze. As some studies indicate, lacking trust belief has become a severe barrier blocking solvers’ participation behavior [37], [50]. Hence, extending the ECM framework by incorporating the trust belief toward the platform as an additional post-expectation may be imperative and valuable to explain solvers’ continuance intention. Third, crowdsourcing contest is essentially an online task-based tournament, thus the extent to which solvers fully understand the task needs and generate solutions that the seeker find valuable enough and the extent to which the tournament is fair may have significant role in helping confirm solvers’ expectation. A high level of interaction can facilitate the communication between solvers and seekers to help solvers develop a better understanding of task needs. Also, high levels of interaction between solvers and seekers are helpful for breaking cognitive fixation (which occurs when a person is unable to solve a problem in a creative way but continues to follow the traditional way) to generate creative solutions [6], [42]. On the other hand, perceived fairness is a crucial factor when evaluating the extent to which the crowdsourcing contest is fair. Solvers have to invest effort into their solutions which are used to compete for the reward, and consequently solvers have an inner tendency to compare their inputs and outputs [22]. When solvers perceive they are unfairly treated, they tend to generate doubts about procedures and outcomes, leading to a reduced confirmation level. Therefore, interaction and fairness perception may have crucial effects in confirming solvers’ expectation level. Some studies also argued that when studying continuance intention under the lens of ECM, it would be more valuable to understand the antecedent of confirmation for its central role in an ECM framework [11]. Thus, this study further tries to broaden the ECM framework by explicitly examining the effect of interaction and perceived fairness on confirmation level. In sum, considering the effectiveness of ECM and these challenges, this study extends ECM to study solvers’ continuance intention by incorporating some unique features.
Using a sample of 306 solvers obtained from a crowdsourcing contest platform in China, the extended ECM model is tested and confirmed. Theoretically, this study contributes to the crowdsourcing research by examining an extended ECM to explain the process in which solvers’ participation intention can be sustained and nurtured. In addition, our results enrich the research stream of ECM by uncovering two significant antecedents of confirmation and by incorporating platform trust as an additional ex post expectation, which, in turn, has positive effect on satisfaction and continuance intention. These results can also provide insights to crowdsourcing practitioners regarding the factors that can retain active and regular solvers and turn infrequent solvers into committed ones.
The rest of the paper is structured as follows. In section 2, we propose the research model and hypotheses based on a discussion of the theoretical background. We then describe the research methodology and present data analysis and results in Section 3 and 4 respectively. Next, we discuss the research, implications in theory and practice, and research limitations and future research in Section 5. Finally, Section 6 closes this paper with a conclusion of the research.
2 Theoretical Background and Hypotheses Development
In this section, the Expectation-confirmation model (ECM) is introduced to serve as our theoretical basis. Based on ECM and the features of crowdsourcing contests, we incorporate platform trust, interaction, and perceived fairness to develop a research model for solvers’ continuance intention.
2.1 Expectation-Confirmation Model
Expectation-confirmation model (ECM) has long been adopted to explain continuance intention in a variety of contexts. It stems from expectation-confirmation theory (ECT) [41] and is developed by Bhattacherjee to understand users’ continued IS usage intention [7].
ECT posits that consumers’ satisfaction and repurchase intention are determined by two major constructs: initial expectation on a product or service and the confirmation level. According to the framework of ECT, consumers have an initial expectation about the product or service prior to purchase. They will build perceptions about the performance based on their consumption experience. By assessing perceived performance against initial expectation, consumes form their confirmation level. Upon the confirmation level and expectation on which that confirmation is based, consumers form satisfaction, which in turn determines repurchase intention.
Drawing on the ECT framework and the substantial differences between initial adoption and continued intention, [7] developed and empirically tested an Expectation-Confirmation Model (ECM) for continued IS usage intention. ECM focuses only on post-acceptance variables instead of examining both pre-consumption and post-consumption variables. This is because [7] argued that the effects of any pre-acceptance variables were already captured within the confirmation and satisfaction constructs. In contrast with pre-acceptance expectation, post-acceptance expectation is typically based on users’ first-hand experience and is, therefore, more realistic [19]. Once updated during the usage process, post-acceptance expectation will replace initial expectation in users’ cognitive memory as the basis for guiding subsequent decision processes. Hence, ECM amends ECT to include ex post expectation. The rationale of this framework is as follows. When using a product, users assess perceived performance with their initial expectation and determine the extent to which their expectation is confirmed (confirmation). Meanwhile, following their usage experience and confirmation level, users form a post-acceptance expectation (perceived usefulness) which may be different from their pre-acceptance or initial expectation. Their confirmation level and post-acceptance expectation lead to form satisfaction. Finally, satisfied users would like to develop a continuance intention. Figure 1 illustrates the process by which users reach continuance intention in an ECM framework.
2.2 Hypotheses Development
In this section, we develop the arguments for the research model (Figure 2) and hypotheses.
2.2.1 Effect of Satisfaction
Satisfaction is referred to as an evaluative affect based on past experience [3], [7]. In crowdsourcing contest setting, a solver’s satisfaction with a crowdsourcing platform signals that he or she has a pleasant experience during the participation. Hence, more satisfied solvers are more willing to continue their participation. In contrast, solvers who are dissatisfied with their participation experience would lack impetus to keep their behavior. For example, [26] indicated that negative experience and dissatisfaction can eventually stop solvers’ contribution. The study of [7] has provided empirical evidence for the association between satisfaction and continuance intention in ECM framework. Indeed, a growing body of literature has widely reached the consensus that satisfaction is a determinant of individuals’ repeated or continuance intention, such as re-purchase products, patronize services, and sustained behavior in various communities [17], [18], [33], [34]. Thus, based on the discussion and prior studies, we propose the hypothesis.
H1: Satisfaction is positively related to continuance intention in crowdsourcing contest.
2.2.2 Effects of Perceived Benefits
As prior arguments state, perceived benefits herein are used to represent solvers’ ex post performance-expectation, including monetary reward, self-improvement, and skill development, etc. [31], [45]. Drawing on ECM framework, ex post expectation is a primary determinant of a high level of satisfaction. This suggests that high perceived benefits may be positively associated with satisfaction. The reason is that with a high level of perceived benefits, solvers have a great perception that they can gain benefits from participating in crowdsourcing contest. When solvers find their participation behavior more valuable, they tend to be more satisfied. On the contrary, negative affections, such as dissatisfaction would be experienced due to the low values associated with the participation behavior. Some studies based on different levels of analysis have empirically examined the impact of perceived benefits on satisfaction and have provided evidence for their relationship [29], [34]. Hence, it is reasonable to believe there is a positive link between perceived benefits and satisfaction in the context of crowdsourcing contest. Based on the discussion and prior studies, we propose the hypothesis.
H2: Perceived benefits are positively related to satisfaction in crowdsourcing contest.
According to self-determination theory, individuals tend to rationally evaluate the outcomes of their behaviors and then adjust their strategies and behaviors to receive more values [14]. Extant literature has indicated that individuals’ behavioral intention can be determined by perceived benefits [34]. In the setting of crowdsourcing contest, externalizing the incentive mechanisms in solvers’ priority has been suggested as an effective way to attract solvers’ participation attention [31], [35]. Hence, when solvers perceive it is a great place to obtain money reward, they tend to pay more effort to continually participate in crowdsourcing tasks [49]. In addition, the value associated with inner needs also shows attraction to solvers [31], [45]. Crowdsourcing contest provides a stage to achieve a sense of achievement through the competition with others. As [49] stated, solvers’ continuance behavior could be evoked when they realized that their participation behavior would satisfy their needs for competence. In sum, we suggest solvers are more likely to continue their participation intention when they perceive the associated benefits to be high. Based on the discussion and prior studies, we propose the hypothesis.
H3: Perceived benefits are positively related to continuance intention in crowdsourcing contest.
2.2.3 Effects of Platform Trust
In contrast to a technology product, the crowdsourcing contest platform has a social attribute enabling a solver interacts with the seeker and other solvers. However, under the virtual environment of crowdsourcing contest, the interaction is vulnerable to some potential uncertainties or risks, such as fraud information and labor squeeze. Thus, it is natural that besides the material or performance expectation, solvers have a demand on psychological expectation. As the intermediary between seekers and solvers, the extent to which the platform is trustworthy will play a significant effect on solvers’ continuance intention. A high level of platform trust implies the platform will behave in accordance with individuals’ expectation to facilitate their socialization. During participation process, solvers can build and adjust their perceptions of trustworthy based on their participation experience. Thus, this study integrates platform trust as a psychological expectation in crowdsourcing contest to explain its consequences on satisfaction and continuance intention. Herein platform trust is defined as a belief held by solvers that the crowdsourcing contest platform is reliable and trustworthy in terms of labor protection, transaction security, and information accuracy.
In the virtual environment of crowdsourcing contest, trust serves as a salient factor to elicit solvers’ positive affect (e.g., satisfaction) [2], [37]. The labor squeeze is recently a severe concern in crowdsourcing contest as some seekers may make solvers work on the task with submitted solutions and then reject all solutions to avoid paying. Belief in labor protection can help solvers build confidence that there is a low risk of labor squeeze, thereby enabling solvers to experience a high level of satisfaction. In addition, with the prevalence of Internet, online fraud has become a concern on many online communities or platforms [55]. Solvers with a high level of platform trust are likely to alleviate their concerns regarding the potential fraud, thus leading to a satisfied participation experience. In contrast, lacking trust would make solvers doubt if their effort is worthy. For example, [16] noted that if trust was not properly fulfilled, the trusting party would experience unpleasant consequences. When experiencing doubts and unpleasant feelings, solvers have a great possibility to generate a low satisfaction. The positive link between trust and satisfaction has also been verified by extant literature. For instance, both [5] and [53] suggested that trust was an essential ingredient in creating satisfaction. Hence, based on the discussion and prior studies, we propose the hypothesis.
H4: Platform trust is positively related to satisfaction in crowdsourcing contest.
In the crowdsourcing contest setting, the potential uncertainties or risks and are calling for a high level of trust to sustain solvers [2]. Platform trust can reduce solvers’ perceptions of risk by allowing them to subjectively rule out the potentially undesirable behaviors [37], [38]. The low payment and labor squeeze have recently become barriers for sustaining participation intention in crowdsourcing tasks [50]. Establishing measures about labor protection, for example, proofreading the disputable tasks to avoid labor squeeze, to improve solvers’ trust and confidence in the platform may have positive effect on continuance intention. In addition, trust concerning the belief in transaction security can reduce solvers’ anxiety about the money transactions. The enhanced confidence in the exchange relationship can also encourage sustained intention in the given platform. Moreover, information fraud is another severe problem for sustained intention and behavior [55]. Trust in information accuracy helps solvers generate high levels of pride and confidence in their membership, which will enhance their sustained participation intention. Therefore, based on the discussion, we propose the hypothesis.
H5: Platform trust is positively related to continuance intention in crowdsourcing contest.
2.2.4 Effects of Confirmation
Confirmation implies the extent to which individuals realize their expected benefits during their past experience. According to ECM, confirmation is a significant antecedent of satisfaction [7]. Based on ECM, we propose that the solver’s confirmation level will positively affect satisfaction in crowdsourcing contest. Solvers tend to be more satisfied when their confirmation is high. Conversely, if solvers’ perceived benefits cannot meet their expectation, they are more likely to be disappointed with their participation behavior, leading to a low satisfaction. The association between confirmation and satisfaction has been tested and verified by a variety of empirical studies [7], [17], [33], [34]. Thus, based on the discussion and prior studies, we propose the hypothesis.
H6: Confirmation is positively related to satisfaction in crowdsourcing contest.
ECM also suggests that confirmation positively impacts individuals’ ex post expectation (e.g., perceived usefulness) [7]. Herein perceived benefits is used to represent solvers’ ex post performance-expectation about their behaviors in crowdsourcing contest. Hence, in the research context of crowdsourcing contest, confirmation is expected to positively relate to perceived benefits. Solvers tend to rate the crowdsourcing task as being more valuable when their expectation is confirmed during the participation process. Solvers’ performance-expectation in crowdsourcing contest includes getting the reward, challenging themselves, and enjoying fun, etc. When their expectation about getting the reward is confirmed, solvers tend to regard crowdsourcing contest as a good way to reach the expectation and generate a high perception of value. In addition, crowdsourcing tasks are always highly competitive and have demand on solvers’ competency. A confirmed solver is also likely to find crowdsourcing participation useful in challenging self and satisfying the need for competency. By contrast, individuals may experience cognitive dissonance or psychological tension if their expectation is disconfirmed [7], [20]. Solvers will then try to remedy this dissonance by reducing their value perceptions in order to be more consistent with their experience. Similarly, [34] found empirical evidence there was a positive association between confirmation and perceived benefits of Internet protocol television. Hence, based on the discussion and prior studies, we propose the following hypothesis.
H7: Confirmation is positively related to perceived benefits in crowdsourcing contest.
After a solver has first-hand participation experience with crowdsourcing contests, informed subjective beliefs (e.g., trust belief) can be formed. This present research further extends ECM by incorporating trust in crowdsourcing platform as an ex post psychological expectation to examine the other consequences of confirmation. Solvers may have low initial trust perception of a crowdsourcing platform because they are unsure what to expect from their participation. Once solvers’ expectation is confirmed, they tend to believe the crowdsourcing platform will behave in accordance with their expectation by exhibiting ability, integrity, and benevolence [38], consequently establishing their trust belief. Otherwise, with a disconfirmation solvers are likely to feel uncertain about a number of issues during the participation process. For example, solvers may be unsure if there are safeguards from seekers’ opportunistic behaviors by having solvers do the task, then rejecting their submissions in order to avoid paying, and whether the online transaction system is safe. Thus, confirmation is expected to be positively associated with platform trust.
H8: Confirmation is positively related to platform trust in crowdsourcing contest.
2.2.5 Antecedents of Confirmation
Interaction describes the extent to which solvers can exchange information and comment on solutions before the seeker decides the winner. Interaction is one of the distinctive attributes that crowdsourcing contest advocates, because it has been generally recognized to facilitate the retrieval of relevant and diverse knowledge during the idea generation process [28], thus helping lead to novel ideas [6]. In the context of crowdsourcing contest, solvers’ expectation ranges from achieving hedonic value (e.g., self-development, skill improvement, and fun) to utilitarian value (monetary reward) [45]. A high level of interaction can help confirm the expectation as it enables solvers mutually share information or knowledge, build connections, and arouse innovative ideas. For example, interactions and communications with diverse others help solvers stimulate their associations in memory, revise their own knowledge, generate various alternatives, and refine their ideas [42], [43], making it more likely that solvers confirm their expectation about hedonic value by improving their skills and challenging themselves [10]. In addition, as [6] suggested solvers who comment on a diverse set of other solvers’ ideas can develop a better understanding of task needs, leading to solutions or ideas that are more likely to be valuable to the seekers and thus have greater chances of being accepted. The improved chances of being accepted will help solvers confirm the utilitarian value. Thus, it seems reasonable to believe that interaction will facilitate solvers’ confirmation process.
H9: Interaction is positively related to confirmation in crowdsourcing contest.
According to [24], there are two dimensions in fairness perceptions, namely, distributive fairness and procedural fairness. Distributive fairness relates to the perceived fairness of the outcomes distribution, and procedural fairness relates to the perceived fairness of the process by which the allocations are made [30], [51]. When evaluating crowdsourcing contest, distributive fairness is mainly concerned with how fair the reward is allocated and whether the winning solution deserves the reward after the seeker decides the winner [22]. For example, once a solution is selected as the winner, there is an announcement for a period. Other solvers can form their perceptions of the fairness based on their appraisal of the fit between the goal of the task and the winning solution. Relatively, procedural fairness is concerned with the transparency of participation process before the seeker decides the winner, including the equal treatment of every solution and the opportunity to have a voice in the process. The participation process should enable every solver to equal right and information. For example, the evaluation of solutions is the way to approach the reward. Thus, every solution should be evaluated in order to protect the solver’s benefits. Voice and appeal opportunities are also showed to contribute to procedural fairness by a variety of studies [51], [54].
Extant literature has suggested that perceived fairness is positively associated with a variety of outcomes, such as individuals’ attitudes and intentions [22], [24], [54]. In the crowdsourcing contest setting, intensive competition and reward make solvers more sensitive to violations of fairness [15], [22]. Solvers tend to establish their own fairness perceptions about the decision-making process and decision based on experience and comparison. When the platform lacks transparency, solvers tend to have little confidence in achieving the reward, leading to a low level of confirmation. As solution stealing recently is a risk in crowdsourcing contest [47], lacking transparency and the opportunities to defend will further frustrate solvers in confirming their expectation. In contrast, when fair procedures are in place, solvers would have confidence in the decision-making process and believe they can protect their performance by freely expressing their doubts. Also, under a high level of distributive fairness, the winner solution is generally believed to be the best solution for the task. The winning solver can easily confirm his or her expectation by gaining the reward and achieving accomplishment, and other solvers whose solutions are rejected can confirm their expectation by appreciating the winning solution to improve their skills and knowledge. In sum, the perception of fairness gives solvers some beliefs regarding the platform’s commitment to help them meet the expectation. Hence, with a high level of fairness perception, solvers can be more likely to confirm their expectation. Based on the discussion, we propose the hypothesis.
H10: Perceived fairness is positively related to confirmation in crowdsourcing contest.
The research model for solvers’ continuance intention is illustrated in Figure 2.
3 Methodology
This section illustrates the method of developing variable measurements, the process and method of data collection, and the common method variance analysis.
3.1 Measurement
In order to ensure the measurement quality, we adopt items from existing research and slightly modify each item to fit our research context. We follow the translation-back-translation procedure to translate the original English language questionnaire into Chinese, which is recommended by [9]. A professor at School of Foreign Languages first translated the English language version into Chinese which was then back-translated into English by another English professor. Subsequently, the two English language versions were compared to check the differences to ensure the measurement quality. Toward the disagreements about a few of the items, we analyzed extant literature and discussed them with the two translators to reach an agreement. The measurement items were first evaluated by an expert panel of four academicians and five solvers with practical knowledge of crowdsourcing contest to increase validity and avoid measurement errors. A pre-test survey was then conducted to check the quality of our measurement. It was based on the responses from 34 MBA students who were introduced with crowdsourcing contest knowledge and required to scan the crowdsourcing platform beforehand. The pilot test showed that both the Cronbach’s alpha and item loading of each construct were greater than 0.70. Hence, the measurement items are suitable for our survey.
Four items for interaction are developed based on the study of [36] to measure the extent of information exchange and comment during the participation process. Perceived fairness is measured with five items from [24] and [22] to reflect the fairness of the participation process and distribution, such as equal treatment toward solutions, opportunities to voice views, and fair distribution toward the winning. Six items for perceived benefits are adapted from [4] and [45] to measure solvers’ perception of expected benefits. Four items for platform trust are taken from [39] to measure the extent to which solvers believe this crowdsourcing platform is reliable and trustworthy. The remainder constructs, confirmation, satisfaction, and continuance intention, are measured with the items of [7]. Appendix A shows the specific items. All items in the survey are presented in Likert scale format with anchors ranging from 1 (strongly disagree) to 5 (strongly agree).
3.2 Data Collection
We conducted the survey on Epwk.com to examine our hypotheses. Epwk.com was founded in 2010 and has become one of the most widely used online crowdsourcing markets in China. On Epwk.com, a seeker first fills out an online request form with the task title, the reward amount(s), the closing date for submissions, and the number of submissions that will be selected as winners. When the closing date is reached, Epwk.com sends a notice to the seeker asking the seeker to select the best solution(s) among all the submissions. The seeker can also choose the best solution(s) before the closing date. In this case, solvers are informed that a solution has been selected and the task is closed. As of April 16 2017, Epwk.com had accumulated more than 4 million tasks, having a demand on solvers’ massive and sustained participation and providing a proper setting to study solvers’ continuance intention.
We administered two surveys separated by a 2-week lag to ensure the quality of our data. At time 1, a task was posted on Epwk.com to recruit respondents for our data collection. The task stated the objective of the survey and the requirements of subject recruitment at the beginning. We specified that only solvers who have participation experience about crowdsourcing contest tasks would be qualified to participate in the survey. This is because participation experience is needed to form the perceptions, to build ex post expectation, and then to make decision about continuance intention in a given crowdsourcing contest platform, but not in a specific crowdsourcing task. The URL of the questionnaire Web page was shown in the description. Every respondent in our survey would get a compensation of RMB 10. At this stage, respondents were invited to evaluate the level of interaction, perceived fairness, perceived benefits, and platform trust. We received 493 responses. At the time 2 stage, we re-invited the 493 solvers who participated at Time 1 survey to fulfill a follow-up survey on basis of the USER ID. The follow-up survey was to assess solvers’ confirmation, satisfaction, and continuance intention. All the measurements were randomized arranged on the two questionnaires. Every respondent would get an additional compensation of RMB 15 at Time 2 survey. Of the 493 solvers, 336 agreed to participate. In both of these two surveys, we put some detective items at the middle part of the questionnaire to test if respondents read the survey carefully, for example, For this question, please choose the second answer. After checking the responses, we discarded inconsistent or erroneous questionnaires, and we finally obtained 306 valid responses.
Of the 306 responses, 51.3% are male, and 48.7% are female. 93.5% have received at least some college education, indicating a good education background. 78.1% are in the age range from 20 to 40. In terms of participation experience, there is a wide range. 26.8% of respondents have participated in 1-2 tasks, 39.5% have participated in 3-5 tasks, 13.1% have participated in 6-10 tasks, and 20.6% have participated in more than 10 tasks.
3.3 Common Method Variance
Although our data was collected with a time-lagged survey, a threat of common method variance (CMV) may exist [44]. Hence, we use two techniques to test if this study suffers from excessive common method variance. Firstly, Harman’s single-factor test is used to ensure that the variance of our data does not largely come from a common method source. All of the measurement items of major constructs and control constructs (i.e., age, education, and experience) were subjected to a principal component analysis to see how many orthogonal components would emerge to explain the variance of the data. Results showed that seven components with eigenvalues of more than 1.0 emerged, and the largest single component accounted for 37.0% of the variance.
Following the study of [32], we secondly conducted a common method factor test by making the measurement items of each construct simultaneously load upon its construct as well as upon a common-method factor, which is created using items from all of the constructs (Appendix B). Results indicate that each item’s variance explained by its substantive construct is much greater than that explained by the common method factor. The sum of squared item-loadings shows that the constructs of this study explain, on average, over 68.4% of the item variance. In contrast, common method variance totally accounts for 33.6% of the variance. Therefore, given the small magnitude of method variance, we contend that the common method variance is unlikely to be a serious concern for this study.
4 Data Analysis and Results
We use partial least squares (PLS) to test the measurements and hypotheses. PLS estimates relationships among latent variables by taking measurement errors into account and thus provides more accurate results [23]. As a component-based approach, PLS has a low demand on sample size and residual distribution [12]. PLS is particularly suitable for testing complex structural models as it avoids two problems: inadmissible solutions and factor indeterminacy [21]. Hence, SmartPLS 2.0 is employed for our analysis.
4.1 Measurement Reliability and Validity
The reliability of measurements is assessed using Cronbach’s alpha and composite reliability (CR). Table 1 shows that the scores of Cronbach’s alpha and CR of all the constructs range from 0.716 to 0.907, all above the recommended threshold value of 0.70 [40]. The convergent and discriminant validity of measurements are tested with average variance extracted (AVE) and item loadings. To achieve adequate convergent validity, AVE should be above 0.5, and item loading should be greater than 0.7 [12], [21]. Our analysis results of AVE (Table 2) and item loadings (Table 1) indicate that our data has a valid convergent validity. The square root of the AVE is suggested to test discriminant validity [12]. Data shown in Table 2 indicates that the square root of the AVE of each construct is much higher than all cross-correlations between the construct and other constructs, confirming the discriminant validity of our measurements.
4.2 Hypotheses Testing
This study assesses the hypotheses using structural equation modeling (SEM) which is suitable to validate multiple causal relationships simultaneously. SmartPLS 2.0 is used to estimate the structural model and the significance of the paths by jointly using path coefficients and the R2 [12]. Figure 3 displays a schematic representation of the testing results. The estimation of our extended model accounts for 42.2% and 54.4% of the variance respectively in continuance intention and satisfaction.
Perceived benefits is found to significantly affect satisfaction (b=0.202, p<0.01) and continuance intention (b=0.210, p<0.01). Results in Figure 3 also show that there is significant positive association between confirmation and perceived benefits (b=0.655, p<0.001). These findings confirm the importance and appropriateness of perceived benefits in accounting for continuance intention in the context of crowdsourcing contest and respectively support Hypotheses 2, 3, and 7. In addition, the links between satisfaction and continuance intention (b=0.374, p<0.001) and between confirmation and satisfaction (b=0.442, p<0.001) are also found to be significant at 0.001 level. Hypotheses 1 and 6 are thus supported. Platform trust, an important psychological factor, is added in our extended ECM framework to examine whether solvers care about social-related aspects in the process of making decision on continuance participation. Our results (Figure 3) show that platform trust is a significant predictor of satisfaction (b=0.208, p<0.001) and continuance intention (b=0.162, p<0.05). Results in Figure 3 also indicates a high level of platform trust can be fostered by the confirmation level (b=0.504, p<0.001). Hypotheses 4, 5 and 8 are consequently supported by these findings. On account of the significant effects of confirmation on satisfaction, perceived benefits and trust, exploring the antecedents of confirmation seems to be important. Our results show that both interaction (b=0.198, p<0.001) and perceived fairness (b=0.506, p<0.001) are significantly associated with confirmation. These two variables, in total, explain 38.5% of the variance in confirmation. These results provide evidence for Hypotheses 9 and 10. In sum, the extended research model and all hypotheses are supported.
In order to test the robustness of our results, we first test the direct links between interaction and satisfaction (b=-0.044, p>0.1), between interaction and continuance intention (b=0.038, p>0.1), between perceived fairness and satisfaction (b=0.143, p<0.05), and between perceived fairness and continuance intention (b=-0.041, p>0.1). Then, we test the impacts of control variables (i.e., age, education, and experience) on continuance intention (b=-0.027, p>0.1; b=0.045, p>0.1; b=0.086, p>0.1), satisfaction (b=-0.017, p>0.1; b=0.014, p>0.1; b=0.039, p>0.1), platform trust (b=-0.003, p>0.1; b=-0.021, p>0.1; b=0.122, p<0.05), and perceived benefits (b=-0.028, p>0.1; b=-0.028, p>0.1; b=0.019, p>0.1). When including above direct links, the path directions and significance levels of our hypotheses remain unchanged.
5 Discussion and Implication
The objective of this study is to examine the determinants of solvers’ continuance intention in crowdsourcing contest. We apply the ECM framework as the overarching theoretical foundation and integrate relevant factors to develop our research model. The empirical testing with a sample of 306 solvers confirms our extended model and all hypotheses.
Being similar with the study of [34] and considering the characteristics of crowdsourcing contest, this study uses perceived benefits to replace perceived usefulness in order to better represent solvers’ performance expectation toward their participation. Empirical results provide support for the suitability of using perceived benefits as it is positively related to continuance intention (b=0.210, p<0.01) and satisfaction (b=0.202, p<0.01). Additionally, empirical results suggest that platform trust is another crucial factor in eliciting a high level of continuance intention (b=0.162, p<0.05) and satisfaction (b=0.208, p<0.001) in crowdsourcing contest. These findings suggest that both performance-related aspects and psychology-related aspects are in the concerns of solvers. A lack of value or trust will significantly reduce solvers’ satisfaction level, which in turn damages their continuance intention.
On the other hand, our results indicate that satisfaction has a greater effect on continuance intention (b=0.374, p<0.001) than perceived benefits and platform trust. Perceived benefits and platform trust are also found to significantly affect the level of satisfaction. These findings suggest a pathway of satisfaction through which perceived benefits and platform trust can enhance continuance intention in crowdsourcing contest. In the virtual transaction environment, perceived benefits and platform trust from two different aspects (i.e., material and psychological) provide the nutriments and confidence that enable individuals to pursuit expectation, thus leading to an enhanced affect. The affection as a result of satisfaction can engender goal-directed intention which refers to continuance participation intention in this study. Meanwhile, compared with perceived benefits, platform trust has a more significant effect on satisfaction rather than on continuance intention. This is perhaps because satisfaction is mainly used to represent individuals’ affect which can be easily influenced by psychological factors, such as trust.
Confirmation is an important element in the ECM framework, and our results shows that confirmation is positively related to satisfaction (b=0.442, p<0.001), perceived benefits (b=0.655, p<0.001) and platform trust (b=0.504, p<0.001). The importance of confirmation makes it meaningful to investigate its antecedents in the applied contexts to better leverage the ECM framework [11]. Our empirical results provide supporting evidence that both interaction and perceived fairness are positively related to a high level of confirmation. Compared with interaction (b=0.198, p<0.001), which is noteworthy is that perceived fairness has a stronger effect (b=0.506, p<0.001) on confirmation. The explanation may rest on the intensive competition and monetary reward in crowdsourcing contest, which make solvers concern more about material outcomes with their effort and wisdom investments. It signifies and stresses fairness to be a salient factor for solvers’ reactions.
5.1 Implications in Theory
Through the examination of an extended ECM to understand solvers’ continuance intention in crowdsourcing contests, this study contributes to research in several ways. First, this study contributes to the crowdsourcing continuance intention literature and provides insights regarding why solvers sustain their participation intention with the ECM framework. To the best of our knowledge, this study is among the first study to apply ECM framework to explain solvers’ continuance intention in crowdsourcing context. Previous studies in this stream mainly focus on expectancy value theory, group engagement model, etc. [8], [49], paying little attention to potential changes in solvers’ expectation and factors of influencing these changes. The ECM model is a useful theoretical framework for explaining continuance behavior with the consideration of post-expectation. Further, ECM framework can provide researchers with ample flexibility to incorporate contextual factors to study continuance intention in various contexts [11]. Our application and confirmation of the ECM framework prove ECM to be a viable theoretical foundation in crowdsourcing contest and suggest ECM as an alternative theoretical framework for future studies.
Second, the original ECM framework only focuses on performance expectation (i.e., perceived benefits which is used herein to replace perceived usefulness). However, we argue that in the virtual transaction environment individuals have demand on psychological expectation, such as trustworthiness. In the crowdsourcing contest context, the crowdsourcing platform is the intermediary for solvers to interact with others and to achieve their rewards. Thus, whether the platform is reliable and trustworthy may have significant effect on solvers’ sustained intention and behavior by reducing or eliminating potential risks. However, limited prior crowdsourcing studies have addressed the role of platform trust [37]. Our study supplements the extant crowdsourcing literature by proving platform trust to be a strong determinant of both satisfaction and continuance intention. In addition, our empirical results verify that platform trust can be significantly evoked by the confirmation level. These findings suggest that solvers not only care about performance-related expectation, but concern psychology-related expectation. Our incorporation of platform trust in the research model provides a new perspective to study continuance intention as prior related research primarily focuses on performance expectation when applying the ECM framework.
Third, due to the importance of confirmation, relevant studies should pay considerable attention to its antecedents in order to strengthen the understanding of confirmation. Understanding the antecedents of confirmation will help better address individuals’ continued behaviors [11]. Echoing to some extant studies in calling for attention on the antecedent of confirmation, this study broadens the research stream of ECM by investigating whether contextual factors can induce a high level of confirmation. Standing on the features of crowdsourcing contest, we employ interaction and perceived fairness to explain solvers’ confirmation process. Results demonstrate that both interaction and perceived fairness can trigger a high level of confirmation. The power of interaction and perceived fairness in predicting confirmation level provides future related studies a new perspective to understand why individuals would like to confirm the expectation. These results are also consistent with prior findings that interaction and perceived fairness have significant influence on attitudinal and behavioral reactions [22], [46]. Future crowdsourcing research should closely notice the importance of interaction and perceived fairness.
5.2 Implications in Practice
Several implications for crowdsourcing practitioners can be drawn from this study. First, this study suggests that besides satisfaction, both perceived benefits and platform trust also have significant effects on continuance intention in the context of crowdsourcing contest. In addition, a high level of satisfaction can be achieved from perceived benefits and platform trust. These findings provide crowdsourcing contest managers clues about how to improve solvers’ satisfaction level and encourage them to engage in more tasks. When solvers perceive their participation as valuable and trustworthy, they are more likely to be satisfied and consequently foster continuance intention. Therefore, managers should pay close attention to mechanisms that help solvers promote their perceptions about value and trust. For example, crowdsourcing platforms can establish the threshold of reward to improve solvers’ effort compensation and periodically carry out trainings to enhance the perception of value. Additionally, a perception of value can be achieved with a rating system which can be used to exhibit solvers’ reputation and skills. Given the importance of platform trust in crowdsourcing contest, our study recommends managers some suggestions to build trust. For instance, as recently information fraud is a severe problem in online environment, crowdsourcing platforms should first provide reliable information to improve trust level. Toward the disputable tasks, proofreading the solutions to supervise seekers’ opportunism behavior is also an important mechanism to increase trust belief. Using a reliable trading system to improve solvers’ confidence in their account security is another effective way to increase trust.
Second, confirmation is verified to be a primary determinant of satisfaction, perceived benefits, and platform trust. Crowdsourcing managers should notice the importance of confirmation in evoking solvers’ positive reactions (e.g., satisfaction, perceived benefits, and platform trust), which in turn can encourage solvers to sustain their participation intention. Besides the initial expectation and perceived performance, our study suggests that solvers can achieve a high level of confirmation from interaction and perceived fairness. In order to enhance the confirmation level, therefore, platforms can take actions to ensure high levels of interaction and fairness. For example, on the one hand, smooth interaction and communication and active comment should be highly enabled to allure positive reactions. Importantly, on the other hand, fair procedures and distribution should be paid close attention in crowdsourcing contest, because results show that perceived fairness has a more significant effect on confirmation than interaction does. Solvers may disconfirm their expectation if they find there are unfairness even high-quality interaction exists. Therefore, it is important for crowdsourcing managers to notice why the contest is perceived as unfair. Our study suggests low procedure transparency and unfair distribution on the winning solution can lead to perceptions of unfairness. Mechanisms, such as ensuring that every solution is assessed, and giving solvers opportunities to express their opinions when there are any disputes, are suggested to be able to increase procedural fairness. Also solvers will compare the winning solution with their own solutions to evaluate the distribution fairness. It would be better to force seekers give explanation about their decisions to eliminate or reduce potential doubts.
5.3 Limitations and Future Research
Despite some implications in theory and practice, this study has some limitations. First, this study extends the expectation-confirmation model by incorporating particular factors in the specific context of crowdsourcing contest. Thus, these factors may not generalize to other different research contexts. Related research is needed to identify context-dependent factors and how they impact across contexts. This study provides an example for using ECM framework by incorporating both performance-related and psychology-related factors. Second, the data for our empirical testing is cross-sectional and only provides a one-time illustration of the perception and confirmation process. [7] argued an ideal empirical design for testing ECM framework would be longitudinal, thus it will be more valuable if a longitudinal study is considered. Further, prior studies have indicated that national culture (e.g., individualism versus collectivism, uncertainty avoidance, long-term versus short term orientation, power distance, masculinity versus femininity, and culture tightness) can influence an individual’s general tendency to trust [25] and the success in creative task [13]. The single-source of our data may be another limitation for the generalizability of our findings. Future studies should pay attention to the cross-cultural issue.
6 Conclusion
This study responds to the call for a better understanding of the triggers of solvers’ continuance intention in crowdsourcing contest by using an extended ECM framework. Specifically, perceived benefits, platform trust, interaction and perceived fairness are integrated into the extended ECM framework. Using a sample of 306 solvers, our research model are tested and verified. Results show that solvers have the intention to continue their participation in crowdsourcing contest not only because they enjoy high levels of satisfaction, also because they enjoy high levels of perceived benefits and platform trust. Besides the positive predictions on continuance intention, perceived benefits and platform trust are found to have influence on satisfaction. Further, our empirical results demonstrate that interaction and perceived fairness are two significant determinants of confirmation level. With these findings, this study contributes to theory and practice in several ways by explaining how to sustain solvers’ continuance intention with an ECM framework.
