Moderating Effects of the Timing of Reward Determination and Performance Standards between Rewards and Self-Efficacy for Sustainable Intrinsic Motivation

Abstract

The purpose of this study is to determine the moderating effects of the timing of reward determination and performance standards on the relationship between pay-for-performance and self-efficacy. It is an experimental study; the sample included 352 participants from Amazon Mechanical Turk, and an online experiment was conducted on an external website. The model was tested for mediation and moderation processes using regression analysis and analysis of variance (ANOVA). The results showed a mediating effect of self-efficacy between pay-for-performance and intrinsic motivation. A moderating effect of performance standards (absolute, relative, ambiguous) on the relationship between pay-for-performance and self-efficacy was also found. Moreover, performance standards were found to be more important moderators than the timing of reward determination. The theoretical contribution of this paper was to observe the concept of timing of reward determination and empirically validate self-determination theory. The results also infer that people measure their own efficacy or competence by comparing themselves with others more than with other performance standards. The use of absolute performance standards is recommended for sustainable self-efficacy and intrinsic motivation of employees. This study introduced the concept of the timing of reward determination (i.e., before or after completion of a performance-related task) and verified the moderating effect of performance standards.

1. Introduction

According to Dodger’s Digest [1], the Los Angeles Dodgers, a Major League Baseball team in the U.S., signed a contract with Kenta Maeda for 8 years. The total amount of money Maeda could earn in 8 years was $106.2 million; however, only $25 million was guaranteed. He could also earn $3 to $13 million depending on his season record every year. This strategy was used to motivate the athlete to make efforts in maintaining good health and meeting incentives [2]. Athletes’ annual salaries are generally set according to their performance from previous years, with the expectation of a similar season to come. However, Maeda’s contract suggested that the timing of reward determination may vary, which may cause discontent to the employee and create difficulties in a sustainable management environment if the situation is different from the employee’s expectations [3].

Most athletes’ contracts include both the base salary and incentives. An incentive is defined as “a promise of pay for some objective, pre-established level of performance” [4] and it is regarded as one of the positive consequences [5]. Gross annual salaries are determined based on previous performance and incentives, while rewards can be recognized as being determined after the job has been completed. By contrast, the incentive conditions are determined before the job begins. Thus, the timing of determining the reward may vary as it may occur before or after the job is completed. We determined it is necessary to investigate the effect of timing of reward determination on managerial conditions because this variation can have effects on some variables or change the effects between variables. Therefore, we investigate the effects of the timing of reward determination for sustainable managerial conditions in this paper [6].

As the term ‘timing’ represents the moment that an event is likely to happen [7], we define the term ‘timing of reward determination’ as the moment when a reward is decided upon. Although it is generally assumed that rewards are determined after the performance is completed, there are many cases in which a reward is determined before the job is finished. It is not problematic if rewards are provided as promised, but the pre-determined reward may be canceled or reduced if any complications or unforeseeable problems arise. An example of one can be found in the contracts containing the penalty conditions, the cancellation or reduction of rewards occurs if the worker fails to meet a required performance goal [8]. This can be applied as a clawback to maintain the performance of some workers such as salespeople and lawyers who have to return their commissions to the firm if they fail to meet a goal level [9,10]. Chief Executive Officer (CEO) contracts are also examples of pre-performance determination. These include stock options, providing executives stock as an incentive for better stock price performance and conditions related to early termination for poor performance [11]. Pre-determined reward can be operated not only as a clawback but also as bonus deferrals or a bonus cap [12]. These negative consequences are also regarded as a form of punishment [13].

Contrarily, a post-performance reward is determined after the task has been performed. Year-end bonuses and merit-pay increases are the examples of post-performance rewards. The concept, timing of reward determination, as far as we know has not yet been studied, and this paper may be the first to suggest the concept and examine its effect.

What we wanted to analyze was not only the effects of the timing of reward determination but also the effects of performance standards with self-efficacy because self-efficacy is positively related to sustain performance [14,15,16]. In previous research, effects of performance standards on employee behavior have been elucidated. The performance standards can be divided into an absolute standard, relative standard and an ambiguous standard [6,17]. Both absolute and relative standards involve providing explicit standards, which require absolute goal performance or relative goal performance such as exceeding others. The ambiguous standard involves providing implicit and unspecific performance goal, such as “doing well”. Eisenberger, Pierce, and Cameron [6] found that different performance standards cause different effects of pay-for-performance on intrinsic motivation, and this may be caused by the different goal characteristics of performance standards [18]. We conceptualized and examined this difference of effect as a moderating effect.

Both the timing of reward determination and performance standards may affect self-efficacy and the intrinsic motivation of employees. We are interested in comparing the effects of the relatively unknown concept of the timing of reward determination and the effects of performance standards on particular measures of employee behavior. In this study, we investigated the moderating effects of the timing of reward determination and performance standards on the relationship between pay-for-performance and self-efficacy. In addition, the mediating effect of self-efficacy on the relationship between pay-for-performance and intrinsic motivation was also examined.

2. Theoretical Background and Research Hypotheses

2.1. Rewards and Intrinsic Motivation

Generally, motivation can be divided into intrinsic motivation, extrinsic motivation, and achievement motivation. Intrinsic motivation refers to behavior derived from internal factors such as interest or curiosity, and extrinsic motivation is driven by external rewards such as money, praise or fame [19]. Achievement motivation is the need to achieve the performance goal to be judged as competent [20].

Extrinsic motivation can be maintained and maximized by providing an appropriate and sustained reward [21]; otherwise, when there is no reward given, extrinsic motivation decreases [21,22]. In organizations, employees are often motivated by their need to succeed or climb the ladder, or else known as achievement motivation. Achievement motivation is related with goal setting theory which argues that challenging goals motivate people to strive for success and achievement [23,24]. It is related with self-efficacy [25] and there is some overlap with self-determination theory [26], which will be discussed further in the section of self-efficacy with goal-setting theory. Intrinsic motivation is the product and function of psychological factors [27]. The factors that affect intrinsic motivation has been studied widely because of its importance on sustainable management [6,22,28,29,30,31] and it has been found to have a positive effect on sustainable work behavior [32]. Therefore, in this study, we have tried to focus on intrinsic motivation for sustainable effort and performance.

According to self-determination theory, intrinsic motivation can be increased to satisfy the basic human needs of autonomy and competence [27]. Conversely, rewards that cause feelings of being controlled can decrease intrinsic motivation. However, rewards based on achievement can increase intrinsic motivation. Some studies [22,28] emphasized the effect of control and concluded that most of the expected tangible rewards have negative effects on intrinsic motivation, while verbal rewards can have a positive effect on intrinsic motivation. Others [17,30] emphasized the effect of competence, and they concluded that external rewards can have positive effects on intrinsic motivation. The direction of the effect of external reward on intrinsic motivation depends on which aspect, autonomy or competence, is emphasized.

The study of Eisenbeger et al. [6] based on meta-analysis concluded that with respect to pay-for-performance (performance contingent), the rewards given to those who meet specific performance standards have positive effects on intrinsic motivation. This is also consistent with self-determination theory’s informational aspect of competence, which is to fulfill the human need to feel competent, thereby increasing intrinsic motivation [27]. Several field studies support Cameron et al.’s findings [28] and have shown that pay-for-performance has positive effects on the enjoyment of tasks and job satisfaction [33,34].

2.2. Self-Efficacy and Intrinsic Motivation

We will focus and discuss on the competence aspect of the two factors that determine intrinsic motivation; autonomy and competence. Intrinsic motivation is associated with psychological factors and it is directly related to perceived competence of how an individual perceives their self-capability [19]. The term ‘perceived competence’ is similar to self-efficacy, so we will discuss the concept and role of self-efficacy further.

Self-efficacy refers to a person’s belief that he/she can do well on his/her task [35]. Similar concepts include perceived competence and confidence [36]. Self-efficacy is an employee’s comprehensive assessment of his/her perceived competencies to perform certain tasks. This is an important factor because it is highly associated with various positive factors of organizational sustainability. Self-efficacy has been found to increase performance [37,38], job satisfaction [39,40], career commitment [41], OCB [42], work engagement [43], and affective commitment, as well as decrease turnover intention [44], stress, and burnout [45].

Self-efficacy is affected by a number of factors, such as others’ and the organization’s opinions of the employee, the arrival of new information, how well the employee performs his/her duties, and new experiences [46,47]. Enactive mastery, the most important way to increase self-efficacy [36], refers to the positive experience of successfully completing a job or task. It infers that achievement increases self-efficacy because it informs the individual that their behavior produced results satisfactory enough to have earned a reward. Harackiewicz et al. [48] supported that performance-based rewards signal achievement, the importance of completed tasks, and competencies of employees. Receiving rewards under a pay-for-performance paradigm means that recipients performed better than certain performance standards, which could be seen as an experience of success and positive feedback. Thus, receiving pay-for-performance can increase self-efficacy [49].

People have an inherent psychological desire for praise in order to increase confidence [27]. Rewards interpreted as an indicator of ability can satisfy this psychological need in such a way that intrinsic motivation increases. In social cognitive theory, the events associated with achievement may also increase intrinsic motivation. Pay-for-performance has a significant informative effect on self-efficacy, and increased self-efficacy escalates interest in completing tasks, which in turn increases intrinsic motivation [35].

Self-determination theory asserts that perceived competence increases intrinsic motivation [27]. It also suggests a mediating effect of self-efficacy between pay-for-performance and intrinsic motivation. Ferla et al. [50] demonstrated that self-efficacy functions as a mediator in this relationship between difficulty level of task and interest. In addition, Cameron et al. [30] determined that perceived competence has a mediating effect between reward performance and task interest. Furthermore, Young [51] showed the mediating effect of perceived competence between performance climate and intrinsic motivation. A previous study surveyed 609 employees and showed that the two factors identified in self-determination theory—perceived competence and perceived autonomy—partially mediated the relationship between receiving rewards due to achievement and intrinsic motivation [52]. Other researchers showed fairly distributed financial rewards cause feelings of competence and autonomy, which then increases autonomous motivation [53]. Based on the findings of these studies, we suggest the following hypothesis:

Hypothesis 1.

Self-efficacy mediates the positive relationship between pay-for-performance and intrinsic motivation.

Figure 1 is a graphical representation of Hypothesis 1.

2.3. Concept, Type, and Effect of Performance Reward

As mentioned earlier, reward systems can be categorized as pre-performance or post-performance. Pre-performance rewards are the rewards decided before the actual performance of a given task. If the reward is not provided as previously promised, it may cause discontent and disappointment to employees. This kind of problematic situation includes clawback [9,10], stock options, contract clause of early determination of CEO [11], bonus deferrals, and bonus caps [12]. In most pre-performance reward systems, rewards may be reduced or revoked if an expected level of performance is not met. In comparison, post-performance rewards are the rewards decided after the actual performance. In most post-performance reward systems, the potential recipients are unsure if they will receive a reward until the task is complete; however, there is no risk of cancellation of promised compensation. Although post-performance reward systems are more familiar, pre-performance reward systems have gained popularity and are more prevalent than we may think.

Post-performance rewards can have positive effects, representing competence, and the rewards do not cause any negative effects; rather, there is an opportunity to receive additional rewards, which may increase positive emotions. On the contrary, failure to receive rewards causes negative effects on self-efficacy because it represents the negative feedback of competence [35]. It refers that pay-for-performance will work in line with self-efficacy theory.

In contrast, in a pre-performance reward environment, people may feel pressure, unlike in a post-performance reward system. Individuals may feel at risk of their rewards being reduced or revoked according to their performance. Furthermore, this risk may be considered as a method of punishment. A punitive reward is defined as unpleasant measures or even harmful actions taken in response to certain results [54]. Some examples include reprimands, dismissals, salary cuts, and suspended promotions [55]. Pre-performance rewards can be considered similar to punitive rewards as there is a possibility for cancellation of bonuses or revocation of rewards. Though pre-performance rewards can entail both positive effects, such as representing competence, and negative effects, such as feeling controlled, the negative effects are likely to be greater because “bad is stronger than good” [56,57,58]. As job control is one of the factors that cause job stress [59], lower autonomy under pre-determination could cause stress to employees. Stress can cause negative feelings related to competence and achievement; efficacy and accomplishment can be reduced though they achieve their performance goal [60]. Finally, it is a state of tension where successful experiences can turn into failure experience.

Therefore, we hypothesize that the timing of reward determination acts as a moderating variable. Though pay-for-performance intends to increase perceived competence, a pre-performance reward would weaken the positive effect of pay-for-performance on self-efficacy. It is because pre-performance reward environment causes the tense state that compensation may be cancelled; also, the environment would make employees recognize that their behaviors are motivated by extrinsic factors and this could cause the disregard of internal factors such as interest, effort, skill, and ability [30]. For these reasons, we hypothesize that pre-performance rewards may have a moderating effect between pay-for-performance and self-efficacy. Thus, we suggest the following hypothesis:

Hypothesis 2.

The timing of reward determination (pre- or post-performance) moderates the relationship between pay-for-performance and self-efficacy such that the relationship is weaker under pre-performance rewards than post-performance rewards.

2.4. Types and Effects of Performance Standard

When Eisenberger et al. [6] analyzed the effect of reward on intrinsic motivation; they divided performance standards into relative standard, absolute standard, and ambiguous standard. Relative standard involves comparison with other’s performance; absolute standard involves meeting a specific score or completing a given task, and an ambiguous standard is an unclarified performance goal (i.e., “doing well”). Eisenberger et al. [6] asserted that different performance standards result in pay-for-performance having different effects on intrinsic motivation. It was observed in this study that relative performance standards have the most positive effects on intrinsic motivation.

This result can be explained using goal-setting theory and social comparison theory. Goal-setting theory asserts that people are more motivated by challenging and difficult goals [61]. To increase motivation, employees must have a clear idea of the goal and how difficult it will be to achieve [62]. An acronym has been created to help management formulate goals: they should be SMART—specific, measurable, attainable, relevant, and time bound [18,63,64]. Social comparison theory also supports the different effects as the theory argues that people tend to evaluate their self-capability by comparing with others [65]. Several studies have supported the idea that the social comparison process occurs to identify one’s own self capability in comparison to others [66,67] and is also concerned with the development of the self-concept [68,69].

Based on the theories, we could infer that the difference in performance standards can also result in differences on the relationship between pay-for-performance and self-efficacy. We inferred that ambiguous and relative performance goals would strengthen the negative effect of failure on self-efficacy. A person working under ambiguous performance standards may have difficulties in finding the meaning of the task because the goal is not specific or measurable. This perception may have a negative effect on the relationship between pay-for-performance and self-efficacy. It could cause a lower sense of self-efficacy if they failed to meet the performance goal, compared to an absolute performance goal. Also, a person working under relative performance standards may strengthen the negative effect of failure. As people usually judge their own competence by comparing their own performance with that of others [70], the relative performance goal can strengthen the negative effect if they did not reach the performance goal, compared to an absolute performance goal. Thus, we propose the following hypothesis:

Hypothesis 3.

Different performance standards moderate the relationship between pay-for-performance and self-efficacy; the negative effect of no reward under pay-for-performance would be stronger under relative standards and ambiguous standard than absolute standards.

Figure 2 displays a graphical representation of Hypotheses 2 and 3.

3. Methodology

3.1. Participants and Procedure

A total of 504 individuals participated in the study and after 152 were excluded, data for 352 participants were used in the statistical analysis. Participants were excluded due to multiple submitted entries ($\mathrm{n}=27$) or provided untrustworthy responses, such as repeating the same answers or achieving unrealistically low scores on the task ($\mathrm{n}=125$).

Participants were recruited through the Amazon Mechanical Turk (MTurk) platform and completed the online experiment through an external website. Previous studies have recognized the MTurk platform as a reliable tool for recruiting participants for online surveys and experiments. MTurk is recognized as a promising platform for conducting research with diverse users [71] and an appropriate data collection tool for several reasons, including the fact that samples are more demographically diverse and reliable than traditional samples [72]. MTurk workers have been found to be socio-economically and ethnically diverse [73] and highly educated and younger than the general population. Thus, MTurk can be a reliable source of data for experiments in judgment and decision-making, and it reduces the probability of non-response errors in online research [74]. Not only is an MTurk sample more ethnically diverse, but the demographics of such a sample have been found to be similar to those of community samples [75]. Based on these studies, we assumed the characteristics of the participants were sufficiently random and evenly distributed. Before starting the experiment, participants were randomly assigned to each experimental group in the order that each individual had accessed the website.

3.2. Instruments

Variables that were measured in this study include performance (the number of correct answers), self-efficacy, and intrinsic motivation. A questionnaire was used to measure self-efficacy and intrinsic motivation. Participants were asked to answer the perceived competence subscale and task interest/enjoyment subscale of the Intrinsic Motivation Inventory [76]. The subscale of perceived competence comprised 6 items (e.g., “I think I’m pretty good at this activity”) and task interest/enjoyment contains a total of 7 items (e.g., “I enjoyed doing this activity very much”). The questionnaire items were all measured using a 7-point Likert scale, ranging from 1 to 7 (1: not at all true, 4: somewhat true, 7: very true). The mean value of the items was used for analysis, and the questionnaire was administered to participants after the completion of each task. The variables measured after the first task, ‘self-efficacy 1’ and ‘intrinsic motivation 1’, were used as control variables and other variables measured after the first task were used to check the credibility of responses. Variables measured after the completion of the second task, ‘self-efficacy 2’ and ‘intrinsic motivation 2’, were used for analysis. The internal consistency of the scales was estimated using Cronbach’s alpha and the Cronbach’s alpha coefficients showed that all scales had acceptable reliability. The Cronbach’s alpha coefficient of ‘self-efficacy 1’ was 0.860, ‘self-efficacy 2’ was 0.882, ‘intrinsic motivation 1’ was 0.875, and ‘intrinsic motivation 2’ was 0.877.

This study manipulated several variables. As participants’ performance in the experimental task was not easy to predict and the time limit for each question was very tight, we assumed that the participants would not know the exact score of the task. Therefore, the manipulated variables included (1) rewards (whether received or not), (2) the timing of reward determination, and (3) performance standards.

3.3. Design of the Experiment

Participants were selected from the Amazon Mechanical Turk (MTurk) website where they were then given access to the experimental website and were provided with an explanation of the task. After they gave their consent to take part in the study, participants were then directed to complete the first experimental task. The task provided was used to measure each participants’ performance. Furthermore, the task was designed to require a low level of skills and cognitive abilities to prevent personal preference effects. The experiment was set up to be simple, but it is not easy to create tasks that could be done with sufficient effort. As a result of these concerns, the task given was to count the number of specific alphabets within a certain time limit.

Participants were given a one-minute time limit to count five specific letters from five lines of random alphabet arrangements containing 125 letters (sample question: “How many ‘A’s in the above?”). This task was then repeated with new sets; a total of 10 questions was given to each participant. The maximum points participants could receive was 10 points per problem, with a grand total of 100 points. The problems required the full allotted time of one minute and a pilot test was conducted beforehand to determine the optical difficulty level. After completing the first task, participants received feedback about their performance. They were not informed of their real performance, but were given fake results and feedback according to their experimental group. The variables were measured with a questionnaire, and this process was repeated in the following order: performing the task, receiving feedback, and completing the questionnaire.

During the experiment, different instructions were provided to participants depending on their experimental condition.

Table 1. Statements Provided to Each Group.

	Statement Provided to Participants
Performance standard	(A) Your score on the first part of the experiment is 70 points. You have performed well on the first part of the experiment.
	(B) Your score on the first part of the experiment is in the top 28%. You have performed well on the first part of the experiment.
	(C) You have performed well on the first part of the experiment.
Reward system	(1) You earned an extra $0.20 in addition to the $0.30 you were initially promised, so you are entitled to $0.50 in total. … If you perform as well as you did on the first part of the experiment, you will earn $0.50 in total. However, if you do not …	(2) You can earn an extra $0.20 if you
Performance standard	(A) score as well on the second part (below 70 points) …	(A) score above 70 points
	(B) score as well on the second part (rank in the top 28%) …	(B) rank within the top 28%
	(C) score as well on the second part	(C) perform well
Reward system	(1) you will lose the right to earn the extra $0.20, leaving you with a total $0.30.	(2) on the second part of the experiment. So, you have a chance to earn $0.50 in total.

Note: The meanings of each letter or number are as follows: performance standard: absolute (A), relative (B), and ambiguous (C). Reward: pre-performance reward (1), and post-performance reward (2).

lists the statements provided to participants in sequence.

All participants completed the tasks under two settings: one of two reward conditions (1) or (2) and one of the three performance standards (A), (B) or (C). In a timely manner, participants received instructions about the experiment; then, one of the statements next to the first heading, “Performance standard” was given as their condition. Next, one of the statements besides the second heading, “Reward system” was given. Finally, one of the statements next to the third heading, “Performance standard” and fourth heading “Reward system” was given. For participants under condition (1), the statements located on the left side of the line were given and statements located to the right of the line were given to participants under condition (2). For example, if a participant was assigned to conditions (1) and (B), the statements given were, “Your score on the first part of the experiment is in the top 28%. You have performed well on the first part of the experiment” (first statement) and “If you perform as well as you did on the first part of the experiment, you will earn $0.30 in total. However, if you do not score as well on the second part (i.e., rank in the top 28%), you will lose $0.20, leaving you with a total of $0.10.” This latter statement combined the second, third, and fourth statements.

After completion of all the processes, in the last stage of the experiment, participants were debriefed about the experiment. After all the experimental tasks and measurements were complete, the same reward was given to all participants.

Participants were divided into 6 experimental groups.

Table 2. Number of Participants in Each Group.

Timing of Reward Determination	Absolute Standard	Relative Standard	Ambiguous Standard	Totals
Pre-performance	Group 1 (30, 30)	Group 2 (29, 30)	Group 3 (28, 30)	N = 177 (87, 90)
Post-performance	Group 4 (29, 28)	Group 5 (31, 29)	Group 6 (28, 30)	N=175 (88, 87)
Totals	N = 117 (59, 58)	N = 119 (60, 59)	N = 116 (56, 60)	Total N = 352

Note: The numbers in parentheses represent the number of participants in the group or condition; the former refers to those who receive rewards and the latter to those who do not.

shows the breakdown of the groups and the number of participants in each group. Experimental groups were formed according to a 2 × 2 × 3 between-subjects design. Participants were divided into three conditions: whether or not the reward was received, the timing of reward determination (i.e., pre- or post-performance) and performance standards. Each group contained participants who exceeded their performance standard and those who did not. The categories for the timing of reward determination divided participants into two; pre-performance reward determination and post-performance reward determination. Those under the pre-performance reward condition were promised to receive additional rewards during the task. However, similar to the cancellation of a bonus, some of them did not get the promised reward. In addition, one of three possible performance standards was assigned to each participant: absolute, relative, or ambiguous.

3.4. Data Analysis

Our analysis consisted of the mediating effect analysis and moderating effect analysis. In our study, we used the PROCESS [77] method to observe the mediating effect, and the ANOVA method was used to analyze the moderating effect. All statistical analysis was conducted using SPSS 23.

A representative method was suggested by Baron and Kenny [78] to analyze the mediating effect. According to Baron and Kenny [78], for a mediating effect to be identified as significant, the following four conditions should be met: (1) the independent variable significantly affects the mediator, (2) the independent variable significantly affects the dependent variable, (3) the mediator significantly affects the dependent variable, and (4) the significance level of the independent variable on the dependent variable should be lower when both the independent variable and mediator are input together than the significance level of independent variable (1). The decreased level of significance represents the indirect effect through the mediator.

For this method, Zhao, Lynch Jr. and Chen [79] developed and extended Baron and Kenny’s method [78]. One extension of Zhao et al. [79] is the division of the mediation effect; full mediation, partial mediation, and competitive mediation. Full mediation occurs if there is only an indirect effect through the mediator, while partial mediation involves a direct effect of the dependent variable and an indirect effect through the mediator in the same direction. Competitive mediation contains both direct and indirect effects in opposite directions. Lastly, Zhao et al. [79] identified that the mediating effect can be identified to be significant only if the bootstrap analysis of the indirect effect is found to be significant.

Based on these arguments, Hayes [77] developed a macro method called PROCESS for analyzing the mediating effect. By using the PROCESS method, we can obtain results for direct and indirect effects and can easily verify the mediating effect. In conclusion, we used PROCESS [77] to analyze the mediating effect for this study.

We conducted an ANOVA to observe for moderation. There are several assumptions that must be satisfied when conducting an ANOVA: the independence of each condition, normally distributed dependent variables, and homogeneous variances of dependent variables under each condition [80]. Our data meet and satisfy all these assumptions.

Among the assumptions, we assumed that the independence condition was satisfied because we randomly distributed the experiment participants under diverse and reliable potential experiment candidates [72,73]. Before conducting the ANOVA, we conducted the statistical analysis to determine whether the other assumptions were satisfied, and the results showed that all assumptions were satisfied. The Kolmogorov-Smirnov test showed that the data were not significantly different from a normal distribution in all of the experimental groups ($p_{G1}= 0.199, p_{G2}= 0.200, p_{G3}=0.200, p_{G4}=0.200, p_{G5}=0.107, p_{G6}=0.067$; all p > 0.05). Also, Levene’s test showed that the data did not have significantly different variances of dependent variables in any of the experimental groups ($p=0.179$).

4. Results

Table 3. Descriptive Statistics of Variables.

Group		Minimum	Maximum	Mean	Standard Deviation	N
1	Performance 1	2	10	5.37	2.008	60
	Performance 2	2	10	5.6	1.607	60
	Self-efficacy 1	2.33	7	5.33	1.08	60
	Self-efficacy 2	2.17	7	5.08	1.26	60
	Intrinsic motivation 1	2.71	7	5.78	1.17	60
	Intrinsic motivation 2	2.29	7	5.44	1.29	60
2	Performance 1	2	10	5.53	1.86	59
	Performance 2	2	10	6.17	1.87	59
	Self-efficacy 1	2.67	7	5.35	0.84	59
	Self-efficacy 2	2.33	7	5.03	0.98	59
	Intrinsic motivation 1	1.71	7	5.75	1.19	59
	Intrinsic motivation 2	1.43	7	5.52	1.21	59
3	Performance 1	2	10	5.26	1.86	58
	Performance 2	1	10	5.28	2.04	58
	Self-efficacy 1	1.83	7	5.08	1.29	58
	Self-efficacy 2	1.67	7	4.74	1.51	58
	Intrinsic motivation 1	2.57	7	5.47	1.28	58
	Intrinsic motivation 2	1.71	7	5.30	1.43	58
4	Performance 1	1	10	5.32	2.27	57
	Performance 2	2	10	5.88	1.98	57
	Self-efficacy 1	2.5	7	5.38	0.98	57
	Self-efficacy 2	3	7	5.28	1.11	57
	Intrinsic motivation 1	2.43	7	5.55	1.21	57
	Intrinsic motivation 2	2.14	7	5.35	1.27	57
5	Performance 1	2	10	5.35	1.85	60
	Performance 2	2	9	5.63	1.87	60
	Self-efficacy 1	1.67	7	4.93	1.27	60
	Self-efficacy 2	1.83	7	4.82	1.25	60
	Intrinsic motivation 1	1.29	7	5.18	1.31	60
	Intrinsic motivation 2	1.43	7	5.15	1.32	60
6	Performance 1	2	10	5.74	1.74	58
	Performance 2	3	10	6.33	1.84	58
	Self-efficacy 1	2.33	7	5.32	1.13	58
	Self-efficacy 2	1.83	7	5.23	1.35	58
	Intrinsic motivation 1	2.14	7	5.68	1.06	58
	Intrinsic motivation 2	1.43	7	5.66	1.24	58
Total	Performance 1	1	10	5.43	1.93	352
	Performance 2	1	10	5.81	1.89	352
	Self-efficacy 1	1.67	7	5.23	1.11	352
	Self-efficacy 2	1.67	7	5.03	1.26	352
	Intrinsic motivation 1	1.29	7	5.57	1.22	352
	Intrinsic motivation 2	1.43	7	5.40	1.30	352

provides the descriptive information for performance, self-efficacy, and intrinsic motivation. Performance represents the scores of participants on the task. Variables labelled with the number 1 (“self-efficacy 1”, “intrinsic motivation 1”, and “performance 1”) were measured after the first task and before the second task to determine baseline values. Variables labelled with the number 2 (“self-efficacy 2”, “intrinsic motivation 2”, and “performance 2”) were measured after the second task and those were used for analysis.

Table 4. Correlations among Variables.

	1	2	3	4	5	6	7	8	9
1. Pay-for-performance
2. Self-efficacy 1	−0.078
3. Self-efficacy 2	0.309 **	0.615 **
4. Intrinsic motivation 1	−0.064	0.622 **	0.435 **
5. Intrinsic motivation 2	0.042	0.555 **	0.561 **	0.865 **
6. Performance 1	−0.058	0.183 **	0.090	0.030	0.006
7. Performance 2	−0.004	0.126 *	0.146 **	0.039	0.055	0.313 **
8. Timing of reward determination	0.011	−0.022	0.060	−0.082	−0.015	0.022	0.069
9. Performance standard 1	0.010	−0.060	−0.060	−0.062	−0.038	0.004	0.033	0.010
10. Performance standard 2	0.010	0.079	0.083	0.056	−0.003	−0.031	−0.029	−0.014	−0.504

Note: * p < 0.05, ** p < 0.01. Dummy variables: pay-for-performance (0: no reward, 1: receiving reward), timing of reward determination (0: pre-performance reward, 1: post-performance reward), performance standard 1 (1: absolute, 0: others), and performance standard 2 (1: relative, 0: others).

shows the correlations among all the variables of the total sample. As the timing of reward determination and performance standards are categorical variables, they were converted into dummy variables. The results show that pay-for-performance is significantly correlated with ‘self-efficacy 2’ but not with ‘intrinsic motivation 2’. It also shows that ‘self-efficacy 2’ and ‘intrinsic motivation 2’ is significantly correlated. The moderating variables, timing of reward determination and performance standards, do not have correlations with other variables. There are correlation tables that contain correlation coefficients among variables of each group in Appendix A. The correlation analysis results consistently shows that the correlation between ‘self-efficacy 2’ and ‘intrinsic motivation 2’ is significant when it was analyzed within each group.

Hypothesis 1 suggested a mediating effect of self-efficacy on the relationship between pay-for-performance and intrinsic motivation. The results showed the mediating effect was significant. The total effect size of pay-for-performance on intrinsic motivation was 0.25 ($\mathrm{SE}=0.07,\mathrm{t}=3.72,p<0.001)$, showing that pay-for-performance has positive effect on intrinsic motivation. The direct effect size of pay-for-performance on intrinsic motivation was 0.06 ($\mathrm{SE}=0.07,\mathrm{t}=0.92,p>0.05$) and this means the direct effect of pay-for-performance on intrinsic motivation is not significant. The indirect effect size, the difference between the total and direct effects, was 0.19 $\left(\mathrm{Boot}\mathrm{SE}=0.04,\mathrm{Boot}\mathrm{LL}95\%\mathrm{CI}=0.11,\mathrm{Boot}\mathrm{UL}95\%\mathrm{CI}=0.28\right)$ and it means the indirect effect through self-efficacy is significant. The results verified the mediation effect and also showed that it is a full mediation, which has no direct effect but only an indirect effect. Results of the normal theory-based Sobel test ($\mathrm{Z}=5.27,p<0.001)$ also supported the results of the other analysis on the mediating effect of self-efficacy on the relationship between pay-for-performance and intrinsic motivation.

Hypotheses 2 and 3 posit the moderating effects of the timing of reward determination and performance standards on the relationship between pay-for-performance and self-efficacy. The results showed that the interaction effect of the timing of reward determination was not significant ($\mathrm{F}\left(1,348\right)=0.145,\mathrm{df}=1,\mathrm{not}\mathrm{significant}$). On the other hand, the results showed a statistically significant interaction effect of performance standards on rewards ($\mathrm{F}\left(2,346\right)=3.62,p<0.05$), which shows that performance standard moderates the relationship between pay-for-performance and self-efficacy.

Table 5. ANOVA Results of Moderating Effect of Reward Determination Timing.

Source	Type III Sum of Squares	df	Mean Square	F	Effect Size	Sig.
Pay-for-performance	71.447	1	71.447	626.558	0.998	0.022
Timing of reward determination	2.771	1	2.771	25.136	0.963	0.130
Pay-for-performance * Timing of the determination	0.111	1	0.111	0.145	0.000	0.707

and

Table 6. ANOVA Results of Moderating Effect of Performance Standards.

Source	Type III Sum of Squares	df	Mean Square	F	Effect Size	Sig.
Pay-for-performance	71.462	1	71.462	25.416	0.927	0.037
Performance standard	0.573	2	0.287	0.102	0.092	0.908
Pay-for-performance * Performance Standard	5.649	2	2.824	3.620	0.021	0.028

represent the ANOVA results of the moderating effects.

Figure 3 represents the mean values of self-efficacy classified as performance standards and shows the moderating effect of performance standards on the relationship between pay-for-performance and self-efficacy. It shows that the relative and ambiguous performance standards strengthen the relationship between pay-for-performance and self-efficacy.

We further analyzed whether there was an interaction effect between the two performance standards. As there are three performance standards—absolute, relative, ambiguous, three combinations could be made; (1) absolute and relative standards, (2) absolute and ambiguous standards, and (3) relative and ambiguous standards. The interaction effects were analyzed with a sample with two different performance standards. The results showed a significant interaction effect was found between the absolute standard and relative standard ($\mathrm{F}=6.513,p<0.05)$, and between the absolute standard and ambiguous standard ($\mathrm{F}=4.453,p<0.05)$. However, there was no significant interaction effect between the relative standard and ambiguous standard ($\mathrm{F}=0.018,p>0.05)$. Therefore, we can conclude that the moderating effect is significant, and both the relative and ambiguous standards have different interaction effects in comparison to the absolute standard.

5. Discussion

This paper analyzed the mediating role of self-efficacy and the moderating effects of the timing of reward determination and performance standards. For hypothesis testing, we gathered participants using the MTurk platform, and an online experiment was conducted. Participants completed certain tasks, and a questionnaire was administered to measure the variables. The results of the online experiment revealed that the mediating effect of self-efficacy between pay-for-performance was significant, the moderating effect of timing of reward determination between pay-for-performance and self-efficacy was not significant, and the moderating effect of performance standard between pay-for-performance and self-efficacy was significant.

The first hypothesis regarding the mediating effect of self-efficacy was supported. It can be understood in the context of the self-determination theory. The results showed the positive outcomes for competency increased self-efficacy [35,48,49] and positive competence perception increased intrinsic motivation [27,53]. This is an analysis of the effect on intrinsic motivation including the variables affecting self-efficacy. This is consistent with the results of previous studies that identified the mediating effect of perceived competence between pay-for-performance and intrinsic motivation of self-determination theory [30,52].

The second hypothesis was not supported. We assumed that pre-determined rewards would reduce the positive effects of pay-for-performance. However, it seems that the conditions of our experiment were not sufficient to create and cause such effects and it was difficult to find out the reason. This may be because the timing of reward determination is not strongly related to feelings of stress or risk of losing, or because participants did not recognize the differences between the various types of rewards. Moreover, it could also be because participants do not care very much about the timing of reward determination, but only about whether they received their rewards or not.

The third hypothesis concerning the moderating effect of performance standards was supported. Those who could not receive a pay-for-performance reward have stronger negative effects under relative and ambiguous standards. They would experience greater disappointment because they did not achieve their goal, especially when they performed poorly in comparison to others [66,67] or even when the task seemed meaningless [61,62]. This result proposes a method to reduce the negative effects of pay-for-performance system on self-efficacy; the use of the absolute standard has less negative effects on employees. The findings are somewhat consistent with Eisenberger et al. [6] in that pay-for-performance had different effects on intrinsic motivation by different performance standards.

5.1. Implications

This study has implications for practitioners. The first implication for practitioners is in relation to employees whose rewards are not administered using a pay-for-performance system. The results of this paper showed that those who failed to receive any reward at all had much lower self-efficacy and intrinsic motivation. Lower intrinsic motivation could then be related to lower performance, which could lead to another failure to meet or exceed the performance standards. In other words, a person’s failure to meet performance standards could lead to a vicious downward cycle in performance. One of many methods to prevent this negative cycle is related to group composition and is represented in this study by dividing a large group into multiple groups depending on the levels of competency. Using this method, organizations can utilize several reward standards, providing different rewards depending on the difficulty or importance of a task. This could lead to more employees meeting performance standards, help employees feel successful, and increase their self-efficacy and intrinsic motivation.

The second implication is that the recommendation of the absolute standard for performance measurement in comparison to other standards. In practice, many organizations use relative performance standards without much consideration of the benefits and drawbacks. People who fail to receive their rewards under relative or ambiguous performance standards may experience a drastic drop in self-efficacy. Also, a relative performance standard might cause interpersonal deviance by those who were stressed because of social comparison and competition [81]. Therefore, a relative or an ambiguous performance standard should be used with cautious deliberation.

There are several theoretical contributions in this paper. First, this study contributed to elucidate a new concept, the timing of reward determination. Despite the lack of statistical significance of the experimental results, it is still important to investigate the timing at which rewards are determined. Although rewards are typically determined after the completion of a task, some rewards are determined beforehand. In future research, systems involving different rewards given at different times may be investigated.

The second contribution of our study is to support the self-determination theory. Recipients in the pay-for-performance group reported higher levels of self-efficacy. This relationship was associated with a higher level of intrinsic motivation, as self-determination theory purports [27]. Therefore, self-efficacy had positive effects on intrinsic motivation. This study showed that pay-for-performance increased intrinsic motivation through the process of improved self-efficacy.

The third contribution of our findings is related to the role of performance standards as a moderating factor. This paper verified the moderating effect of performance standards quantitatively. The contribution of our study is that we developed the different effects of performance standards [6] into a moderating effect between pay-for-performance and self-efficacy, which is a one-step preceding variable. The result is also meaningful in that we suggested a way to reduce the negative effects of failure on pay-for-performance. While many studies tend to focus on the positive effects of reward [28,29,30,82,83,84], research on reducing negative effects of failure is also important, and few studies focus on reducing these negative effects [85,86].

5.2. Limitations and Further Study

First, there are some limitations of an online experiment that are hard to overcome. Despite our efforts to maintain a high level of internal validity, the methods of measurement were limited in comparison with a laboratory experiment. Laboratory experiments could incorporate multiple ways of measuring intrinsic motivation through self-reported interest and free-time choice behaviors [6,22,52]. It was difficult to find a reliable way to measure free-time choice behavior. We suggest further research to utilize such various methodologies, including field experiments, surveys, and qualitative approaches to support and generalize our findings, and to include diverse control variables such as gender, age, and socioeconomic levels.

Second, this paper proposed the concept of the timing of reward determination (pre-performance and post-performance), but we were unable to find a significant effect of its moderation effect. In the future, this variable could be manipulated more carefully to better represent how rewards are offered in actual organizations. Also, qualitative approaches could be used to investigate feelings and responses at different periods of reward determination. For example, in-depth interviews with professional athletes who have minus-option clauses or clawback conditions in their contracts could improve our understanding of the pre-performance reward determination system.

Finally, future research can analyze the effects of performance standards. Performance standards may affect other relations between variables. Therefore, it is recommended to investigate the role of performance standards as a moderating factor between pay-for-performance and other outcome variables such as self-esteem [87]. Also, future research can be conducted with other variables related to the perception of being controlled and perceived competence. Measuring and incorporating the perception of being controlled in different reward systems may shed light on the conflicting results in the literature on rewards. Just as reward methods could affect both autonomy and perceived competence simultaneously, researchers can consider other variables that simultaneously affect both aspects of self-determination. This is beneficial for practitioners and researchers to maintain a sustainable management environment. For example, both autonomy and feedback can be studied simultaneously with the job characteristics model [88,89] and the inconsistencies between individual performance and team performance [90].