An Internet-based program for depression using activity and physiological sensors: efficacy, expectations, satisfaction, and ease of use

Introduction

Depression is already estimated to be the second leading cause of disability worldwide.¹ There is evidence of a significant relationship between depression, stress, and emotion regulation ability.² In addition, people when depressed often use maladaptive emotion regulation strategies and have low coping behavior.³ Therefore, to prevent depression, it is important to develop strategies to promote adaptive emotion regulation strategies and resilience in people exposed to high levels of stress. Furthermore, we need to develop better tools to identify people at risk and design effective prevention programs.⁴ In addition, it is important to reach everyone in need. Less than 50% of people with emotional disorders receive appropriate treatment, and many patients are reluctant or find difficult to get help either due to its cost, the time required for its application, and/or the lack of well-trained professionals.⁵

Computerized cognitive behavioral therapy (CCBT) and the use of the Internet to support treatment implementation have been shown to be powerful means for its effective deployment in providing general mental health care helping to deliver interventions that promote healthy lifestyle and behaviors.^6–8

Systematic reviews and meta-analyses support the efficacy of Internet-based CCBT for depression, anxiety disorders, chronic tension/migraine, trauma, insomnia, obesity, complicated grief, headache, and eating disorders.^9–13

Research on this topic is growing at an impressive rate, but little is known about the expectations, acceptability, or satisfaction with CCBT.^14–16 These variables can inform us about the feasibility of the intervention, which can help to optimize CCBT effectiveness.^17,18 Although it is important to provide data about acceptability and opinions,^19–23 most studies do not measure or report acceptability directly, but instead offer indirect indicators, such as take-up rates and patient dropout rates.¹⁴

Moreover, CCBT interventions may be improved by incorporating tools to collect useful information about progress, improvement, or stagnation throughout treatment and provide feedback to participants and clinicians in order to improve the effects of the intervention. The use of sensors, biosensors, or other technological tools within personalized health-care systems is a growing trend in the application of treatments.^24,25 These emerging tools are fed by continuous technological advances, becoming wearable systems that can provide powerful support in health care for treating and preventing several psychological problems.²⁶ Although there are a lot of mobile technologies for enhancing psychological wellness, there is little evidence supporting their efficacy, and even less regarding their acceptability.^27,28 In the past few years, several European projects have focused on these tools, demonstrating the importance of this issue.^2,29–33 All these projects have used noninvasive sensors and self-monitoring tools to enhance diagnosis, prevention, or treatment.³⁴ As for sensors, it becomes important to ask not only about expectations and satisfaction, but also about the ease of use for participants. To the best of our knowledge, however, there are no published studies on their utility and acceptability. Moreover, to date, no study has examined variables such as participants’ expectations about them and satisfaction with an Internet self-administered CCBT program that includes the use of sensors. Furthermore, few studies have assessed usability issues in CCBT programs.³⁵ So far, there are no studies on CCBT with sensors that directly assess these variables (expectations, satisfaction, and ease of use) to determine the level of acceptability by users.

Taking all of this into consideration, we developed a self-administered, multimedia and interactive Internet-based program. Its objective is to allow the individual to learn and practice adaptive ways to cope with depression and daily problems. It can be used with or without physiological and activity sensors.

The first aim of this study is to revise the efficacy of this CCBT program in people at risk of depression (unemployed men). The second aim is to analyze the participants’ expectations, satisfactions, and perceptions of ease of use of this CCBT program, both with and without the use of physiological and activity sensors. We hypothesize that the two intervention groups (with and without sensors) will improve significantly more than the control group on the clinical variables. Furthermore, taking into account that sensors provide additional information and feedback (electrocardiograhpy [ECG] for heart rate and motion, actigraphy [ACT] for ambulatory activity, electroencephalography [EEG] to derive affective status – sensitivity, and sleep quality using ECG and ACT), we hypothesize that the improvements on the clinical variables will be higher in the group with sensors than in the group without them. In addition, we expect all participants will score high on expectations, satisfaction, and ease of use of the CCBT program. We hypothesize that in both intervention groups (with and without sensors) the scores will be high, but we cannot predict whether they will be higher in one group than in the other. Although the group with sensors receives more feedback, their use might be cumbersome.

Material and methods

Intervention program

“Smiling is Fun” is an Internet-delivered, multimedia (video, images, etc), interactive, self-administered CCBT program for the prevention and treatment of depression and adjustment disorders. It allows the individual to learn and practice adaptive ways to cope with daily problems and depressive symptoms. It is designed for optimal use on a PC, but it can also be used on a tablet. “Smiling is Fun” combines the most effective psychological procedures for stress management with strategies to promote coping capacity, emotion regulation, and resilience. It is based on classic CBT techniques, such as behavioral activation,^36,37 and it also includes other psychological strategies to improve positive mood.³⁸ The intervention protocol consists of eight interactive modules. Each module includes exercises for practicing each technique. The program has two addition modules. The first one, named “Home module” (Figure 1), explains what “Smiling is Fun” is, its goal, who can benefit from it, terms and conditions, and who we are. The second one, “Welcome module,” informs users about the content of each module and explains how to benefit from it.

Figure 1 Home module.

The content of the eight modules is designed to break the vicious cycle of stress, isolation, depressive or anxiety symptoms, poor coping, and poorer functioning (self-efficacy). Each of the modules has the specific objectives described as follows: 1) “Motivation for change,” in which the advantages and disadvantages of change are analyzed and the importance of motivation for change is emphasized; 2) “Understanding emotional problems,” in which information regarding the recognition and understanding of emotional problems is provided; 3) “Learning to move on,” motivates the user to get involved in meaningful activities and to be engaged in their life; 4) “Learning to be flexible,” teaches the user how to consider different alternatives in order to be more flexible in interpreting situations; 5) “Learning to enjoy,” explains the importance of positive emotions and ways to promote them; 6) “Learning to live,” teaches the user how to identify their psychological strengths and the importance of carrying out activities based on their goals and values in life; 7) “Living and learning” focuses on developing and putting in practice the psychological strengths identified; and 8) “From now on, what else…?,” strengthens what has been learned during the program and emphasizes the importance of continuing the practice.

“Smiling is Fun” has three complementary transversal tools that accompany the user throughout all intervention program (IP). These tools are in the main menu of the IP and the participants can use them every day, when they access the program. 1) “Activity report,” it is a tool where the person is asked for his/her coping ability, mood state, and stress, on a 0–10 scale. Then, the user is asked about the degree of satisfaction with each activity performed during the past 24 hours and to what extent these activities are related to his/her own goals and values in life. Furthermore, the user is asked about the percentage of the day that he/she has been active or involved in his/her life for the last 24 hours (Figure 2). With all this information, the Activity Report provides graphical feedback to the users, showing the relationship between their coping ability, stress, and mood and the activities performed, and the benefits of being active. 2) “The calendar” is a tool that provides information about homework and tasks already achieved (listing the number of fulfilled tasks with a symbol represented by a yellow sun), reminding the user of those still outstanding, and it allows the users to know what point they stand throughout the program. Moreover, it marks with a green tick those days the users have been active in the Activity Report (Figure 3). 3) “How am I?,” which offers a set of graphs and feedback to chart the user’s progress, including activity level, emotional distress (anxiety and sadness), positive emotionality (active, enthusiastic, energetic, etc), negative emotionality (angry, fearful, stressed, tense, moody, etc), and information from sensors.

Figure 2 Activity report.

Figure 3 Calendar.

Sensors

“Smiling is Fun” includes several physiological and activity sensors: EEG and ECG sensors to detect subjects’ cognitive and physiological state, and ACT to characterize their physical activity (Figure 4). Participants can receive feedback from the sensors in graphic form.

Figure 4 From left to right, the heart rate and heart rate variability sensor worn on the neck, the activity sensor worn on the ankle, and the brain wave sensor. Note: The red arrow indicates the correct position of the ACT sensor. The circular washer must be placed in the direction of the arrow. Abbreviations: ECG, electrocardiography; ACT, actigraphy; EEG, electroencephalography.

The ECG sensor records heart rate, heart rate variability, and physical motion. This sensor provides information that may be an indicator of stress based on heart waveform variability.^39,40 The ECG waveform is well known, and under normal conditions it behaves in a well-defined and very predictable manner.

The ECG sensor is used continuously for 24 hours. The information is stored in the sensors’ FLASH memory, which has a limited capacity of 27 hours. Therefore, a routine is defined whereby the user has to download data once a day. It works as follows: remove the ECG sensor, put it close to the home PC (explained in the Home PC platform section), run the download program on the home PC, put the sensor on the battery charger, get a second fully charged sensor, run the home PC software to activate it to start recording, and put it again on the body.

The ACT sensor records several physical motion parameters: resting, walking, jogging, running, stairs, and cycling. More specifically, it monitors the frequency and intensity of regular exercise, sports activity, and sleep patterns. Evidence shows that regular physical exercise is a moderator of mood and prevents and improves depression.⁴¹ The ACT sensor has been designed to be worn on the lower leg. It samples the three axis accelerations of the leg, and implements Hidden Markov Model-based classifiers that recognize activity and estimate total energy expenditure. The procedure follows the same routine as the one described for ECG. When users download data and activate the ECG sensor, they repeat the routine with the ACT sensor, which has also been worn for 24 hours.

The EEG sensor records the electrical potential gradient oscillating around the brain.⁴² It detects anterior brain activities related to emotional stress, and it has been used to detect changes in the behavior, sleep anomalies, and mental acuity related to depression.⁴³ Alpha asymmetry and C0 complexity are of particular interest, as well as Beta/Alpha and Alpha/Theta power band ratios, as they can reflect a person’s mood. This sensor records the user’s EEG signals over periods of approximately 90 seconds to obtain sufficient data for the algorithms. Additionally, the sensor provides a means to grade the quality of contact between the electrodes and the skin on the forehead, so that the contact quality is sufficient for the raw data to be useful. The EEG recording session lasts for 3 minutes. The user locates the sensor on the forehead and performs several actions as defined by the home PC application, which stores data automatically. The participants were free to perform the EEG task every day, but they had to do it at least once a week during the entire trial (6–10 weeks).

Home PC platform

A home PC platform has been designed to enable the users to collect, download, and store information using the sensors. The home PC has a “look and feel” wizard that guides the users through the different steps. Participants receive a training session in the laboratory to practice all the processes.

Research design

A randomized controlled trial compared three experimental conditions: 1) IP (N=22): participants had access to the CCBT IP without sensors. 2) Intervention program plus sensors (IP + S, N=19): participants had access to the CCBT IP, and they used sensors. Participants in both intervention groups could use the three transversal tools explained in the Intervention program section. However, only the participants in the IP + S group had access to the see the graphical feedback regarding the sensor information, provided by the “How am I” tool. 3) Control condition (C, N=19): the participants answered the pretreatment questionnaires, waited 10 weeks, and then answered the posttreatment and follow-up questionnaires.

Sample

Unemployed men were selected because they are considered a high-risk population. As the Analyses of Epidemiologic Catchment Area panel data reveal, the unemployed have over twice the risk of becoming clinically depressed as those who continue to be employed.⁴⁴ The sample consisted of 60 unemployed males. The mean age was 32.35 (SD =9.55). The mean duration of unemployment was 15.30 months. Regarding marital status, the majority of the participants (66.7%) were single, 30% were married or had a partner, and 3.3% were divorced. Regarding the educational level, 8 participants (13.3%) had a high-school diploma or less, 17 participants (28.3%) had some college, and 35 participants (58.3%) had a college degree or graduate degree. On the Beck Depression Inventory second edition (BDI-II; possible range of scores: 0–63), participants had a mean preassessment score of 4.82 (SD =4.61).

Inclusion and exclusion criteria

Inclusion criteria were as follows: unemployed men suffering stress due to financial concerns and family obligations, aged between 18 and 65, who agreed to participate in the study and signed an informed consent, and who had access to a computer with Internet connection. Exclusion criteria included proneness to skin allergies that might be exacerbated by wearing stick-on sensors, personal or first-degree relative history of depression/psychosis, BDI II ≥19, epilepsy, heart conditions (could interfere with ECG measurement), daily use of recreational drugs (eg, cannabis), regular use of sleep medication, regular use of medication for heart, and/or cardiovascular conditions (eg, antihypertensive).

Measures

Mini International Neuropsychiatric Interview (MINI)

This is a structured diagnostic psychiatric interview that yields key diagnoses from the Diagnostic and Statistical Manual of Mental Disorders, fourth edition and the International Classification of Diseases. The MINI can be administered by clinical interviewers who need only brief training. The MINI has been translated into Spanish and validated.⁴⁵

Beck Depression Inventory II (BDI II)

The BDI II is one of the most widely used questionnaires for evaluating depression severity in clinical trials. The instrument has shown good internal consistency (Cronbach’s α of 0.76–0.95) and test–retest reliability of about 0.8.⁴⁶ The Spanish version of this instrument has also shown high internal consistency for both the general population (Cronbach’s α of 0.87) and the clinical population (Cronbach’s α of 0.89).⁴⁷

Overall Anxiety Severity and Impairment Scale (OASIS)

The OASIS consists of five items that measure the frequency and severity of anxiety, as well as the level of avoidance, work school/home interference, and social interference associated with anxiety. A psychometric analysis of the OASIS scale found good internal consistency (Cronbach’s α=0.80), test–retest reliability (k=5.82), and convergent validity.⁴⁸

Positive and Negative Affect Scale (PANAS)

The PANAS consists of 20 items that evaluate two independent dimensions: positive affect (PA) and negative affect (NA). The range for each scale (10 items on each) is from 10 to 50. The Spanish version has demonstrated high internal consistency (0.89–0.91 for PA and NA, respectively, in women, and 0.87 and 0.89 for PA and NA, respectively, in men) in college students.⁴⁹

Perceived Stress Scale (PSS)

The PSS is a 14-item self-report questionnaire that assesses the perceived stress associated to recent life situations. The Spanish validation of this scale has an internal consistency of 0.86.⁵⁰

Expectation of Intervention Scale and Opinion of Intervention Scales

These questionnaires were adapted from Borkovec and Nau.⁵¹ Each scale consists of five items, rated from 0 (“strongly disagree”) to 10 (“strongly agree”), which cover how logical the intervention seemed, to what extent it satisfied the participant, whether it could be used to treat other psychological problems, and its usefulness for the participant’s specific problem. The expectation scale is applied once the intervention rationale has been explained, at the end of the second module. Its aim is to measure the participant’s subjective expectation of the intervention. The opinion scale is administered when the participant has finished using the program, and its aim is to assess satisfaction with this intervention. Our group has used these questionnaires in several studies.¹⁹

System Usability Scale (SUS)

The SUS scale was developed by Brooke⁵² to assess the usability of a given product or service. It is a simple, 10-item Likert scale that provides a global view of the subjective assessment of usability. The SUS has proven to be a valuable and robust tool for assessing the quality of a wide range of user interfaces. It has several characteristics that make it a good choice for general practitioners: it is technology neutral, allowing the evaluation of a wide range of interface technologies; it is quick and easy to use for participants and administrators; it can provide a single score; and it is easy to understand.⁵³

Figure 5 shows the mapping of the adjective ratings corresponding to acceptability ranges and quartiles.⁵³

Figure 5 A comparison of mean SUS scores by quartile, adjective ratings and the acceptability of the overall SUS score. Abbreviation: SUS, System Usability Scale.

Statistics and data analysis

Group differences in demographic data and pretreatment measures were analyzed with one-way analysis of variance (ANOVA) and chi-square tests to find out if there were significant differences among the groups. For the clinical variables, all posttreatment analyses involved a conservative intention-to-treat (ITT) design, where missing data were addressed by carrying forward the last available data (ie, last-observation-carried forward model). To test the treatment efficacy and the differential efficacy of the three experimental conditions, mixed ANOVAs with repeated measures were used to compare the time effect on the measures (pre- to posttreatment, and pre, post, 3 month follow-up) and the time interactions among the conditions. The ANOVAs were followed by Sidak’s post hoc tests whenever it was needed. Because we found differences in the three experimental groups at pretreatment in the BDI-II variable, it was analyzed using univariate analyses of covariance (ANCOVAs), assigning pretreatment scores as the covariate. Effect sizes (Cohen’s d) were calculated for within- and between-group changes, based on the pooled standard deviation. The expectation, satisfaction, and usability scores were also analyzed using mixed ANOVA with repeated measures. All statistical analyses were conducted using IBM SPSS Statistics 20 (IBM Corporation, Armonk, NY, USA).

Participant’s flow

Regarding the flow of the participants throughout the investigation process, Figure 6 shows that 622 people were interested in the study. The most important reason for exclusion was sex of the participant (354 women were excluded because only unemployed men were studied). We called all men (n=286) by telephone (prescreening). About 189 participants were excluded after this phone interview (53 did not answer the phone call, 52 did not meet the inclusion criteria, 4 were not interested in the study, and 60 did not live in Castellón or Valencia) and 79 men were invited to participate and performed the face-to-face interview. Finally, 69 participants were included and randomly assigned to the three experimental conditions: IP + S, n=20; IP, n=25; C, n=24, and 60 participants did the pretrial assessment: IP + S, n=19; IP, n=22; C, n=19.

Figure 6 Flow chart. Abbreviations: IP, intervention program group; IP + S, intervention program plus sensors group; C, control group; BDI-II, Beck Depression Inventory II.

During the IP, there were three dropouts, and all of them in the IP group. In group C, there were two dropouts at postassessment.

At the 3-month follow-up, there were two dropouts in the C group.

Procedure

The study was approved by the ethics committee at Jaume I University. Participants were recruited through direct contact with local job centers, advertisements in the media and Internet, and announcements to the university community. The advertisements described the research as a study on stress management and depression prevention, and as an opportunity for personal growth specially designed for unemployed people. The advertisements also stated that prospective participants would receive 15 Euros to cover travel expenses any time they had to come to the laboratory. Finally, the advertisements outlined the exclusion criteria and the final incentive offered to participants: the possibility of keeping the netbook they used during the study if they were in the group that wore sensors.

All the volunteers had to answer an online questionnaire to see if they met the minimum requirements for joining the study (unemployed men). Women were excluded. Next, we prescreened possible participants by phone to filter other important criteria: personal or first-degree relative history of depression, epilepsy, heart conditions, daily use of recreational drugs, sleep medication, or medication for heart and/or cardiovascular conditions (eg, antihypertensive).

Participants who passed the prescreening by phone were given a date for an initial face-to-face interview (screening) in our laboratory, where a research team member explained the study. Participants interested in participating signed an informed consent form. Subsequently, the investigator asked for demographic information, evaluated the presence of depression with the MINI and the BDI-II, and gathered other information about the inclusion and exclusion criteria. The selected participants were then randomly assigned to the three experimental conditions.

Finally, 60 participants took part in the study and were assigned to each of the three experimental conditions: IP =22, IP + S =19, C =19. All the participants completed baseline measures, including questionnaires. Only after filling in the assessment protocol were participants told to which group they had been allocated. This marked the beginning of the use of the CCBT program, which lasted between 6 and 10 weeks (participants were asked to complete a fixed number of CCBT recording sessions, but set their own pace for the number of sessions per week).

At the end of the second module, the participants in the intervention groups completed the scale on their expectations about this program (see “Measures” section). Both when finishing the program and at the 3-month follow-up, the participants had to again answer the assessment protocol. Furthermore, the participants in both intervention groups were asked about their level of satisfaction with the program and about its ease of use.

Results

Pretreatment data

The results indicated that there were no significant differences between the three groups before treatment on any of these variables: marital status (χ² [2] =3.332; P=0.504), educational level (χ² [2] =8.963; P=0.062), and age (F_(2,57)=0.701; P=0.500).

Regarding the clinical characteristics (Table 1), the data indicate that there were no significant differences among the three groups on any of the variables before treatment, with the exception of BDI-II (F_(2,57)=3.404, P=0.040). Post hoc analysis (Tukey HSD) revealed that the IP group scored higher than the other two conditions (IP + S and C).

Table 1 Clinical characteristics of participants at preassessment Abbreviations: Mea, measures; Gr, group; N, number of participants; M, mean; SD, standard deviation; IP, intervention program group; IP + S, intervention program plus sensors group; C, control group; BDI II, Beck Depression Inventory II; OA, Overall Anxiety Severity and Impairment Scale; PA+, Positive Affect Scale; PA−, Negative Affect Scale; PSS, Perceived Stress Scale.

Differential effectiveness of three conditions: change in outcomes pretreatment, posttreatment, and follow-up

Means and standard deviations and effect sizes for all outcome measures are listed in Table 2.

Table 2 Mean, SD, and effect sizes for the outcome measures at pretreatment, posttreatment, and 3-month follow-up (ITT analysis) Notes: aPre-F-up (3m). bIP + S vs IP; IP + S vs C; IP vs C. Abbreviations: Mea, measures; Gr, group; N, number of participants; M, mean; SD, standard deviation; Pretreat, pretreatment assessment; Posttreat, posttreatment assessment; F-up (3m), 3-month follow-up assessment; IP, intervention program group; IP + S, intervention program plus sensors group; C, control group; BDI II, Beck Depression Inventory II; OA, Overall Anxiety Severity and Impairment Scale; PA+, Positive Affect Scale; PA−, Negative Affect Scale; PSS, Perceived Stress Scale.

The analyses revealed a significant time effect on all measures: BDI-II (F_(2,114)=10.63, P<0.001), OASIS (F_(2,114)=3.24, P=0.043), PANAS + (F_(2,114)=12.99, P<0.001), PANAS − (F_(2,114)=7.36, P=0.001), and PSS (F_(2,144)=7.32, P=0.001).

More interesting, the analysis revealed a significant interaction effect on all measures, except OASIS.

Regarding the BDI-II, an ANCOVA was performed to control for differences found at pretreatment. Covariate analysis indicated that the groups did not differ significantly at posttreatment (F_(2,56)=0.426, P=0.655).

Regarding PANAS PA scores (Time × group (F_(4,114)=2.649, P=0.037), the IP + S group improved significantly pre- to posttreatment (P<0.001) and pretreatment to follow-up (P=0.001), and improvements obtained at posttreatment were maintained at follow-up. The IP group did not improve significantly pre- to posttreatment (P=0.114), but it improved significantly pretreatment to follow-up (P=0.050). The C group did not improve pre- to posttreatment (P=0.990) or pretreatment to follow-up (P=0.763). Regarding PANAS NA scores (Time × group (F_(4,114)=2.503, P=0.046), the IP + S group improved significantly pre- to posttreatment (P=0.029) and pretreatment to follow-up (P=0.024), and improvements obtained at posttreatment were maintained at follow-up. In the IP group, the participants also improved significantly pre- to posttreatment (P=0.002) and pretreatment to follow-up (P=0.046), and improvements obtained at posttreatment were maintained at follow-up. The C group did not improve pre- to posttreatment (P=0.978) or pretreatment to follow-up (P=1).

As for perceived stress (PPS scores) (Time × group (F_(4,114)=2.536, P=0.044), the IP + S group improved significantly pre- to posttreatment (P=0.002), and improvements obtained at posttreatment were maintained at follow-up. The IP group did not improve significantly pre- to posttreatment (P=0.337), but pretreatment to follow-up, the improvement was marginally significant (P=0.052). The C group did not improve pre- to posttreatment (P=0.999) or pretreatment to follow-up (P=0.996).

Regarding OASIS, the ANOVAs did not reveal a significant interaction effect (F_(4,114)=0.426, P=0.655). However, the mean scores and the effect sizes suggest a benefit for the intervention groups. Their mood and anxiety improved more than in the control group pre- to posttreatment and pretreatment to follow up. In the C group, we observed only a minimal change.

Expectations and satisfaction

Table 3 lists the results for both intervention groups. As can be observed, before the treatment the scores were very high. The ANOVA did not find significant differences between the two groups on any measures: treatment logic (F_(1,36)=0.009, P=0.925), treatment satisfaction (F_(1,36)=0.056, P=0.815), recommending the treatment to others (F_(1,36)=0.006, P=0.941), usefulness of the treatment for other psychological problems (F_(1,36)=0.091, P=0.765), and usefulness of the treatment for the patient (F_(1,36)=0.018, P=0.895).

Table 3 Means and standard deviations for expectations (pretrial) and satisfaction (posttrial) Abbreviations: N, Number of participants; M, mean; SD, standard deviation; IP, intervention program group; IP + S, intervention program plus sensors group.

After using the program, scores of participants on satisfaction were very high. The ANOVA did not show any differences between groups on treatment logic (F_(1,36)=0.012, P=0.914), treatment satisfaction (F_(1,36)=0.268, P=0.608), recommending the treatment to others (F_(1,36)=0.002, P=0.963), usefulness of the treatment for other psychological problems (F_(1,36)=0.019, P=0.890), or usefulness of the treatment for the patient (F_(1,36)=0.385, P=0.539).

See the video of a participant giving his opinion about the CCBT program (Video S1).

Ease of use

Table 4 provides the results of the SUS overall score and items. ANOVAs (Table 5) did not show any significant effect, indicating that all participants found the system very usable and acceptable, and that wearing sensors did not affect a good rating in terms of the system’s usability and acceptability (see the video of a participant giving his opinion about the sensors (Video S2).

Table 4 Means and standard deviations for the SUS Abbreviations: N, number of participants; M, mean; SD, standard deviation; IP, intervention program group; IP + S, intervention program plus sensors group; SUS, System Usability Scale.

Table 5 ANOVA results for overall scoring and single items in the SUS Abbreviations: ANOVA, analysis of variance; SUS, System Usability Scale.

Discussion

The first objective of this work was to analyze the efficacy of the self-administered treatment via the Internet (with and without sensors) in improving the different clinical variables evaluated in a population under prolonged stress (unemployed men) at risk of developing depression.

The results show that the two intervention groups improved more than the control group on their clinical variables, and that the improvements were greater posttreatment in the group that used sensors than in the group that did not use them. Considering the effect sizes, the IP + S group obtained medium effect sizes in all the variables, with the exception of the OASIS (small effect size). The IP group obtained a medium effect size on the BDI-II, and small effect sizes on the other measures. The effect sizes in the C group were minimal.

Furthermore, the improvements on some of these variables were maintained at the 3-month follow-up.

It is important to mention that participants were a population at risk of depression (unemployed men), but they were healthy volunteers with no psychological problems. Despite their long-term unemployment situation, their score on the BDI-II (4.82, SD =4.61) and other measures at pretreatment were low. Even so, depression and other clinical variables improved more in both intervention groups, compared to the control group. Our results are in line with those presented in systematic reviews and meta-analyses showing that computerized and Internet-based treatments are effective interventions.^12,54–57

The other objective of this study was to analyze the expectations, satisfaction, and perceived ease of use by the participants in both intervention groups. The participants in this study reported positive expectations of the program, coinciding with other studies.^9,14 The expectations were high for both groups, regardless of the use of sensors, suggesting that participants found the treatment very logical, useful even for other psychological problems, and they felt confident about recommending it to a friend. Participants’ opinions and of satisfaction after using the program were also high, with no statistically significant differences between both intervention groups.

The results obtained regarding the ease of use would place the IP between the third and fourth quartile, among the best products and close to truly superior products (Figure 5). It should be noted that the overall scores on usability (SUS questionnaire) for both groups were very positive. High ratings on satisfaction and ease of use have been obtained, indicating that “Smiling is Fun” with and without sensors is considered very usable. All these results support our first hypothesis, as they were positive in both intervention groups. The results highlight that all the participants, regardless of grouping, accepted the program very readily, whether they used sensors or not.

Our results are in line with the literature that indicates that people treated with CCBT Internet-based programs report high levels of acceptability, positive expectations, and high satisfaction (of those completing the treatment).^9,13 Very few studies include instruments specifically designed to determine the participants’ attitude toward CCBT.^21,23 As previously mentioned, most studies offer limited information on indicators of acceptability (recruitment, take-up, and dropout rates).¹⁴ Similarly, although there is an increasing use of noninvasive sensors and self-monitoring tools,³⁴ we have no data on participants’ satisfaction and usability, specifically in the literature that use behavioral and physiological sensors in combination with CCBT. In our study, the participants seem to accept these technological innovations readily, and they are willing to use different kinds of sensors.

In conclusion, the results of this study show that the use of Internet-based psychological treatments can result in good participant evaluation and acceptability.

Sensors can provide participants and clinicians with very useful feedback and information (especially for adapting and tailoring the treatment to the needs of each user), but they also have drawbacks. First, in many cases they are still not very robust prototypes and can result in failures or problems when used. Second, they are not yet well integrated into the CCBT systems (in our case, the participants had to use two technological platforms), and this requires extra work. Despite all these potential difficulties, however, the observed trend is that more and more people are using different sensors and specific applications for health promotion.³⁴ These findings open the door to the use of CCBT plus sensors. We think this is the beginning of a new era in the psychological intervention field. The use of CCBT Internet-delivered programs and sensors is innovative, and further developments in the field are inevitable, such as the union between virtual reality techniques and the Internet; the use of avatars that guide, motivate, and help participants throughout the treatment process; the inclusion of serious games designed for specific people with specific difficulties; and the possibility of delivering important parts of the treatment using mobile devices. Information and communication technologies (ICTs) have arrived on the psychological treatment scene, and they are here to stay.

We need to perform additional research to make the ICTs use easier and define who can benefit from what type of application or what type of sensor.

We are aware that this study has limitations, such as the sample size and difficulties related to external validity because all the participants were men. Another limitation was the availability of sensors. They were prototypes, and we had a maximum of 25 of each type, which determined the small sample size. It is possible that the small sample size influences the results obtained regarding the nonsignificant differences between intervention groups. Furthermore, the fact that the sensors were prototypes may have influenced the results, and more solidly built versions could produce more definitive results. The opportunity to keep the netbooks could be a quite important reward for participants in the IP + S group. It would have been interesting to analyze whether this had any influence on their satisfaction and opinion. In any case, the total cost of each notebook (€400) is not something extraordinary, given the registration tasks participants were asked to perform.

It should also be kept in mind that participants used several sensors, but the specific utility of each sensor was not assessed. In future studies, participants will be able to choose (or the clinician will be able to suggest) a specific sensor or group of sensors with a precise goal. Finally, data are limited at 3-month follow-up.

Conclusion

Despite its limitations, this is the first study to use an Internet-based program plus sensors, and it demonstrates that the self-administered program was effective with and without sensors. In some cases, the improvements were greater for the sensor group than for the group that did not use sensors (although no significant differences were found between the two intervention groups). Furthermore, the self-administered program is very easy to use and well accepted (with and without sensors), and it opens the door to the use of devices such as physiological sensors in the field of Internet-delivered psychological interventions.

Acknowledgments

This research was funded by OPTIMI (European Union’s Seventh Framework Program Personal Health Systems – Grant Agreement number 248544), Ministerio de Ciencia e Innovación, PSI2010-17563, the CIBER of Physiopathology of Obesity and Nutrition, an initiative of ISCIII. We confirm that the manuscript has been read and approved by all named authors. We further confirm that the order of authors listed in the manuscript has been approved by all of us.

Disclosure

The authors confirm that there are no conflicts of interest in this work and there has been no financial support for this work that could have influenced its outcome.

We confirm that we have given due consideration to the protection of intellectual property associated with this work and that there are no impediments to publication, including timing regarding to intellectual property. In so doing, we confirm that we have followed the regulations of our institutions concerning intellectual property.

---

References

1.		Ferrari AJ, Charlson FJ, Norman RE, et al. Burden of depressive disorders by country, sex, age, and year: findings from the global burden of disease study 2010. PLoS Med. 2013;10(11):e1001547.
2.		Botella C, Moragrega I, Baños R, García-Palacios A. Online predictive tools for intervention in mental illness: the OPTIMI project. Stud Health Technol Inform. 2011;63:86–92.
3.		Barlow DH, Ellard KK, Fairholme CP, et al. The Unified Protocol for Transdiagnostic Treatment of Emotional Disorders: Client Workbook. New York, NY: Oxford University Press; 2011.
4.		Cuijpers P, Beekman ATF, Reynolds CF. Preventing depression: a global priority. JAMA. 2010;307:1033–1034.
5.		Gaston JE, Abbott MJ, Rapeey RM, Neary SA. Do empirically supported treatments generalize to private practice? A benchmark study of a cognitive-behavioural group treatment programme for social phobia. Br J Clin Psychol. 2006;45(1):33–48.
6.		Kaltenthaler E, Brazier J, De Nigris E, et al. Computerised cognitive behaviour therapy for depression and anxiety update: a systematic review and economic evaluation. Health Technol Assess. 2006;10(33):1–168.
7.		Kaltenthaler E, Parry G, Beverley C, Ferriter M. Computerised cognitive-behavioural therapy for depression: systematic review. Br J Psychiatry. 2008;193(3):181–184.
8.		Tate DF, Zabinski MF. Computer and Internet applications for psychological treatment: update for clinicians. J Clin Psychol. 2004;60(2):209–220.
9.		Andrews G, Cuijpers P, Craske MG, McEvoy P, Titov N. Computer therapy for the anxiety and depressive disorders is effective, acceptable and practical health care: a meta-analysis. PLoS One. 2010;5(10):e13196.
10.		Griffiths KM, Christensen H. Review of randomised controlled trials of internet interventions for mental disorders and related conditions. Clin Psychol. 2006;10:16–29.
11.		Postel MG, de Haan HA, De Jong CA. E-therapy for mental health problems: a systematic review. Telemed J E Health. 2008;14(7):707–714.
12.		Richards D, Richardson T. Computer-based psychological treatments for depression: a systematic review and meta-analysis. Clin Psychol Rev. 2012;32(4):329–342.
13.		Titov N, Dear BF, Schwencke G, et al. Transdiagnostic internet treatment for anxiety and depression: a randomised controlled trial. Behav Res Ther. 2011;49(8):441–452.
14.		Kaltenthaler E, Sutcliffe P, Parry G, Beverley C, Rees A, Ferriter M. The acceptability to patients of computerized cognitive behaviour therapy for depression: a systematic review. Psychol Med. 2008;38(11):1521–1530.
15.		Kay-Lambkin F, Baker A, Lewin T, Carr V. Acceptability of a clinician-assisted computerized psychological intervention for comorbid mental health and substance use problems: treatment adherence data from a randomized controlled trial. J Med Internet Res. 2011;13(1):e11.
16.		Mohr DC, Siddique J, Jin L, Fokuo JK. Interest in behavioral and psychological treatments delivered face-to-face, by telephone and by internet. Ann Behav Med. 2010;40(1):89–98.
17.		De Graaf LE, Huibers MJH, Riper H, Gerhards SAH, Arntz A. Use and acceptability of unsupported online computerized cognitive behavioral therapy for depression and associations with clinical outcome. J Affect Disord. 2009;116(3):227–231.
18.		Osgood-Hynes DJ, Greist JH, Marks IM, et al. Self-administered psychotherapy for depression using a telephone-accessed computer system plus booklets: an open U.S.-U.K. study. J Clin Psychiatry. 1998;59(7):358–365.
19.		Botella C, Gallego MJ, Garcia-Palacios A, Baños RM, Quero S, Alcañiz M. The acceptability of an Internet-based self-help treatment for fear of public speaking. Br J Guid Counc. 2009;37(3):297–311.
20.		Carrard I, Crépin C, Rouget P, Lam T, Van der Linden M, Golay A. Acceptance and efficacy of a guided internet self-help treatment program for obese patients with binge eating disorder. Clin Pract Epidemiol Ment Health. 2011;7:8–18.
21.		Cavanagh K, Shapiro DA, Van Den Berg S, Swain S, Barkham M, Proudfoot J. The acceptability of computer-aided cognitive behavioural therapy: a pragmatic study. Cogn Behav Ther. 2009;38(4):235–246.
22.		Gun SY, Titov N, Andrews G. Acceptability of Internet treatment of anxiety and depression. Australas Psychiatry. 2011;19(3):259–264.
23.		Wootton BM, Titov N, Dear BF, Spence J, Kemp A. The acceptability of Internet-based treatment and characteristics of an adult sample with obsessive compulsive disorder: an Internet survey. PLoS One. 2011;6(6):e20548.
24.		Bonato P. Clinical applications of wearable technology. Proc IEEE Eng Med Biol Soc. 2009;2009:6580–6583.
25.		Teng XF, Zhang YT, Poon CC, Bonato P. Wearable medical systems for p-Health. IEEE Rev Biomed Eng. 2008;1:62–74.
26.		Puiatti A, Mudda S, Giordano S, Mayora O. Smartphone-centred wearable sensors network for monitoring patients with bipolar disorder. Proc IEEE Eng Med Biol Soc. 2011;2011:3644–3647.
27.		Holzinger A, Searle G, Wernbacher M. The effect of previous exposure to technology on acceptance and its importance in usability and accessibility engineering. Univ Access Inform Soc. 2011;10:245–260.
28.		Holzinger A, Searle G, Wernbacher, M. The effect of previous exposure to technology on acceptance and its importance in usability and accessibility engineering. UAIS. 2011;10:245–260.
29.		Cipresso P, Gaggioli A, Serino S, et al. Inter-reality in the evaluation and treatment of psychological stress disorders: the INTERSTRESS project. Stud Health Technol Inform. 2012;181:8–11.
30.		Gruenerbl A, Muaremi A, Osmani V, et al. Smartphone based recognition of states and state changes in bipolar disorder patients. IEEE J Biomed Health Inform. 2014;19(1):140–148.
31.		Spanakis EG, Chiarugi F, Kouroubali A, et al. Diabetes management using modern information and communication technologies and new care models. Interact J Med Res. 2012;1(2):e8.
32.		Tsipouras MG, Karvounis EC, Tzallas AT, et al. Automated knowledge-based fuzzy models generation for weaning of patients receiving ventricular assist device (VAD) therapy. Proc IEEE Eng Med Biol Soc. 2012;2012:2206–2209.
33.		Warmerdam L, Riper H, Klein M, et al. Innovative ICT solutions to improve treatment outcomes for depression: the ICT4Depression project. Stud Health Technol Inform. 2012;181:339–343.
34.		Riva G, Gaggioli A, Banos RM, Botella C, Wiederhold BK. Personal health systems for mental health: the European projects. Stud Health Technol Inform. 2011;163:496–502.
35.		Stjernswärd S, Ostman M. Illuminating user experience of a website for the relatives of persons with depression. Int J Soc Psychiatry. 2011;57(4):375–386.
36.		Algoe SB, Fredrickson BL. Emotional fitness and the movement of affective science from lab to field. Am Psychol. 2011;66(1):35–42.
37.		Ekers D, Richards D, McMillan D, Bland JM, Gilbody S. Behavioural activation delivered by the non-specialist: phase II randomised controlled trial. Br J Psychiatry. 2011;198(1):66–72.
38.		Botella C, Riva R, Gaggioli A, Wiederhold BK, Alcaniz M, Baños RM. The present and future of positive technologies. Cyberpsychol Behav Soc Netw. 2012;15(2):78–84.
39.		Holzinger A, Stocker C, Bruschi M, et al. On applying approximate entropy to ECG Signals for knowledge discovery on the example of big sensor data. Lect Notes Comput Sci. 2012;7669:646–657.
40.		Sun FT, Kuo C, Cheng HT, Buthpitiya S, Collins P, Griss ML. Activity-aware mental stress detection using physiological sensors. LNICST. 2012;76:282–301.
41.		Taliaferro LA, Rienzo BA, Pigg RM Jr, Miller MD, Dodd VJ. Associations between physical activity and reduced rates of hopelessness, depression, and suicidal behavior among college students. J Am Coll Health. 2009;57(4):427–436.
42.		Stickel C, Fink J, Holzinger A. Enhancing universal access – EEG based learnability assessment. Lect Notes Comput Sci. 2007;4556:813–822.
43.		Nieber D, Schlegel S. Relationships between psychomotor retardation and EEG power spectrum in major depression. Neuropsychobiology. 1992;25(1):20–23.
44.		Dooley D, Catalano R, Wilson G. Depression and unemployment: panel findings from the Epidemiologic Catchment Area study. Am J Community Psychol. 1994;22(6):745–765.
45.		Ferrando L, Bobes J, Gibert J, Lecubrier Y. Mini International Neuropsychiatric Interview. In: Bobes J, Bousoño M, González MP, editors. Manejo de los trastornos mentales y del comportamiento en Atención Primaria [Management of mental and behavioral disorders in primary care]. Oviedo: Gofer; 1997. Spanish.
46.		Beck AT, Steer RA, Brown GK. Manual for the Beck Depression Inventory-II. San Antonio, TX: Psychological Corp; 1996.
47.		Sanz J, Navarro ME Vázquez C. Adaptación española del Inventario para la Depresión de Beck-II (BDI-II): Propiedades psicométricas en estudiantes universitarios [Spanish adaptation of the Depression Inventory Beck-II (BDI-II): Psychometric properties in college students]. Análisis y modificación de conducta. 2003;29:239–288. Spanish.
48.		Norman SB, Cissell SH, Means-Christensen AJ, Stein MB. Development and validation of an overall anxiety severity and impairment scale (OASIS). Depress Anxiety. 2006;23(4):245–249.
49.		Sandín B, Chorot P, Lostao L, Joiner TE, Santed MA, Valiente RM. Escalas Panas de afecto positivo y negativo: validación factorial y transcultural [Panas scales of positive and negative affect: factorial and transcultural validation]. Psicothema. 1999;11(1):37–51. Spanish.
50.		Campo A, Bustos G, Romero A. Consistencia interna y dimensionalidad de la Escala de estrés percibido (EEP-10 y EEP-14) en una muestra de universitarias de Bogotá, Colombia Aquichan [Internal consistency and dimensionality of perceived stress scale (PSS-10 and PSS-14) in a university sample of Bogota, Colombia Aquichan]. Red de Revistas Científicas de América Latina, el Caribe, España y Portugal. 2009;9(3):271–280. Spanish.
51.		Borkovec TD, Nau S.D. Credibility of analogue therapy rationales. J Behav Ther Exp Psychiatry. 1972;3(4):257–260.
52.		Brooke J. SUS: a “quick and dirty” usability scale. In: Jordan PW, Thomas B, Weerdmeester BA, McClelland IL, editors. Usability Evaluation in Industry. London, UK: Taylor & Francis; 1996:189–194.
53.		Bangor A, Kortum PT, Miller JT. An empirical evaluation of the system usability scale. Int J Hum Comput Interact. 2008;24(6):574–594.
54.		Cuijpers P, Donker T, Johansson R, Mohr DC, van Straten A, Andersson G. Self-guided psychological treatment for depressive symptoms: a metaanalysis. PLoS One. 2011;6(6):e21274.
55.		Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V. Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a metaanalysis. Psychol Med. 2007;37(3):319–328.
56.		Andersson G, Cuijpers P. Internet-based and other computerized psychological treatments for adult depression: a meta-analysis. Cogn Behav Ther. 2009;38(4):196–205.
57.		Cuijpers P, Donker T, van Straten A, Li J, Andersson G. Is guided self-help as effective as face-to-face psychotherapy for depression and anxiety disorders? A systematic review and meta-analysis of comparative outcome studies. Psychol Med. 2010;40(12):1943–1957.