As professionals interested in building technologies for wellbeing, we'll need to leverage established methods to validly evaluate what gets built. Below is a psychology study on wellbeing conducted via the internet with 1,364 participants. This 2012 study by Schueller and Parks brings to light findings that could inform the design of wellbeing technologies.
Psychological interventions are evaluated by experimental designs accountable for the same methodological rigor as medical interventions. Many studies in both Psychiatry and Positive Psychology use the CES-D scale to measure depression and wellbeing.
Amongst the most prestigious journals for this type of study is the Journal of Medical Internet Research which has published a host of studies (including Randomized Control Trials) for different Internet based treatments. In these studies one or more treatments are evaluated using the treatment as independent variable and CES-D measure (or something similar) as dependent variable.
In one such study Schueller and Parks (Schueller & Parks, 2012) compared combinations of six types of self-help Positive Psychology type exercises: Active-constructive responding, Gratitude visit, Life summary, Three good things, Savoring and Strengths feedback.
The Schuller and Parks study is particularly useful to understand how design decisions for wellbeing software could be made. The study's main contribution to the Psychology literature is an evaluation of multi-exercise packages, rather than the more commonly studied approaches with a single exercise.
The researchers recruited 1,364 self-help-seeking people who were randomly allocated to one of three designs: 2, 4, or 6 online activities (or assessments only) over a 6-week period. These exercises drew from the content of group positive psychotherapy. Participants interacted with a website that provided instructions, sent email reminders, and contained the baseline and follow-up assessments.
The evaluation method has similarities with approaches in Human-Computer Interaction research. After evaluating each design, participants reported their enjoyment of it, how often they had used it, and completed a questionnaire to measure impact (CES-D).
Relevance of the results to the design of Positive Computing technologies
Perhaps the most important result was strong evidence for efficacy of these internet-based treatments--all the designs produced significant reductions in depressive symptoms (F1,656 = 94.71, P < .001). The efficacy is measured using a repeated measures analysis of covariance on the measures of depressive symptoms. When participants started, the average scores on the CES-D scale indicated mild-to-moderate levels of depressive symptoms (with a mean 16.93 and SD 12.64 remember that 16 was the minimum for a clinical depression). Therefore, participants in each design group (the packages with 2, 4, and 6 exercises) and in the control conditions, all started with similar levels of depressive symptoms.
But also important for technology designers, is that the study showed a larger improvement (reduction in the depression measure) for the groups that received 2 or 4 exercises compared to those receiving the 6-exercise package or control condition. This suggests a peak efficacy after which adding interventions ceases to increase wellbeing.
Schueller, S. M., & Parks, A. C. (2012). Disseminating self-help: positive psychology exercises in an online trial. Journal of medical Internet research, 14(3), e63.