Analysis of cognitive behavioural therapy apps for generalised anxiety disorder: Evidence-based content and user experience

Vol.18,No.4(2024)

Abstract

Mental illness substantially contributes to the global burden of disease, with anxiety high in prevalence. The increase of mobile technology, mental health apps have potential to lessen this burden. However, within apps, the use of evidence-based interventions, such as cognitive behavioural therapy (CBT) are limited. Regardless, many commercially available mental health apps are highly rated by users, highlighting the need to understand what makes mental health apps valuable to the user. The contribution of this study was to uncover apps that support generalised anxiety disorder (GAD) and worry with a CBT basis, explore app functionality, and user experience. Firstly, by identifying apps that support GAD and worry and included CBT. Secondly, by identifying and analysing therapeutic and engagement functions within the apps, and finally, by thematically analysing user reviews. Six apps were identified to support GAD and worry that purported to be CBT-based. However, CBT therapeutic features and engagement features were minimally present in the apps. User reviews yielded 112 comments about the apps and key themes were identified about the app users’ global experiences with the app, and about the combination of technological (e.g., useability, reliability) and therapeutic experiences (e.g., learning and using skills). Future development of quality apps to support GAD and worry must consider the empirical standing of both therapeutic and technology aspects, to provide efficacious and engaging interventions.


Keywords:
GAD; CBT; mobile apps; user experience
Author biographies

Teresa Munteanu

University of the Sunshine Coast, School of Health, Discipline of Psychology, Queensland, Australia

Teresa Munteanu is a graduate student of the University of the Sunshine Coast, Australia, with a degree in Bachelor of Psychology (Honors). She conducted this research as part of her Bachelor of Psychology (Honors) degree. Her research interests include digital mental health, anxiety, neurodivergence, and educational psychology.

Lee Kannis-Dymand

University of the Sunshine Coast, School of Health, Discipline of Psychology, Queensland, Australia; Dr Jay, Digital Mental Health Care, Paris, France

Lee Kannis-Dymand is Senior Lecturer in Clinical Psychology. He is a registered Clinical Psychologist with a Postgraduate Degree in Cognitive Behaviour Therapy from the University of Oxford. He has clinical expertise in the assessment and treatment of anxiety and trauma conditions, including Generalized anxiety Disorder. His research focuses on digital mental health, user experience, awe (human-wildlife/nature tourism, psychedelic experiences), environmental psychology, and metacognitive processes.

Prudence Millear

University of the Sunshine Coast, School of Health, Discipline of Psychology, Queensland, Australia

Prudence Millear is a Senior Lecturer in Psychology. She completed her PhD in Occupational Health Psychology at the Queensland University of Technology. She has research expertise in structural equation modelling and the design and analysis of quantitative and qualitative research projects. Her research projects focus on individual differences across the lifespan and how the characteristics of the person drive their experiences and behaviours in their work, family, and leisure roles.

Monique Jones

University of the Sunshine Coast, Thompson Institute & National PTSD Centre, Queensland, Australia

Monique Jones is a mental health nurse and researcher based at Thompson Institute. She is a Registered Nurse with Postgraduate Diplomas in Cognitive Behaviour Therapy and Mental Health. Monique’s has clinical expertise in the assessment and treatment of anxiety and trauma. Her research interests include early intervention, lifestyle medicine, health coaching, digital interventions, and disaster recovery.

References

Alqahtani, F., & Orji, R. (2019, June). Usability issues in mental health applications. In Adjunct publication of the 27th conference on user modeling, adaptation and personalization (pp. 343–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠348). ACM. https://doi.org/10.1145/3314183.3323676

Alyami, M., Giri, B., Alyami, H., & Sundram, F. (2017). Social anxiety apps: A systematic review and assessment of app descriptors across mobile store platforms. BMJ Mental Health, 20(3), 65–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠70. http://dx.doi.org/10.1136/eb-2017-102664

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (5th ed.). https://doi.org/10.1176/appi.books.9780890425596

Axelsson, E., Andersson, E., Ljótsson, B., & Hedman-Lagerlöf, E. (2018). Cost-effectiveness and long-term follow-up of three forms of minimal-contact cognitive behaviour therapy for severe health anxiety: Results from a randomised controlled trial. Behaviour Research and Therapy, 107, 95–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠105. https://doi.org/10.1016/j.brat.2018.06.002

Balaskas, A., Schueller, S. M., Cox, A. L., & Doherty, G. (2021). The functionality of mobile apps for anxiety: Systematic search and analysis of engagement and tailoring features. JMIR mHealth and uHealth, 9(10), Article e26712. http://dx.doi.org/10.2196/26712

Ballenger, J. C., Davidson, J. R., Lecrubier, Y., Nutt, D. J., Borkovec, T. D., Rickels, K., Stein, D. J., & Wittchen, H. U. (2001). Consensus statement on generalized anxiety disorder from the International Consensus Group on Depression and Anxiety. Journal of Clinical Psychiatry, 62, 53–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠58. https://hdl.handle.net/11858/00-001M-0000-000E-A31C-3

Bandelow, B., Michaelis, S., & Wedekind, D. (2017). Treatment of anxiety disorders. Dialogues in Clinical Neuroscience, 19(2), 93–107. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5573566/

Barlow, D., Rapee, R., & Brown, T. (1992). Behavioral treatment of generalized anxiety disorder. Behavior Therapy, 23(4), 551–570. https://doi.org/10.1016/S0005-7894(05)80221-7

Baumel, A., & Kane, J. M. (2018). Examining predictors of real-world user engagement with self-guided eHealth interventions: Analysis of mobile apps and websites using a novel dataset. Journal of Medical Internet Research, 20(12), Article e11491. https://doi.org/10.2196/11491

Baumel, A., Torous, J., Edan, S., & Kane, J. M. (2020). There is a non-evidence-based app for that: A systematic review and mixed methods analysis of depression-and anxiety-related apps that incorporate unrecognized techniques. Journal of Affective Disorders, 273, 410–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠421. https://doi.org/10.1016/j.jad.2020.05.011

Borkovec, T. D. (2006). Applied relaxation and cognitive therapy for pathological worry and generalized anxiety disorder. In C. L. Graham & D. A. Wells (Eds), Worry and its psychological disorders: Theory, assessment, and treatment (pp. 273–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠287). Wiley. https://doi.org/10.1002/9780470713143.ch16

Borkovec, T. D., Newman, M., Pincus, A., & Lytle, R. (2002). A component analysis of cognitive behavioral therapy for generalized anxiety disorder and the role of interpersonal problems. Journal of Consulting and Clinical Psychology, 70(2), 288–298. https://doi.org/10.1037//0022-006X.70.2.288

Borkovec, T. D., & Ruscio, A. M. (2001). Psychotherapy for generalized anxiety disorder. Journal of Clinical Psychiatry, 62, 37–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠45. https://www.psychiatrist.com/wp-content/uploads/2021/02/16617_psychotherapy-generalized-anxiety-disorder.pdf

Borkovec, T. D., & Whisman, M. A. (1996). Psychosocial treatment for generalized anxiety disorder. In M. R. Mavissakalian & R. F. Prien (Eds.), Long-term treatments of anxiety disorders (pp. 171–199). American Psychiatric Association.

Braun, V., & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠101. https://doi.org/10.1191/1478088706qp063oa

Bry, L. J., Chou, T., Miguel, E., & Comer, J. S. (2018). Consumer smartphone apps marketed for child and adolescent anxiety: A systematic review and content analysis. Behavior Therapy, 49(2), 249–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠261. https://doi.org/10.1016/j.beth.2017.07.008

Caldeira, C., Chen, Y., Chan, L., Pham, V., Chen, Y., & Zheng, K. (2017). Mobile apps for mood tracking: An analysis of features and user reviews. In AMIA annual symposium proceedings (Vol. 2017, pp. 495–504). American Medical Informatics Association. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5977660/

Chandrashekar, P. (2018). Do mental health mobile apps work: Evidence and recommendations for designing high-efficacy mental health mobile apps. mHealth, 4, Article 6. https://doi.org/10.21037/mhealth.2018.03.02

Craske, M., Barlow, D., & Zinbarg, R. (2006). Mastery of Your Anxiety and Worry (MAW): Therapist Guide. Oxford University Press. https://doi.org/10.1093/med:psych/9780195300017.001.0001

Doherty, G., Coyle, D., & Sharry, J. (2012, May). Engagement with online mental health interventions: An exploratory clinical study of a treatment for depression. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1421–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠1430). ACM. https://doi.org/10.1145/2207676.2208602

Doran, C. M., & Kinchin, I. (2019). A review of the economic impact of mental illness. Australian Health Review, 43(1), 43–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠48. https://doi.org/10.1071/AH16115

Drissi, N., Ouhbi, S., Idrissi, M. A. J., & Ghogho, M. (2020). An analysis on self-management and treatment-related functionality and characteristics of highly rated anxiety apps. International Journal of Medical Informatics, 141, Article 104243. https://doi.org/10.1016/j.ijmedinf.2020.104243

Dugas, M. J. (2002). Generalized anxiety disorder. In M. Hersen (Ed.), Clinical behaviour therapy: Adults and children (pp. 125–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠143). Wiley. https://www.wiley.com/en-us/Clinical+Behavior+Therapy%3A+Adults+and+Children-p-9780471392583

Dugas, M. J., & Robichaud, M. (2007). Cognitive-behavioral treatment for generalized anxiety disorder: From science to practice. Routledge. https://psycnet.apa.org/record/2006-20225-000

Fenn, K., & Byrne, M. (2013). The key principles of cognitive behavioural therapy. InnovAiT, 6(9), 579–585. https://doi.org/10.1177/1755738012471029

Firth, J., Torous, J., Nicholas, J., Carney, R., Pratap, A., Rosenbaum, S., & Sarris, J. (2017). The efficacy of smartphone‐based mental health interventions for depressive symptoms: A meta‐analysis of randomized controlled trials. World Psychiatry, 16(3), 287–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠298. https://doi.org/10.1002/wps.20472

Firth, J., Torous, J., Nicholas, J., Carney, R., Rosenbaum, S., & Sarris, J. (2017). Can smartphone mental health interventions reduce symptoms of anxiety? A meta-analysis of randomized controlled trials. Journal of Affective Disorders, 218, 15–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠22. https://doi.org/10.1016/j.jad.2017.04.046

Hirsch, C. R., Beale, S., Grey, N., & Liness, S. (2019). Approaching cognitive behavior therapy for generalized anxiety disorder from a cognitive process perspective. Frontiers in Psychiatry, 10, Article 796. https://doi.org/10.3389/fpsyt.2019.00796

Hong, S., & Pittman, M. (2020). eWOM anatomy of online product reviews: Interaction effects of review number, valence, and star ratings on perceived credibility. International Journal of Advertising, 39(7), 892–920. https://doi.org/10.1080/02650487.2019.1703386

Howell, A. N., Rheingold, A. A., Uhde, T. W., & Guille, C. (2019). Web-based CBT for the prevention of anxiety symptoms among medical and health science graduate students. Cognitive Behaviour Therapy, 48(5), 385–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠405. https://doi.org/10.1080/16506073.2018.1533575

Hunot, V., Churchill, R., Teixeira, V., & de Lima, M. S. (2007). Psychological therapies for generalised anxiety disorder. Cochrane Database of Systematic Reviews, 1, Article CD001848. https://doi.org/10.1002/14651858.CD001848.pub4

Kertz, S. J., Kelly, J. M., Stevens, K. T., Schrock, M., & Danitz, S. B. (2017). A review of free iPhone applications designed to target anxiety and worry. Journal of Technology in Behavioral Science, 2(2), 61–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠70. https://doi.org/10.1007/s41347-016-0006-y

Kessler, D., Lewis, G., Kaur, S., Wiles, N., King, M., Weich, S., Sharp, D., Araya, R., Hollinghurst, S., & Peters, T. J. (2009). Therapist-delivered internet psychotherapy for depression in primary care: A randomised controlled trial. The Lancet, 374(9690), 628–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠634. https://doi.org/10.1016/S0140-6736(09)61257-5

Lui, J. H., Marcus, D. K., & Barry, C. T. (2017). Evidence-based apps? A review of mental health mobile applications in a psychotherapy context. Professional Psychology: Research and Practice, 48(3), 199–210. https://doi.org/10.1037/pro0000122

Marshall, J. M., Dunstan, D. A., & Bartik, W. (2020). Apps with maps—Anxiety and depression mobile apps with evidence-based frameworks: Systematic search of major app stores. JMIR Mental Health, 7(6), Article e16525. https://doi.org/10.2196/16525

Miller, C., Gu, J., Henry, A., Davis, M., Espie, C., Stott, R., Heinz, A. J., Bentley, K. H., Goodwin, G. M., Gorman, B. S., Craske, M. G., & Carl, J. R. (2021). Feasibility and efficacy of a digital CBT intervention for symptoms of generalized anxiety disorder: A randomized multiple-baseline study. Journal of Behavior Therapy and Experimental Psychiatry, 70, Article 101609. https://doi.org/10.1016/j.jbtep.2020.101609

Mudambi, S., & Schuff, D. (2010). Research note: What makes a helpful online review? A study of customer reviews on Amazon.com. MIS Quarterly, 34(1), 185–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠200. https://doi.org/10.2307/20721420

Muroff, J., & Robinson, W. (2020). Tools of engagement: Practical considerations for utilizing technology-based tools in CBT practice. Cognitive and Behavioral Practice, 9, 81–96. https://doi.org/10.1016/j.cbpra.2020.01.004

National Collaborating Centre for Mental Health. (2011). Generalised anxiety disorder in adults: The NICE guideline on management in primary, secondary, and community care. https://www.nice.org.uk/guidance/cg113/evidence/anxiety-full-guidance-pdf-136340461

National Institute for Health Care Excellence. (2019). Generalised anxiety disorder and panic disorder in adults: Management. https://www.nice.org.uk/guidance/cg113

Nicholas, J., Fogarty, A. S., Boydell, K., & Christensen, H. (2017). The reviews are in: A qualitative content analysis of consumer perspectives on apps for bipolar disorder. Journal of Medical Internet Research, 19(4), Article e105. https://doi.org/10.2196/jmir.7273

Neary, M., & Schueller, S. M. (2018). State of the field of mental health apps. Cognitive and Behavioral Practice, 25(4), 531–537. https://doi.org/10.1016/j.cbpra.2018.01.002

Newby, J., Mason, E., Kladnistki, N., Murphy, M., Millard, M., Haskelberg, H., Allen, A., & Mahoney, A. (2021). Integrating internet CBT into clinical practice: A practical guide for clinicians. Clinical Psychologist (Australian Psychological Society), 25(2), 164–178. https://doi.org/10.1080/13284207.2020.1843968

Newman, M. G., Zainal, N. H., & Hoyer, J. (2020). Cognitive‐behavioral therapy (CBT) for generalized anxiety disorder (GAD). In A. Gerlach & A. Gloster (Eds.) Generalized anxiety disorder and worrying: A comprehensive handbook for clinicians and researchers (pp 203–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠230). Wiley. https://doi.org/10.1002/9781119189909.ch10

Nordgreen, T., Havik, O. E., Öst, L. G., Furmark, T., Carlbring, P., & Andersson, G. (2012). Outcome predictors in guided and unguided self-help for social anxiety disorder. Behaviour Research and Therapy, 50(1), 13–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠21. https://doi.org/10.1016/j.brat.2011.10.009

Oyebode, O., Alqahtani, F., & Orji, R. (2020). Using machine learning and thematic analysis methods to evaluate mental health apps based on user reviews. IEEE Access, 8, pp. 111141–111158. https://doi.org/10.1109/ACCESS.2020.3002176

Pearl, S. B., & Norton, P. J. (2017). Transdiagnostic versus diagnosis specific cognitive behavioural therapies for anxiety: A meta-analysis. Journal of Anxiety Disorders, 46, 11–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠24. http://dx.doi.org/10.1016/j.janxdis.2016.07.004

Popa, D., & Porumbu, D. (2017). Bibliotherapy in clinical context: An umbrella review. Bulletin of the Transilvania University of Brasov. Series VII: Social Sciences and Law, 10(2), 165–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠174. https://webbut.unitbv.ro/index.php/Series_VII/article/view/3096

Qu, C., Sas, C., Roquet, C. D., & Doherty, G. (2020). Functionality of top-rated mobile apps for depression: Systematic search and evaluation. JMIR Mental Health, 7(1), Article e15321. https://doi.org/10.2196/15321

Rathbone, A. L., Clarry, L., & Prescott, J. (2017). Assessing the efficacy of mobile health apps using the basic principles of cognitive behavioral therapy: Systematic review. Journal of Medical Internet Research, 19(11), Article e399. https://www.jmir.org/2017/11/e399/

Robichaud, M., & Dugas, M. J. (2006). A cognitive behavioral treatment targeting intolerance of uncertainty. In C. L. Graham & D. A. Wells (Eds.), Worry and its psychological disorders: Theory, assessment, and treatment (pp. 289–304). Wiley. https://doi.org/10.1002/9780470713143.ch17

Robichaud, M., & Dugas, M. J. (2019). Cognitive-behavioral treatment for generalized anxiety disorder: From science to practice. Routledge. https://doi.org/10.4324/9781315709741

Rubanovich, C. K., Mohr, D. C., & Schueller, S. M. (2017). Health app use among individuals with symptoms of depression and anxiety: A survey study with thematic coding. JMIR Mental Health, 4(2), Article e22. https://doi.org/10.2196/mental.7603+

Ruscio, A. M., Hallion, L. S., Lim, C. C. W., Aguilar-Gaxiola, S., Al-Hamzawi, A., Alonso, J., Andrade, L. H., Borges, G., Bromet, E. J., Bunting, B., Caldas De Almeida, J. M., Demyttenaere, K., Florescu, S., De Girolamo, G., Gureje, O., Haro, J. M., He, Y., Hinkov, H., Hu, C., … Scott, K. M. (2017). Cross-sectional comparison of the epidemiology of DSM-5 generalized anxiety disorder across the globe. JAMA Psychiatry, 74(5), 465–⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠475. https://doi.org/10.1001/jamapsychiatry.2017.0056

Shen, N., Levitan, M. J., Johnson, A., Bender, J. L., Hamilton-Page, M., Jadad, A. A., & Wiljer, D. (2015). Finding a depression app: A review and content analysis of the depression app marketplace. JMIR mHealth uHealth, 3(1), Article e16. https://doi.org/10.2196/mhealth.3713

Stawarz, K., Cox, A., & Blandford, A. (2014). Don’t forget your pill: Designing effective medication reminder apps that support users’ daily routines. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 2269–2278). ACM. https://doi.org/10.1145/2556288.2557079

Stawarz, K., Preist, C., Tallon, D., Wiles, N., & Coyle, D. (2018). User experience of cognitive behavioral therapy apps for depression: An analysis of app functionality and user reviews. Journal of Medical Internet Research, 20(6), Article e10120. https://doi.org/10.2196/10120

Sucala, M., Cuijpers, P., Muench, F., Cardoș, R., Soflau, R., Dobrean, A., Achimas‐Cadariu, P., & David, D. (2017). Anxiety: There is an app for that. A systematic review of anxiety apps. Depression and Anxiety, 34(6), 518–525. https://doi.org/10.1002/da.22654

Sundram, F., Hawken, S. J., Stasiak, K., Lucassen, M. F., Fleming, T., Shepherd, M., Greenwood, A., Osborne, R., & Merry, S. N. (2017). Tips and traps: Lessons from codesigning a clinician e-monitoring tool for computerized cognitive behavioral therapy. JMIR Mental Health, 4(1), Article e3. https://doi.org/10.2196/mental.5878

Tay, W., Zhang, X., & Karimi, S. (2020). Beyond mean rating: Probabilistic aggregation of star ratings based on helpfulness. Journal of the Association for Information Science and Technology, 71(7), 784–799. https://asistdl.onlinelibrary.wiley.com/doi/abs/10.1002/asi.24297

Thach, K. S. (2018, November). User’s perception on mental health applications: A qualitative analysis of user reviews. In 2018 5th NAFOSTED conference on information and computer science (pp. 47–52). IEEE. https://doi.org/10.1109/NICS.2018.8606901

Van Ameringen, M., Turna, J., Khalesi, Z., Pullia, K., & Patterson, B. (2017). There is an app for that! The current state of mobile applications (apps) for DSM‐5 obsessive‐compulsive disorder, posttraumatic stress disorder, anxiety and mood disorders. Depression and Anxiety, 34(6), 526–539. https://doi.org/10.1002/da.22657

Wang, K., Varma, D., & Prosperi, M. (2018). A systematic review of the effectiveness of mobile apps for monitoring and management of mental health symptoms or disorders. Journal of Psychiatric Research, 107, 73–78. https://doi.org/10.1016/j.jpsychires.2018.10.006

Wasil, A., Venturo-Conerly, K., Shingleton, R., & Weisz, J. (2019). A review of popular smartphone apps for depression and anxiety: Assessing the inclusion of evidence-based content. Behaviour Research and Therapy, 123, Article 103498. https://doi.org/10.1016/j.brat.2019.103498

Watts, S., Mackenzie, A., Thomas, C., Griskaitis, A., Mewton, L., Williams, A., & Andrews, G. (2013). CBT for depression: A pilot RCT comparing mobile phone vs. computer. BMC Psychiatry, 13, Article 49. https://doi.org/10.1186/1471-244X-13-49

Wells, A. (1999). A metacognitive model and therapy for generalized anxiety disorder. Clinical Psychology and Psychotherapy, 6(2), 86–95. https://doi.org/10.1002/(SICI)1099-0879(199905)6:2<86::AID-CPP189>3.0.CO;2-S

Wells, A. (2010). Metacognitive theory and therapy for worry and generalized anxiety disorder: Review and status. Journal of Experimental Psychopathology, 1(1), 133–145. https://doi.org/10.5127/jep.007910

Wells, A., Welford, M., King, P., Papageorgiou, C., Wisley, J., & Mendel., E. (2010). A pilot randomized trial of metacognitive therapy versus applied relaxation in the treatment of adults with generalised anxiety disorder. Behavior Research and Therapy, 48(5), 429–434. https://doi.org/10.1016/j.brat.2009.11.013

Wootton, B. M., Steinman, S. A., Czerniawski, A., Norris, K., Baptie, C., Diefenbach, G., & Tolin, D. F. (2018). An evaluation of the effectiveness of a transdiagnostic bibliotherapy program for anxiety and related disorders: Results from two studies using a benchmarking approach. Cognitive Therapy and Research, 42(5), 565–580. https://doi.org/10.1007/s10608-018-9921-x

Wright, J. H., Mishkind, M., Eells, T. D., & Chan, S. R. (2019). Computer-assisted cognitive-behavior therapy and mobile apps for depression and anxiety. Current Psychiatry Reports, 21(7), Article 62. https://doi.org/10.1007/s11920-019-1031-2

Zivor, M., Salkovskis, P., & Oldfield, V. (2013). If formulation is the heart of cognitive behavioural therapy, does this heart rule the head of CBT therapists? The Cognitive Behaviour Therapist, 6(4), Article e6. https://doi.org/10.1017/S1754470X1300010X

Additional information

Authors’ Contribution

Teresa Munteanu: conceptualization, data curation, formal analysis, investigation, methodology, project administration, visualization, writing—original draft, writing—review & editing. Lee Kannis-Dymand: conceptualization, data curation, formal analysis, investigation, methodology, project administration, supervision, validation, visualization, writing—original draft, writing—review & editing. Prudence Millear: data curation, formal analysis, investigation, methodology, project administration, supervision visualization, writing—original draft, writing—review & editing. Monique Jones: formal analysis, investigation, validation, visualization, writing—review & editing.⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠⁠

 

Editorial Record

First submission received:
March 11, 2022

Revisions received:
April 25, 2023
January 25, 2024
July 11, 2024

Accepted for publication:
August 28, 2024

Editor in charge:
David Smahel

Full text

Introduction

Mental health substantially contributes to the global burden of disease and can affect daily functioning, employment, financial stability, education, and relationships (Doran & Kinchin, 2019; Drissi et al., 2020). Anxiety disorders, such as generalised anxiety disorder (GAD), are prevalent, with a global 12-month rate of 1.8% and a higher rate of 3.6% in the Australian population (Bandelow et al., 2017; Ruscio et al., 2017). The core feature of GAD is disproportionate and excessive worry and anxiety about a range of life circumstances that is difficult for a person to control (American Psychiatric Association, 2013).

Cognitive behavioural therapy (CBT) is the recommended first line of psychological treatment for GAD according to UK and US clinical guidelines (Ballenger et al., 2001; Hunot et al., 2007; National Institute for Health Care Excellence; 2019). Hirsch et al. (2019) noted that meta-analyses of randomised control trials on CBT for GAD, confirmed its superiority as a treatment modality. CBT for GAD often includes worry postponement, targeting of intolerance of uncertainty, and modification of beliefs about worry (Dugas & Robichaud, 2007; Wells, 1999), which are not typically targeted in other anxiety disorders. Given the established strength of CBT for GAD, the quality of mobile app CBT treatments for GAD should be considered.

CBT interventions can be successfully delivered through various platforms including mobile applications (Howell et al., 2019; Kessler et al., 2009; Wootton et al., 2018). Self-help and digital health interventions have been found to be effective in obsessive compulsive disorder (OCD), depression, and anxiety disorders and can increase access to support, reduce symptoms, and are cost efficient (for example Axelsson et al., 2018; Miller et al., 2021; Nordgreen et al., 2012; Popa & Porumbu, 2017; Wright et al., 2019). Apps that target anxiety can significantly reduce symptoms compared to controls (Firth, Torous, Nicholas, Carney, Pratap, et al., 2017; Firth, Torous, Nicholas, Carney, Rosenbaum, & Sarris, 2017; Wang et al., 2018). CBT delivered this way can significantly ameliorate anxiety symptoms, including for GAD (Miller et al., 2021; Watts et al., 2013).

However, several studies have found that the content of mental health apps were not strongly linked to evidence-based interventions (e.g., CBT; Alyami et al., 2017; Kertz et al., 2017; Lui et al., 2017; Shen et al., 2015). For example, a study on apps for social anxiety found that most of the apps did not state the source of their content or the apps’ efficacy (Alyami et al., 2017). A review of child and adolescent anxiety apps found evidence-based content was scarce and despite claiming to be evidence-based, apps included features that were not evidence-based (Bry et al., 2018).

Further, an apps’ empirical basis and clinical usefulness may be unrelated to its popularity and consumer ratings (Kertz et al., 2017; Neary & Schueller, 2018; Wasil et al., 2019). A review of top-rated apps for depression found that only 48% of apps claimed to be based on psychological interventions with only 7% providing evidence of the app’s efficacy in reducing depressive symptoms, and less than 41% including input from a mental health professional (Qu et al., 2020).

Mental health apps that have engaging user experiences can encourage users to engage with an app, through the inclusion of design features such as gamification, the ability to use apps offline, and action reminders (Chandrashekar, 2018). Amongst user reviews for bipolar disorder apps (N = 2,173) and mental health apps generally (N = 88,125), positive reviews focused on helpfulness and ease of use, whilst negative reviews focused on functionality and useability issues, lack of features, or lack of content (Nicholas et al., 2017; Oyebode et al., 2020). Research on app users’ reviews provides insight into users’ needs, and the strengths, advantages, and disadvantages of mental health apps (e.g., Baumel et al., 2020; Calderia et al., 2017; Stawarz et al., 2014, 2018). Balaskas et al. (2021, p. 2) stated, “…we need to understand the content in mental health apps from the viewpoint of not only evidence-based strategies but also of engagement elements and other features relevant to digital delivery.”

Therefore, understanding the status of evidence-based apps for GAD and users’ experience of such apps, may assist in future development of effective apps for the treatment of anxiety, specifically GAD. The current study aimed to explore the therapeutic content and user reviews of apps that are commercially available in English and target GAD and worry, and which purport to be CBT-based. The apps’ content were assessed for their fidelity to CBT principles in general and for GAD specifically (as outlined as best practice, e.g., in the National Institute for Health Care Excellence, 2019; guidelines). The apps’ user experience was assessed firstly by the features that would increase user engagement (e.g., visual design of the app, notifications and reminders, ability to customise) and secondly, by thematic analysis of the publicly available user reviews on the apps.

Methods

Research Design Overview

Following ethic’s approval by the Univeristy of the Sunshine Coast Human Ethics' Committee (S201480), analyses of commercially available apps for anxiety that employed CBT were undertaken in three steps, consistent with Stawarz et al. (2018). First, identification of the apps (e.g., using keyword searches, identifying CBT in the app) suitable for inclusion was undertaken. Second, the analysis of therapeutic features (based on clinical guidelines for CBT) and engagement features (i.e., what encourages continued use of the app) was undertaken, and third, a qualitative analysis of users’ experiences (based on user reviews of the apps) was conducted.

Identification of Apps

Phase 1: Keyword Search and Data Clean

Searches of Google Play and iTunes App Store (as commercial providers of apps) were conducted with the defined keywords: ‘Anxiety’, ‘generalised anxiety disorder’, ‘generalized anxiety disorder’, ‘GAD’ and ‘worry.’ Searches were manually undertaken for each key word separately, by two researchers in December 2020. As shown in Figure 1, 1,888 apps were identified in the two app stores using the search terms, with 985 being excluded as not being suitable (i.e., not relevant conditions). The names and links of remaining apps (n = 903) were recorded for the next phase (712 from Google Play, 191 from iTunes).

Phase 2: Identify Potentially Relevant Apps

A manual review of the apps’ titles and short descriptions was conducted collaboratively by two researchers to ensure that the apps were CBT-based and designed for GAD and/or worry. Duplicate apps and apps that were identified to be not relevant to the research question were excluded. The exclusion criteria were: (1) Apps that were not relevant to the research such as mobile phone themes, games, podcasts and music; (2) Apps that stated therapeutic theories or frameworks other than CBT; (3) General single purpose apps that only offer one therapeutic strategy/tool; (4) Apps that targeted mental health and/or anxiety generally (i.e., not specific to GAD) or included comorbidities such as depression, mental health conditions other than GAD and worry; (5) Apps that had limited availability (e.g., accessed through an employer or insurance); (6) Apps that were originally identified in the first search, though not found in the app store in the second phase of the review; or (7) Apps not available in English. As shown in Figure 1, the list of 903 apps was reduced to 26 potentially relevant apps that were identified for further study (16 Google Play, 10 iTunes).

Phase 3: Identify CBT Apps Targeting GAD and/or Worry

The 26 apps included in the final step were assessed in detail for the presence of CBT content in the apps (Table 1). This step was completed by two researchers with graduate training in CBT for GAD (Monique Jones and Lee Kannis-Dymand). In this stage, only 8 of the 26 apps were identified as using CBT and stating to assist GAD and/or worry. Four apps self-identified as CBT apps (2 from Google and 2 from iTunes). Another 4 apps (2 from Google Play and 2 from iTunes) that clearly depicted CBT strategies and/or tools in their descriptions or screen shot displays, were identified, and included in the study. As shown in Figure 1, in the final sample, 6 apps were included in the current study for the analysis, as one app would not function when downloaded, and one was found to be a duplicate, in further review stages of analysis.

Figure 1. Data Search and Exclusion Procedure.

Note. Separate searches were initially conducted for “generalised anxiety disorder” and “generalized anxiety disorder”.

Analysis of the Therapeutic and Engagement Features of the Apps

Design of the App

Functionality of the apps was assessed by downloading and viewing the app and rated by (1) the therapeutic value of the app (i.e., has best practice or recommended CBT features, has strategies to reduce anxiety symptoms). This included CBT therapeutic features and other therapeutic features that were identified but not congruent to CBT principles. Then, (2) features that would engage the user and increase the likelihood of future use of the app (purposeful functionality that may increase the appeal of the app content or encourages engagement in the app and/or the therapy within the app).

Presence of Best Practice CBT Content Within the Apps

Best practice, evidence-based CBT criteria, to review the apps against, were drawn from the United Kingdom’s National Institute for Health Care Excellence (NICE; 2019) guidelines and its associated resources (National Collaborating Centre for Mental Health; 2011) and references (Barlow et al., 1992; Borkovec, 2006; Borkovec & Ruscio, 2001; Borkovec & Whisman, 1996; Borkovec et al., 2002; Craske et al., 2006; Dugas, 2002; Dugas & Robichaud, 2007; Robichaud & Dugas, 2006, 2019; Wells, 2010; Wells et al., 2010). The criteria included 26 CBT items that address aspects of GAD, for example, self-assessment, instruction of strategy use, setting user goals, psychoeducation about GAD/anxiety/worry, and information about CBT. The full list of therapeutic items is shown in Table 1. Other CBT criteria, not specific to GAD, were also recorded (and shown in Table 2), for example, general CBT information and distraction.

Two researchers with extensive training and practitioner experience of CBT for GAD conducted the next phase to determine if the proposed CBT focus within the apps were congruent with best practice CBT guidelines for GAD. A clinical psychologist (Lee Kannis-Dymand) and a mental health nurse (Monique Jones); both with postgraduate qualifications in CBT and clinical experience working at a specialised anxiety disorders service in New Zealand. There were two apps removed from the analysis at this stage. One app (Beat Anxiety from Google Play) malfunctioned when downloaded and was inaccessible for analysis. The other app was found in both Google play and iTunes stores with minimal difference between their interface, content, and function, as such, this was analysed as one app. The researchers jointly reviewed and discussed each app (i.e., thoroughly went through the app’s menu and strategies) to determine the therapeutic features offered and how these matched with best practice CBT criteria for GAD, and if there had been CBT expert involvement in the development of the app.

Identification of Engagement Functions of Apps

Based on the engagement features defined in the Stawarz et al. (2018) study and additional relevant research (Muroff & Robinson, 2020; see Table 4), a manual review of the CBT apps was conducted by a researcher (Teresa Munteanu). This was conducted by a researcher who was not an expert in user experience to ensure that engagement features were easily identifiable and would be discoverable by a user/consumer from the public, rather than someone who was an expert in user experience or user design.

Users’ Experiences of the Apps

The final step of the analyses was to qualitatively assess publicly available user reviews of the apps. Explorative research of publicly available app reviews has been previously conducted to gain an understanding of users’ opinions of apps and their features (Caldeira et al., 2017; Shen et al., 2015; Stawarz et al., 2014, 2018). Similar methodology was utilised in this study to explore users’ opinions and attitudes of the CBT apps. Reviews were manually retrieved from each app store with the date, star rating and review comment recorded. Where apps were available on both Google and iTunes, reviews were retrieved from both stores. Visible iTunes reviews on mobile devices were limited and only one iTunes app had a substantial number of reviews. An explanation for this is the iTunes policy that removes reviews from previous versions of an app, unless otherwise requested by the developer. There were 129 comment reviews in total (77 from Google and 52 from iTunes).

Data Analysis

Data analysis followed the study’s main aims, being identification of apps’ therapeutic and engagement functions. The frequencies of the criteria for CBT that apply specifically to GAD, non-GAD-specific CBT, and non-CBT strategies in each app were tallied, with definitions for each criterion. Qualitative analysis of the users’ comments used Braun and Clark’s (2006) stages of thematic analysis. Microsoft Excel was used to organise data and all comments were reviewed independently by two researchers (Lee Kannis-Dymand, Teresa Munteanu). After the initial coding and development of themes, the authors began a collaborative review and discussion of coding, thematic definitions, and discrepancies between the initial findings. This review process enabled greater data familiarity and reduced potential for bias due to high levels of agreement between the coders. Themes were generated and driven by the explicit meaning of the users’ words and the features to which the comments applied (e.g., usefulness of CBT features, ease of app’s technology and user interface); correspondingly, themes and what they encompassed was informed by prior research that identified core themes in user experience of mental health apps (e.g., Stawarz et al., 2018). Themes were generated, reviewed, refined, and defined in relation to the overall data by the two researchers, until clear themes were developed and identified (Lee Kannis-Dymand, Teresa Munteanu).

Results

Identification of the Apps

Following the three phases to identify apps for GAD and worry with a CBT basis (Figure 1), six apps were included for the functional analysis. These apps were, in alphabetic order, from Google (G) and/or from Apple iTunes (A): (1) FearTools (G and A), (2) HappiMe for young people (A), (3) Shrinkinabox (A), (4) WorryTime by ReachOut (A) and ReachOut Worrytime (G) which are the same app with different names, (5) WorryTree (G and A), and (6) Worrywatch habit tracker (A).

Design of Apps and Their Functionality

The analysis of identified CBT apps for GAD and worry found 39 therapeutic features and 13 engagement features, with Fear Tools having the most evidence-based features (10/30) and most non-CBT features (6). The therapeutic features and their prevalence in the apps are detailed in Tables 1 to 4.

Therapeutic Features of CBT Apps for GAD and Worry

As shown in Table 1, of the 26 GAD specific items within the guidelines, 9 were not found within any app. All identified apps included some of the 30 evidence-based features (see Tables 1 & 2), ranging between 2 and 10 features (M = 6.8, Md = 7.5, Mo = 8). All apps were found to include non-CBT therapeutic features ranging from 1 to 6 features (mean = 2.65, median = 1.5, mode = 1; see Table 3). The most frequently identified therapeutic features found across the apps were instructions for a CBT strategy (5/6, 83%), identifying negative or unhelpful cognitions (4/6, 67%), and addressing negative automatic thoughts (4/6, 67%). Fear Tools had the most evidence-based features (10/30), along with the greatest number of non-CBT features (6). Overall, the data uncovered that the apps contained a limited number of CBT strategies, and no app had a complement of strategies consistent with a full GAD treatment protocol (see Tables 1–3 detailing which apps had which strategies).

CBT features were also considered in comparison to app user ratings, although one app did not have a star rating (Shrinkinabox). The average of user star ratings (5-point scale) of apps ranged between 4 and 4.8. For those apps with star ratings, the app (Worry Watch) that had fewest number of CBT features (6) had an average star rating of 4.6 and the app (Fear Tools) with the most CBT features (10) rated 4.2 on average.

Table 1. Therapeutic Features: GAD-Specific CBT Features Within the Apps, With the Number and Names of Apps in Which the Features Appear.

Therapeutic features Definitions Number of Apps, n (%) App Name
Self-assessment Tests, measures and/or scales to assess anxiety, worry, and/or well-being.

1 (16.67)

FT
Instructions on how to use strategies within the app

A guide to the user about using the CBT strategies (e.g., order of, etc.).

5 (83.33)

WTr, WTm, FT, HYP, WW

Setting user goals for using the app Goal setting regarding the user’s desired therapeutic gains after using the CBT app (e.g., anxiety or worry reduces); not training in goal setting itself.

2 (33.33)

WTm, SB

Psychoeducation about GAD/ anxiety/worry Text, videos, and resources explaining GAD/anxiety and worry symptoms.

2 (33.33)

WTm, FT

Information about CBT Resources explaining what is involved in cognitive behavioural therapy, and how it works.

3 (50.00)

WTm, FT, HYP

Formulation/CBT model A form of CBT model that outlines GAD/worry.

0 (00)

na
Self-monitoring strategies for tracking anxiety and worries (i.e., worry awareness training) For example, anxiety/mood record or worry record (worry lists, anxiety situations, etc.).

3 (50.00)

WTr, WTm, WW

Breathing exercises

Instructions for breathing exercises (e.g., diaphragmatic breathing, paced breathing).

2 (33.33)

FT, HYP

Relaxation training for reducing muscle tension

Some form of intervention for muscle tension relaxation (e.g., progressive muscle relaxation).

1 (16.67)

FT
Relaxation training using imagery or meditation

Relaxation techniques such as relaxing imagery or meditation.

1 (16.67)

HYP
Usefulness of worry is targeted

The advantages and disadvantages of worry are addressed.

1 (16.67)

WW
Intolerance of uncertainty

The role of intolerance of uncertainty is noted and targeted.

2 (33.33)

WTr, WW

Identifying unhelpful or negative cognitions

Identifying negative thinking. May include identifying unhelpful thinking styles (e.g., catastrophising). May include thought diaries.

4 (66.67)

WTr, FT, HYP, WW

Addressing negative automatic thoughts

Challenging negative automatic thoughts or images about self or world. Can include reframing, restructuring, or imagery restructuring.

4 (66.67)

WTr, FT, HYP, WW

Imagery rehearsal of coping skills

Use imagery to rehearse coping skills (e.g., relaxation). Can include worry or fear exposure via imagery.

2 (33.33)

HYP, SB

Behavioural experiments or exposure

Tasks designed to test cognitive predictions, reduce avoidance or safety behaviours (can include exposure work).

1 (16.67)

FT

Problem solving

Guidance on how to problem solve (e.g., brainstorming, etc.).

0 (00)

na

Behavioural activation

Guidance on getting active (e.g., physical activity or exercise, social or recreation activities).

0 (00)

na
Time management

Strategies to manage time efficiently.

0 (00)

na

Setting goals

Training in goal setting. Short-term or long-term goals to work towards such as health, career, family, life, etc.

0 (00)

na

Relapse prevention or management

Some form of relapse prevention or therapy blueprint.

0 (00)

na

Metacognitive beliefs

Positive and negative metacognitive beliefs identified and reframed.

0 (00)

na

Thought suppression task

A task (e.g., “white bear”, “blue rabbits”) used to demonstrate that thought/worry suppression is counterproductive.

0 (00)

na

 Attention training

Attention training or attentional refocus tasks.

0 (00)

na

Detached mindfulness or mindfulness.

Detached mindfulness techniques within a cognitive (i.e., metacognitive) model.

0 (00)

na

Worry postponement

Worry postponement strategy is outlined.

1 (16.67)

WTm

 Table 2. Therapeutic Features: Non-GAD-Specific CBT Features Within the Apps, With the Number and Names of Apps in Which the Features Appear.

Therapeutic features

Definitions

Number of Apps, n (%)

App Name

Other CBT information

Website links to CBT information that explains what is involved in cognitive behavioural therapy, and how it works.

2 (33.33)

WTr, WTm

Distraction

List of distractions/activities intended to distract users from worry.

1 (16.67)

WTr

Information regarding external websites

Website links to further information/techniques (e.g., relaxation, breathing techniques).

2 (33.33)

WTr, WTm

Psycho education about lifestyle factors

Information about physical exercise, nutrition, caffeine, and substance use affect anxiety and mood.

1 (16.67)

FT, HYP

 Table 3. Therapeutic Features: Non-CBT Techniques With the Number and Names of Apps in Which the Features Appear.

Therapeutic features

Definitions

Numbers of Apps, n (%)

App Name

Mindfulness and meditation

Mindfulness or meditation that is not anchored to cognitive/metacognitive processes or to decrease physiological arousal.

2 (33.33)

FT, HYP

Fun content

Games or humour content to provide distraction.

1 (16.67)

HYP

Gratitude, positive affirmation, inspirational quotes

Gratitude diary/list, affirmations/creating own affirmations, or questions encouraging positive thinking about self. Quotes to provide inspiration/motivation.

5 (83.33)

WTr, FT, WW, SB HYP

Other non-CBT specific therapy

Emotional Freedom Technique (EFT).

2 (33.33)

SB, HYP

Sounds for relaxation

Audio or links to audio designed to soothe/relax (e.g., water, rain, chimes).

1 (16.67)

FT

Recorded content for education

Recorded content or links to recorded content (e.g., Ted Talks).

1 (16.67)

FT

Access to other support

Direct contact or links to support organisations.

3 (50.00)

FT, WTm, HYP

Note. Abbreviations of App names within Tables 1, 2, and 3: WorryTree (WTr), Worrytime (WTm), FearTools (FT), Happime for young people (HYP) Shrinkinabox (SB) Worry Watch (WW).

Experts and App Development

Each app was investigated for the involvement of one or more CBT experts in the development of an app’s content (i.e., a person publicly listed on that app was a mental health professional or noted as having CBT training or experience). The apps involving CBT experts (4/6, 67%) were found to hold between 2 and 10 CBT features (N = 4, = 6.75, Md = 7.5), whilst apps without CBT expert involvement ranged from 6 to 7 CBT features (N = 2, = 6.5). The app with the least CBT features (Shrinkinabox) and the app with the most CBT features (Fear Tools) both included experts in their development, although exact numbers of experts involved in each app were unclear.

Table 4. Engagement Features Within Identified CBT Apps, With Numbers of Apps in Which the Features Appeared.

Engagement features

Description

Number of Apps, n (%)

Share data from the app with others

Ability to share or export text or graphical data recorded on the app.

3 (42.86)

Notifications and reminders

Ability to schedule times of use or reminder notifications to use.

5 (71.43)

Customisation

Ability to change font, text, themes, colours, light/dark mode/ style, avatars, make own categories, playlists.

7 (100)

Games and gamification

Games to play e.g., for distraction. Gaining ‘rewards’ through progression.

1 (14.28)

Audio content

Audio recordings of information or music.

1 (14.28)

Video content

Video for instructions or content information.

3 (42.86)

Additional text content

Direct access or web links to blogs, newsletters, articles.

1 (14.28)

Statistics, graphs and/or charts

Visual representations of changes to symptoms or general progress.

1 (14.28)

Ability to add content

Ability to add pictures, videos and/or audio.

2 (28.57)

Contacts available for support

Direct contact details or weblinks to a therapist or support organisation.

4 (57.14)

Offline availability

Does not need internet or data connection to use app.

6 (85.71)

Security features

Passcodes or pins to access the app.

3 (42.86)

Engagement Features

As shown in Table 4, features that were considered to increase, sustain and/or encourage engagement within the app itself or the therapy offered within, were recorded. All apps offered some aspect of customisation, and most contained a degree of features available offline. Notifications and reminders were found in the majority of apps, while additional support such as weblinks, data shareability, security features, and video and/or other informational content were present in 50% of the apps. A minimal number of apps were found to have the ability to add user content, employed gamification, or contained audio content, graphs or statistics of changes or progress, or additional text content such as blogs.

Analysis of User Reviews of CBT Apps for GAD and Worry

When applicable, user reviews were retrieved for both Google Play and Apple iTunes versions of the identified apps (N = 120), as shown in Table 5. Three Apple iTunes apps did not have written reviews (Worry Tree, Shrinkinabox, Happime). Eight comments that were either of a general nature (Thank u; Great app love it so much), unclear if referring to functionality or therapeutic aspects (Does not work), or not specific to the use of the app (Feeling ungood at being fine) were excluded. Due to some apps being available on both platforms the resulting sample was seven apps (4 Google play, 3 iTunes) with 112 reviews for analysis. Of the reviews, 69.64% (78/112) were positive, 14.29% (16/112) were negative, 3.57% (4/112) were neutral, and 12.5% (14/112) were mixed. Three key themes of user experience were identified across all the user comments: technological experience, global experience, and therapeutic experience. The components of the themes of user experience are shown in Figure 2 and are explored in more detail below. The examples comments given relate to the predominant theme of the user’s comment, however, it is acknowledged that comments could relate to several themes.

Table 5. Numbers of Comments and Ratings for Each Individual App.

App Name

Source

App comments (N)

App ratings (N)

Average app rating (1 to 5)

WorryTree

G, A

39 (G); 0 (A)

82 (G); 1 (A)

4.1 (G); 4.0 (A)

FearTools

G, A

27 (G); 2 (A)

119 (G); 35 (A)

4.2 (G); 4.8 (A)

Shrinkinabox

A

0

0

NA

HappiMe for young people

A

0

5

4.8

Worrywatch, the habit tracker

A

49

228

4.6

ReachOut Worrytime (G)

WorryTime by ReachOut (A)

G, A

2 (G); 1 (A)

5 (G); 3 (A)

4.2 (G); 4.7 (A)

Note. Source of app ‘G’ from Google, ‘A’ from Apple iTunes. All apps were rated from 1 (lowest) to 5 (highest).

 Figure 2. Thematic Map of User Reviews.

Technological Experience

Throughout the theme of technological experience, there were 2 sub-themes uncovered: App operation and App features.

App Operation

A small group of reviews focused on the operation of an app overall. Comments tended to be negative as the use of the app was perceived to be due to development issues: “New interface is unusable. The new design is so confusing and cluttered that you can’t even operate the app. And you can’t come up with a logical interface, definitely don’t fill half the screen with upbeat mottos! A mess.” (Worry Watch)

“Where there was overall dysfunction of the app, several users indicated the negative impact this may have on their mental health: “Great way to burn $2.49. Crashes on my iPad touch. Never got it open. Actually increased my anxiety as a result. Such a shame.” (Worry Watch)

“Instructions? All I get is a blank page. Not sure this soothes my anxiety.” (Worry Watch)

App Features

Most comments relating only to a user’s technological experience mentioned app features. These comments focused on feature requests, such as: “It’s ok for when you [are] at home but why not have the meditation downloadable so you can use while you are out without data or wifi.” (Fear Tools: Anxiety Aid)

Additionally, customisation requests of features were common: “My worries usually pop up during night time and I find it very unpleasant to my eyes. Please add dark mode Edit: Still no dark mode.” (Worry Tree)

Among these customisation comments there were some that specifically mentioned security and privacy features as important: “I like the fact you can add a PIN. The UI [user interface] is also amazing (Worry Tree), and …I like that you can’t create an account so they’re not storing your data anywhere (hopefully) and it’s securely…” (Worry Watch)

Global Experience

This theme captured comments that described a mix of features and experiences and was found to span the spectrum of positive to negative sentiments. Comments described the importance of the combination of technological and therapeutic experience: “Love this app!! Simple. Effective. Easy to use and making entries is quick. Has definitely helped me put life things in perspective! Interface is clean and professional. Has not crashed. Highly recommended.” (Worry Watch)

Another comment highlighted that apps could assist in accessibility to therapy and therefore necessitate reliability:

“Ok so I have no idea what to do because every time I try to type in the Situation in the Thought Diary, it glitches then kicks me out of the app! My counselor recommended this app because I can’t go there often because of COVID-19 and it’s super hard for me to talk about my anxiety so please fix that.” (Fear Tools)

A single comment suggesting the therapeutic experience was not useful, also commented regarding the technological experience: “Star for graphic layout and for idea. No stars for delivery and execution. not actually assisting at all. Too basic.” (Worry Tree)

Even when users indicated that the app was therapeutically useful, engagement and therapeutic feature suggestions were common:

“Awesome with room to grow! I have used this app for more than two weeks and it has been very helpful in helping me identify and get in control of my anxiety… This app is not cloud-based so I’m concerned that my saved information will be deleted if I need to uninstall and reinstall this app.” (Worry Watch)

Therapeutic Experience

A large proportion of comments focused solely on the therapeutic experience of the identified apps; comments were almost exclusively positive. The sub-themes identified within this theme were: General comments of app helpfulness, app usefulness specific to anxiety and worry, and specific therapeutic strategies.

General Comments of App Helpfulness

General comments regarding the therapeutic helpfulness of an app mentioned that the app helped them feel active in their mental health: “Definitely helps reading through this APP! It relates to my circumstances really well and helps me handle the smaller things in life and be able to tackle anxiety on my own! Great app!” (Beat Anxiety)

Or assisted the user to enact change for their mental health: “Super simple yet super helpful. It helps me stay calm and realize that everything will be okay if I just take it slowly. It really deserves more reviews than it has, it’s an amazing app.” (Worry Tree)

Usefulness Specific to Anxiety and Worry

Many specifically mentioned a positive impact on their experience of anxiety and/or worry through the use of the app: “Super helpful!. This app really helps me sort out my worries and how I’m feeling. It’s really helped me manage my anxiety. I highly recommend downloading it!” (Worry Watch)

Specific Therapeutic Strategies

Whilst there were limited CBT features within the apps, many users stated their appreciation of CBT-specific strategies, such as thought diaries (first quote) and postponing worry (second quote): “I think the thought diary thing is an excellent thing...I just thought I was going to be writing my bad day down but it actually really helped.” (Fear Tools: Anxiety Aid); “Helpful. Very helpful to unload worries onto this list / app and then review the outcomes later. Helps put things in perspective.” (Worry Watch)

Some related these strategies to other therapeutic experiences in the user’s life:

“Went alongside my therapy perfectly, GAD tests very similar to what I went through with my therapist, and the app is a fantastic reminder of helpful and unhelpful thinking. Brilliant! Great tool for anyone going through or has gone through therapy.” (Fear Tools)

Discussion

The current study evaluated GAD apps, for their use of CBT, evidence-based content, and the nature of user reviews. Despite the large number of available apps (N = 1,888) proposing to assist with GAD and worry, the current study found only a small proportion of these targeted GAD (n = 26), and only 6 apps had CBT elements. This is not dissimilar to a study on apps for anxiety (Balaskas et al., 2021) that found few evidence-based CBT apps for anxiety. In the present research, the most common CBT features within the apps included identifying negative or unhelpful cognitions, addressing negative automatic thoughts, as well as instructions for using the various CBT strategies within the app. Although experts were involved with 4 of the 6 apps, the apps still varied greatly in how many CBT features were included. This highlights the difficulties the public may have in finding an app specific to GAD that provides quality, evidence-based, therapeutic strategies and supports previous research that has found that few mental health apps have a strong evidence base, including apps for GAD (Kertz et al., 2017; Marshall et al., 2020; Sucala et al., 2017; Wasil et al., 2019). Analysis of the engagement features found a limited range of features, with customisation and offline availability being the most common. The themes of ‘technological experience,’ ‘global experience’, and ‘therapeutic experience’ showed that both technological experience and therapeutic content were valuable to users’ overall experience, coherent with prior mental health app research (Alqahtani & Orji, 2019; Nicholas et al., 2017; Oyebode et al., 2020; Thach, 2018). Thus, apps that are grounded in CBT and have a ‘good’ user experience may be more likely to be seen as beneficial by users.

Design of the Apps

All apps in this study included CBT features; albeit few. The apps also included non-CBT based features, consistent with previous findings (Stawarz et al., 2018). While non-CBT features may include content that is engaging (games), non-CBT features fall outside of best practice guidelines and may or may not be consistent with CBT strategies for GAD. For example, there is insufficient evidence supporting the emotion freedom technique for GAD or indeed anxiety, yet it is being employed in apps. The lack of evidence-based therapeutic features within the apps of this study is consistent with prior, abundant research raising this concern (Alyami et al., 2017; Firth, Torous, Nicholas, Carney, Pratap, et al., 2017; Lui et al., 2017; Rathbone et al., 2017; Shen et al., 2015; Stawarz et al., 2018; Wang et al., 2018). This is disconcerting as the aim of quality self-help CBT apps is to convey core CBT principles that are consistent with therapist guided CBT such as, psychoeducation, self-monitoring, cognitive and behaviour change, problem solving, communication skills, and relapse prevention (Newby et al., 2021). Within the apps espousing they were CBT driven, few CBT strategies were offered, and often presented in isolation or lacked a comprehensive framework. That is, strategies lacked a coherent structure wherein one strategy should follow on from another in a logical, systematic, and therapeutically beneficial manner, as you would experience within a CBT treatment protocol for GAD (Borkovec et al., 2002). For example, no app offered a version of a CBT model or formulation for GAD. Formulation of a GAD CBT model is essential in therapy (Fenn & Byrne, 2013; Zivor et al., 2013), to outline maintaining factors of worry/GAD and orientate and socialise the user to the importance of the suggested CBT strategies. Other effective core interventions such as behavioural experiments, behavioural activation, metacognitive beliefs about worry, worry postponement, and problem solving were startlingly minimal or absent.

The limited inclusion or absence of CBT features may potentially be linked to contributions of CBT trained experts to the development of these apps. It was difficult to find information on who the relevant experts were and their training, which reflects findings of previous research (Oyebode et al., 2020; Stawarz et al., 2018; Sucala et al., 2017). This highlights assertions from earlier researchers that app developers should adhere to evidence-based guidelines, involve mental health professionals and mental health consumers in their development, and the necessity for regulation for standardised, evidence-based evaluations of the apps and their efficacy (Alyami et al., 2017; Oyebode et al., 2020; Sundram et al., 2017; Van Ameringen et al., 2017). Currently, consumers are left unable to determine which apps are based on evidence-based interventions to offer them any therapeutic benefit (Sucala et al., 2017). This was reiterated by Neary and Schueller (2018, p. 532) who stated, “With a bewildering abundance of apps available, clinicians and consumers will continue to face challenges in attempting to discern which are most effective, usable, engaging, or safest; how do we separate the good from the bad?”

Despite the minimal number of CBT strategies with the apps, a large proportion of the user reviews indicated these strategies were positive and beneficial, consistent with previous CBT app research (Stawarz et al., 2018; Thach, 2018). The value of including CBT, self-directed activities in the apps was noted within the current sample; this is important as CBT principles have been shown to empower individuals to understand their challenges and build skills to cope and effect change (Newby et al., 2021; Newman et al., 2020).

Given the user reviews regarding a sense of control gained through the use of strategies within the apps, it may be reasonable to consider that many users experience a sense of empowerment with just one or a few strategies. However, the longevity of gained benefits needs to be considered, as some research suggests that improvements gained through CBT apps are not maintained long term and may decline with the cessation of app use (Rathbone et al., 2017). The apps within this study had low therapy protocol adherence; consequently, lasting therapeutic gain may be questionable. Future research should seek to resolve the seeming contradiction between the limited CBT content and higher user ratings in apps, by using longitudinal designs to assess how and by how much users improve whilst using the apps for longer periods.

Mental health apps can be employed in many situations such as preparation for, or as an adjunct to, therapy, relapse prevention, as well as a standalone self-help therapy (Newby et al., 2021). Similar to Thach (2018) and Starwaz et al.’s (2018) finding that apps can augment therapy, another explanation for the perceived benefit of the apps included in this study is that some users utilised the apps as part of, or as an extension to therapy, as some indicated the similarities of features to therapeutic experiences. The tools within the apps may serve to support, remind, or enhance skills that have been acquired or are developing in therapy, therefore, the apps are not expected to be a full therapeutic intervention.

Irrespective of the number of CBT features, all apps were found to have a high user rating, mirroring previous research (Kertz et al., 2017; Stawarz et al., 2018; Wasil, 2019). Notably, Kertz’ study on anxiety and worry apps revealed that 50% of the apps had no CBT content but higher user ratings. Commonly, users consult star ratings and reviews when deciding to download apps (Mudambi & Schuff, 2010), therefore, the high star ratings for all apps in this study may be problematic for individuals looking for evidence-based apps. Star ratings are generally considered to represent the quality of the product; however, these rely upon a large number of reviews to be a meaningful communication (Tay et al., 2020). Star ratings influence the perceived credibility of a product, with high star ratings more often trusted when there are also positive written reviews (Hong & Pittman, 2020). Most of the apps in this study had small review numbers, a high proportion of positive comment reviews, and low therapeutic evidence-based features, these high star ratings may extend the difficulties of identifying apps to provide quality therapeutic support. This may place the users of these apps, who are a potentially vulnerable population, at a disadvantage as they may believe they are receiving a high quality, comprehensive CBT intervention.

The analysis of engagement features found a variable, sometimes limited number of features, similar to previous research regarding CBT apps for anxiety and depression (Balaskas et al., 2021; Stawarz et al., 2018). Interactive and tailored strategies, along with supportive (guided self-help) and social (community and/or peer connection) strategies have been identified as important in the role of engagement in online mental health interventions (Balaskas et al., 2021; Doherty, 2012; Nicholas et al., 2017). Whilst many apps contained information on how to connect to therapists or support services, no supportive or social strategies, such as peer connection forums, were identified within the apps. This may be appropriate as these apps are designed as self-help interventions and would require supervision of mental health professionals if including peer interaction.

Self-management features, such as monitoring/tracking symptoms or activity, and graph or chart features that demonstrate progress or changes, allow for interactive engagement in an app and have been found to be one of the most positively viewed features in other mental health apps (Muroff & Robertson, 2020; Thach, 2018). Balaskas and colleagues (2021) found that apps for anxiety with these self-management features were more frequently used, however, we found these features were limited within the apps of this study. Additionally, notifications, reminders, and gamification are considered forms of customisation that can substantially increase motivation and engagement in app use. Customisability has been positively reviewed and noted as a requested feature in previous mental health app research, as well as multiple customisations being found to be beneficial (Chandrashekar, 2018; Muroff & Robertson, 2020; Nicholas et al., 2017; Oyebode et al., 2020). While, customisations were the most frequent features found, multiple customisations within an app were found in just 2 apps and only one app employed gamification.

User Experience

User experience is known to impact the efficacy of an app (Baumel & Kane, 2018; Rubanovich et al., 2017; Doherty et al., 2012). Coherent with existing research, user reviews illustrated that operational dysfunction of apps can negatively impact user experience and individual’s mental health, and that app features including security and privacy, and the customisation of features are valued (Alqahtani & Orji, 2019; Nicholas et al., 2017; Oyebode et al., 2020; Thach et al., 2018). Therapeutic features were identified to assist users in feeling active and in control of managing their mental health generally, and some with their anxiety and worry in particular. CBT strategies were often highlighted as benefits of the app, with users relating them to therapy experiences, such as strategies used with a therapist previously. The largely positive therapeutic and global experience comment reviews emphasise that many perceived a therapeutic benefit through the use of an app. Even though evidence-based features were minimal, the perceived benefit from the strategies that were present may explain the high user review ratings reported here and in previous studies (Kertz et al., 2017; Stawarz et al., 2018; Wasil, 2019). Whilst minimal engagement features were identified within the apps, users highlighted these features were wanted and important to their experience. Concordant to research on apps for various mental health conditions (Alqahtani & Orji, 2019; Nicholas et al., 2017; Oyebode et al., 2020; Stawarz et al., 2018; Thach et al., 2018), we found that the user reviews for CBT GAD apps identified that the functioning of both therapeutic and engagement features impact on user experience, including access to support and their engagement in the use of the app. Considering therapeutic benefits may not be retained after discontinuation of app use (Rathbone et al., 2017), and engagement features encourage, and motivate app use (Chandrashekar, 2018), the inclusion of quality therapeutic and engagement features may be crucial to the effectual long-term use of an app for mental health support.

Limitations

A number of limitations need to be considered when interpreting our findings. First, the small number of apps suitable for inclusion limited the breadth of analysis and reflects the limited availability of CBT-based apps for GAD and worry. In addition, other therapeutic interventions, such as mindfulness, were not included in this review due to not being included in best practice guidelines for GAD. Second, the diagnoses of the users whose data was captured in this study was unknown; that is, while this research focused on CBT apps for GAD, the number of users who experience GAD in this sample was not identified. Further, few apps had substantial user reviews available, which may limit the generalisability of the resultant themes. Further, it is important to note that despite two researchers coding themes, that qualitative analyses of user reviews are limited by potential subjective biases. Assessing reviews of apps and their CBT content within the context of the apps’ costs (i.e., free versus requiring payment) was also beyond the scope of the current paper. Evaluation of engagement features was conducted by a researcher who was not a user experience researcher, arguably this may have influenced the identification of some of the apps’ engagement features. Finally, this study relied on publicly available information on app developers and anonymous user reviews rather than interviews with users, and such information and reviews may be biased or fictitious. Whilst analysis of app store reviews can expand the understanding of mental health app use, and the users’ needs and user experience (Baumel et al., 2020; Calderia et al., 2017; Stawarz et al., 2014), interviews of app users may yield greater knowledge of these aspects through purposeful and defined questioning.

Conclusions and Future Directions

The current study found there are few CBT-based apps commercially available to support GAD and worry, with limited features that promote engagement. However, users identified that therapeutic and engagement elements were appreciable to their experience of an app. The future development of apps for GAD should consider evidence-based content in the context of engaging, easy-to-use, and reliable design features to ensure a quality user experience. Additionally, user experience including an app’s design and interface needs to be prioritised. Notably, Alqahtani et al. (2019) and Oyebode et al. (2020) provide excellent guidance on improving mental health apps’ content and useability such as participatory design, high quality evidence-based content, reminders to complete therapeutic tasks, simplicity and ease of use, allowing users to customise (e.g., font size and colour, layout), ability to back-up data, data privacy, and guides on how to use the app, and regular app updates to ensure quality, up-to-date content and technology. Another area for consideration is whether apps that are advertising as CBT-based need to be regulated to ensure these claims can be substantiated and are not misleading. Further, as highlighted by Balaskas et al. (2021), an app’s consistency with evidence-based strategies does not ensure the efficacy and effectiveness of intervention elements delivered within an app. Thus, future research should consider randomised control trials to investigate the overall user experience, including long-term efficacy of symptom reduction and/or mental health maintenance, as well as comparing functionality elements to evaluate user engagement and the effect on user evaluations of an app.

Research should establish what percentage of CBT strategies makes an app compliant with best practice and what level of use is effective in reducing symptoms (i.e., like treatment adherence in clinical practice). Further, given a meta-analysis found that transdiagnostic CBT demonstrated effectiveness similar to disorder-specific CBT in the treatment of anxiety disorders (Pearl & Norton, 2017), it is worth considering if apps that are non-disorder specific to GAD (i.e., transdiagnostic) are effective for GAD. Likewise, if apps that target stress or anxiety, regardless of diagnosis, are effective for people experiencing a range of anxiety conditions, depression, or stress. Finally, the ways in which apps are used, being self-directed, an adjunct to therapy, or post therapy, are important for future research, along with the technological experience that enhances the efficacy of these apps. The ubiquitous nature of smart phones and the easy access to apps means that therapeutic assistance can be just a click away. Therefore, apps that are evidence-based and offer a high-quality user experience are much more likely to be able to offer assistance in a timely manner and contribute to better mental health and well-being. 

Conflict of Interest

The authors have no conflict of interest to declare.

Metrics

0

Crossref logo

0


256

Views

51

PDF views