Online research on COVID-19—The role of content ranking and COVID-19 fear

Introduction

Cyberchondria is defined as excessive searching for health-related content on the internet that results in heightened distress (Starcevic & Berle, 2013). The first and most cited theoretical model of cyberchondria is the reassurance model proposed by Starcevic and Berle (2013) and later adapted for the circumstances of COVID-19 pandemic (Starcevic et al., 2021). This model suggests that, while people use the internet with the intention of alleviating their distress and health anxiety, this online health research (OHR) results in even further distress, or cyberchondria, for some individuals. The reassurance model offers several reasons why people continue engaging in OHR even when it causes distress, including the large quantity of online information and need for a perfect explanation of symptoms, continuously updated information which makes it difficult to filter out unnecessary information, the ambiguity and uncertainty of online information, and difficulty in establishing the trustworthiness of online information sources. Finally, according to the model, people’s tendency to click on search results that describe fascinating and more serious illnesses means that these results are ranked higher by the search engine (Starcevic & Berle, 2013). Because this ranking is wrongly associated with the probability that the illness described in the search result is the cause of a given symptom (spurious probabilities; White & Horvitz, 2009), distress is further heightened and thus encourages continued searching for a more reassuring explanation (Starcevic & Berle, 2013; Starcevic et al., 2021).

So far, several assumptions of this model gained support. Studies reported that cyberchondria is associated with OCD (Fergus & Russell, 2016) and intolerance of uncertainty (Fergus, 2015; Norr et al., 2015) and that source trustworthiness (Baumgartner & Hartmann, 2011) and information overload (Honora et al., 2022; Laato et al., 2020) do play a role in the occurrence of cyberchondria. However, we did not find any studies exploring the role of severity ranking, an occurrence where serious and rare diagnoses appear disproportionately as search results for benign symptoms (White & Horvitz, 2009). Google found this important enough to try to address the issue, so now, when searching about COVID-19, a paragraph with presumably validated COVID-19 information is displayed before search results (Pichai, 2020).

Both the general reassurance model and its application in the context of the COVID-19 pandemic suggest that two important factors play a role in the occurrence of cyberchondria: (1) characteristics of the internet (content, information ranking, amount and quality of information) that make escalation to serious diagnosis more likely; and (2) vulnerability factor of its user (general health anxiety or COVID-19 fear, difficulties in tolerating uncertainty). Furthermore, these two factors are in complex and continuous interaction, where the user chooses how to conduct a search and which content to consume, while the information they consume may affect their health knowledge (Moreland et al., 2015), beliefs about symptoms (Marcu et al., 2019), emotional reaction (Medlock et al., 2015), and willingness to continue searching (Hämeen-Anttila et al., 2014). Similarly, search engines tailor and personalize search results based on the previous actions of a given user (Lewandowski, 2017), perhaps making it even more likely for an anxious user to come across frightening content over time.

However, it is difficult to explore both individual and internet factors at the same time. Unsurprisingly, most studies have focused on users’ self-reports about their search experiences and outcomes using cross-sectional studies. The results of these studies have provided support for the model, suggesting that highly health anxious individuals search for health information online more often and that their search sessions are longer and more often followed by anxiety (Doherty-Torstrick et al., 2016; Singh & Brown, 2014; te Poel et al., 2016). A positive correlation between cyberchondria and COVID-19 related distress has also been demonstrated (Jungmann & Witthöft, 2020; Jokić-Begić et al., 2020), suggesting that such findings might be true for COVID-19 anxiety as well. However, these studies rely on users’ perceptions and memory when reporting on their search process and do not allow for causal conclusions. Other studies have focused on exploring search data using Google trends and search logs (Effenberger et al., 2020; White & Horvitz, 2009). These studies have described the process of escalation from a benign symptom to a serious illness (White & Horvitz, 2009) and have allowed an examination of when people start searching for information about COVID-19 (Effenberger et al., 2020; Kurian 2020), the COVID-19 symptoms that are most commonly searched (Ciofani et al., 2021; Walker et al., 2020; Zhao et al., 2020), and how COVID-19 search topics change over time (Zhao et al., 2020). However, these studies do not allow us to gain insight into the psychological triggers and consequences of OHR.

Indeed, very few experimental studies exploring both aspects of cyberchondria at once have been conducted to date. One study demonstrated that reading about symptoms and illnesses was more likely to trigger a distress response than reading about exercise and diet, but only for participants who were intolerant of uncertainty (Norr et al., 2014). Another study indicated that reading about a new disease from a trustworthy online source was more distressing than from an untrustworthy source, but only for participants who were health anxious (Baumgartner & Hartmann, 2011). Finally, a thoughtful study demonstrated that, while all users experienced distress when a search for their own symptom pointed to a serious cause, this escalation in distress was more frequent in health anxious users (Singh & Brown, 2016). Together, these studies indicate that conclusions might get obscured if we do not explore both factors at the same time. Furthermore, the first two studies used content prepared by the researchers, making the search somewhat artificial. In contrast, while the last study allowed users to search in their typical, more authentic way, it is more difficult to unravel the role of content features and content ranking on users’ experiences.

As such, in order to simultaneously explore the role of user characteristics and content features, an innovative software was developed and used in the present study. While this software appears like a regular search engine to the user, it enables manipulation of the presented content and content ranking while also recording users’ online behavior. By combining the application of this software with a survey, users’ psychological reactions to the search process can also be registered.

The aim of this study was to explore the role of users’ COVID-19 fear and presentation of digital COVID-19 content (higher or lower search engine ranking of more serious content) on the search process (articles/results selected by the user and time spent on those articles) and search outcomes (experienced distress post-search and expressed wish to continue searching). To make the content relevant to both those who have had and have not had COVID-19, we decided to focus on the long-term health effects of COVID-19 that have been widely discussed in the media, such as lung damage, long fatigue, concentration problems, etc. Since specific health consequences may not be equally relevant and frightening to people of different age groups, we decided to focus on young adults in this study. Previous research has indicated that this population is more prone to OHR (Jacobs et al., 2017), more likely to experience anxiety during pandemic (Moghanibashi-Mansourieh, 2020), and that general mental health is more affected during pandemic in this group (Daly et al., 2020).

Applying Starcevic et al. (2013, 2021) model, we hypothesized (H1) that the group presented with search engine results that rank long-term health effects of COVID-19 from more to less severe (MLS) would consult articles with a higher average severity index, spend more time on articles depicting more severe consequences, exhibit more anxiety post-search, and express a greater wish to continue searching than the group presented with search engine results that rank the same articles in reversed order (LMS). Similarly, (H2) we hypothesized that participants with more pronounced COVID-19 fear would consult articles with a higher average severity index and spend more time on articles depicting more severe consequences, would exhibit more anxiety post-search and would express a greater wish to continue searching than participants with less pronounced COVID-19 related fear. Starcevic and Berle (2013) suggested that cyberchondria primarily occurs in those who are already heath anxious. Conversely, a longitudinal study suggested that cyberchondria may occur in those who were not previously health anxious (te Poel et al., 2016). Therefore, (H3) we hypothesized that the effects of severity ranking would be stronger in those with more pronounced COVID-19 fear.

Methods

Design and Setting

A quasi-experimental design was used to test the hypotheses. To select online articles about COVID-19 for the main study, a preliminary study with a different sample was conducted first.

The main part of the study took place during the third wave of the COVID-19 pandemic in Croatia (from May to September 2021). At that time, most young people in Croatia had not been vaccinated.

Preliminary Study

In the preliminary study, the authors conducted an online search for long-term health effects of COVID-19 and selected 18 articles that fit the purpose of the study (exclusion criteria: very short or very long articles, articles only partially related to the topic, outdated articles). Next, 35 graduate students of psychology (M age = 23.88 years, SD = 1.39; 21 females) rated each article via an online survey (SurveyMonkey) on several variables: severity of the long-term COVID-19 health effects described, article trustworthiness, and distress level after reading the article. All ratings were made using a sliding scale ranging from 0 to 100. Raters also measured their reading time for each article using a stopwatch and were able to leave an (optional) comment for each article. To avoid the possibility of habituation to the material, the order of article presentation was randomized between participants. Upon completion, students received course credits for their participation.

A total of 3 problematic raters (1 outlier rater and 2 raters with low item-total correlation) were excluded from the subsequent analysis of student ratings. Average item-total correlation for the remaining raters was .73. Interclass correlations were as follows: severity of COVID-19 health effects—ICC = .88 (CI .78–.95) and trustworthiness—ICC = .93 (.87–.97), indicating good to excellent rater reliability (Perinetti, 2018). Finally, based on the above-described ratings, 10 articles with the most desirable qualities were selected for the main study (excluded articles included 3 articles that were rated low in trustworthiness, 4 articles that were deemed less personally relevant and subsequently received high severity but low distress ratings, and 1 article that was notably shorter than all remaining articles). Further details about this procedure are available in the Appendix.

Materials

Online Articles

Based on the preliminary study, a total of 10 online articles describing long-term health effects of COVID-19, such as persistent fatigue, permanent lung damage, problems with concentration and memory, anxiety, and depression were used in the main study. All articles were written in Croatian and were published within several months prior to their selection. The articles contained between 355 and 576 words and took on average 1 and a half minute to read. Each article was rated based on the severity of the described COVID-19 health consequences and subsequently placed into one of 5 categories ranging from: 1 = least severe content to 5 = most severe content. There were 2 articles in each of the 5 categories. Details of each article are presented in Table 1. Articles with a higher severity index described severe health consequences (e.g., brain damage), while those with a lower severity index described less severe consequences (e.g., fatigue, cough).

Table 1. Features of the 10 Selected Online Articles About Long-Term COVID-19 Health Effects (Based on Preliminary Study With Students, N = 32).

Foogle Software

The Foogle software was developed for the purpose of this study and as part of a Master’s thesis at the Faculty of Electrical Engineering and Computing, University of Zagreb (Šaravanja, 2020). The software appears as a regular search engine to participants but allows the researcher to manipulate the links that appear as search results for a particular search term, the order in which links are presented, and the title and brief description that appear below each search result. Combining these features the researcher can create a number of different experimental conditions, and for each condition the Foogle creates a link. It is necessary to choose and input web articles to Foogle and designate an indefinite number of keywords to each. This program also records participants’ actions while searching, including each link that a participant clicks on, the order in which each link has been clicked on, and the duration spent on each article. Finally, the program allows for the search session to be time limited. Screenshots of the Foogle software can be seen in the Appendix.

The online articles listed in Table 1 and relevant key search terms (50 key words, e.g.,: COVID-19 consequences, COVID-19 long-term health problems) were uploaded onto Foogle. For the MLS group, the order of articles presented as search results went from those describing more to less severe effects, where severity ratings were: 5, 5, 4, 4, 3, 3, 2, 2, 1, and 1. For the LMS group, articles were presented in reverse order. Article titles and brief descriptions presented to the user were also adjusted within the Foogle search results to make them more consistent with the article content and severity level (where some articles had a more severe content, but a benign title and vice versa).

Measures

The Multidimensional Assessment of COVID-19-Related Fears (MAC-RF; Schimmenti, Starcevic, et al., 2020) is an 8-item measure of 4 domains of fear during COVID-19: bodily, relational, cognitive, and behavioral. Each item is rated on a scale ranging from 0 (very unlike me) to 4 (very like me). Authors reported a single-factor structure and a reliability of .84. In the present study, the recently validated Croatian version of this scale was used, for which a 2-factor structure was reported: Fear of infection and Fear of choosing an inadequate strategy when dealing with the pandemic (Bagarić & Jokić-Begić, 2022). The MAC-RF has been found to be associated with general psychopathology and to have a stronger association with health anxiety and cyberchondria than with depression, thus attesting to its validity (Bagarić & Jokić-Begić, 2022; Schimmenti, Starcevic, et al., 2020). In this study, the reliability of the two subscales was .84 and .73, respectively. This instrument was chosen because it is the only theoretically based COVID-19 distress instrument whose theory also mentions OHR as a strategy in dealing with COVID-19 distress which results in cyberchondria (Schimmenti, Billieux, et al., 2020).

The State-Trait Anxiety Inventory (STAI; Spielberg, 2000) is one of the most frequently used instruments of anxiety (Booth et al., 2016). It consists of two scales that measure anxiety as a state and trait. In this study, only the validated Croatian version of the state anxiety scale (STAI-S; Spielberg, 2000) was used. The STAI-S consists of 20 items (e.g., I feel tense), where each item is rated on a scale ranging from 1 (not at all) to 4 (very much). Participants are to provide information on how they are feeling at the present moment. Average total score for neutral situations is reported to be 38.80 for males and 40.40 for females (Spielberg, 2000). After stress induction, total scores increased, thus attesting to the scale’s sensitivity (Spielberger et al., 1970). In the present study reliability was .96.

A sociodemographic questionnaire was developed for the purpose of this study and included questions for age, sex, education level, and COVID-19 experiences. Data gathered from these questions are presented in the description of the study sample.

A post-search questionnaire was developed for the purposes of this study and included questions regarding the credibility of accessed articles (0 = entirely non credible; 100 = entirely credible), difficulty in estimating credibility (0 = not at all difficult; 100 = extremely difficult), wish to continue searching (0 = not at all, 100 = extremely), familiarity with article content (0 = not at all familiar; 100 = entirely familiar), and degree to which participants were bothered by limitations arising from using the software (0 = not bothered at all, 100 = extremely bothered). This questionnaire also asked participants to report whether they were speaking to anyone while searching and if they were using another smart device during the study (Yes/No). A single item assessed which of various long-term COVID-19 health effects participants fear the most (neurological effects, effects on the respiratory system, effects on the cardiovascular system, mental health effects, or effects on the integumentary system). Finally, using a list of 19 symptoms, participants were asked to tick all the symptoms they believed might be a long-term health effect of COVID-19 (see the Appendix). Besides the item measuring the wish to continue the search (which served as one of the outcome variables), most of the items in the post-search questionnaire served for the purpose of validity or reliability check, while the 19-symptoms list was used within an exploratory analysis. Authors could provide the interested reader with items via email.

Participants and Recruitment

Planned sample size was 73 which would, according to G*Power, allow us to detect an interaction of small to medium effect size with power of .80 (interaction effects are usually of smaller size). Participants’ inclusion criterion was having between 18 and 35 years of age, while exclusion criterion were exhibiting long-term consequences of COVID-19. We assumed that having personal experience with long-term consequences of COVID-19 might affect participants’ searching preferences, emotional reaction to COVID-19 content, and their wish to continue the search, thus obscuring the effect of studied variables.

In order to reach participants with differing levels of COVID-19 fear, study was advertised on several websites that focused on various topics (CBT counseling center, psychiatry clinic, Facebook groups for exchanging product reviews and meeting new people and the websites of 3 colleges). Potential participants were invited to participate in a study exploring how people search for COVID-19 on the internet and were informed that the study would consist of filling out questionnaires, searching about COVID-19, and reporting their reactions to searching. Data collection went slower than anticipated. When epidemiological situation in the country started to improve, with no new participants being enrolled in 2 weeks, data collection was stopped due to concern that changes in epidemiological situation might affect the results.

A total of 76 adults (community sample) initially participated in the main study. For 11 participants, an entire matrix regarding their search data was missing, and for another 2 participants, the outcome variables’ scores were missing. As such, all data from these participants were omitted from the data set. One participant was excluded due to still suffering from COVID-19 health consequences (memory and concentration problems). In addition, a single multivariate outlier was detected (Mahalanobis distance = 42.05; p < .001) and subsequently omitted from the data set.

Final sample consisted of 64 young adults (M_age = 26.15 years, SD = 4.98, TR 19–35), of which 48 were female. With respect to their education, 20 participants held a high school diploma, 9 held a bachelor’s degree, 33 held a master’s degree and 2 participants held a doctoral degree. Only 2 participants reported being employed in the health care system. A total of 10 participants reported having a chronic health condition (epilepsy, rheumatoid arthritis, asthma, Chron’s disease, Hashimoto, PCO, gastritis, panic disorder and OCD). Most participants reported not knowing anyone (N = 19) or knowing a single person (N = 16) who had recovered from COVID-19 but that still suffered health consequences (M = 2.80, SD = 4.32). A total of 28 participants had never been tested for COVID-19, 27 had been tested as negative, and 9 were tested as positive at least once. Participants in the latter group estimated their COVID-19 symptoms to have been mild.

Procedure

The study was conducted online and consisted of 3 stages (Figure 1). First, participants completed the first part of the online survey (SurveyMonkey) that consisted of the sociodemographic questionnaire and the MAC-RF. Upon completion, participants were instructed to click on a link that would open a search engine with which they were to conduct a search for long-term health effects of COVID-19. In the meantime, the initial survey remained open in a separate tab. Participants were also instructed to open the pages one at the time. Second, participants clicked on the link that took them to the search engine (Foogle), having been informed that this was a search engine developed for research purposes (due to ethical reasons). Following the participants’ search for information on long-term COVID-19 health effects, the online articles described earlier appeared in one of two possible orders: MLS or LMS (randomized between participants). Participants had 5 minutes to complete their search but were unaware of the exact time limit. This limit was determined based on the preliminary study and was used to prevent participants from reading all the articles. When 5 minutes elapsed, participants were presented with a message on the screen announcing that the allotted search time had elapsed and were instructed to return to the original survey (SurveyMonkey). Finally, participants completed the second part of the survey, which included the STAI-S, the post-search questionnaire, and a validity check item. To allow pairing of data between SurveyMonkey and Foogle, participants entered a code name into both programs. At the end of the survey, participants were cautioned that online health content can be of varying quality and were directed toward several high quality COVID-19-related web pages and to several sources of support in the event that they were feeling distressed about COVID-19.

This study was approved by the Ethical committee of the Department of Psychology at the Faculty of Humanities and Social Sciences, University of Zagreb (EPOP–2021–004).

Figure 1. The Sequence of the Study Procedure.

Data Analyses

Data were analyzed using SPSS 26 (IBM Corporation, 2019). First, a total score for the MAC-RF was calculated. Using this score, participants were divided into two groups (high and low COVID-19 fear) according to the cut-off proposed by the scales’ authors, who found that a score of 12 suggests high levels of current psychopathology (Schimmenti, Starcevic, et al., 2020). Participants in the low COVID-19 fear group scored M = 5.90 (SD = 3.50) on the MAC-RF, while those in the high COVID-19 fear group scored M = 17.04 (SD = 3.74).

Time spent on more severe content was operationalized as the total time spent on the four articles with the highest severity rating (5 and 4 severity level in Table 1). Similarly, time spent on less severe content was operationalized as the total time spent on the four articles with the lowest severity rating (1 and 2 severity level). Pages with medium severity rating were omitted since these are not needed to test the hypothesis. Including only the extremes provides more power to the analyses.

Differences in the mean severity index of all accessed pages between groups with differing COVID-19 fear levels (high vs. low) and experimental condition (LMS vs. MLS) were tested using a 2x2 ANOVA. Differences in time spent on articles based on article severity (more severe vs. less severe health content—within subjects), COVID-19 fear levels (high vs. low), and experimental condition (LMS vs MLS) were calculated using a mixed 2x2x2 ANOVA. Differences in state anxiety post-search (measured using the STAI-S) and the wish to continue searching between groups with differing COVID-19 fear levels and experimental condition were tested using a 2x2 MANOVA (because these two outcomes are theoretically related and represent two aspects of cyberchondria).

Results

Preliminary Analysis, Manipulation, and Validity Check

All dependent variables were normally distributed (Kolmogorov-Smirnov Z = 0.91–1.92, all p values >.001). A total of 7 participants reported talking to someone while participating in the study and none reported using another smart device while participating. There were no differences in estimates of article credibility or participants’ reported difficulties in estimating credibility between experimental conditions, multivariate: F(2, 59) = .02, λ = .97; p = .453, η² = .03, nor between participants with differing levels of COVID-19 fear, multivariate: F(2, 59) = 1.56, λ = .95, p = .222, η² = .05. In general, participants rated the articles to be moderately credible (M = 49.22, SD = 23.96) and that it was moderately difficult to make this estimate (M = 54.70, SD = 26.23). There were no differences in reported familiarity with the content of the articles between experimental conditions, F(1, 64) = 0.19, p = .887, η² < .001, nor between participants with varying levels of COVID-19 fear, F(1, 63) = 0.05, p = .819, η² = .007. On the whole, participants reported being moderately familiar with the content of the articles (M = 55.12, SD = 28.43) and only mildly bothered by the limitations placed on online searching during the study (using this particular search engine, searching in Croatian only, opening only one page at the time, etc.; M = 37.77, SD = 38.36).

Descriptive data is presented in Tables 2 and 3. A review of this data indicates that participants were most afraid of the neurological consequences of COVID-19, followed by consequences related to the cardiorespiratory system. This is consistent with ratings from the preliminary study, thus attesting to the reliability of this data.

Table 2. Descriptive Statistics for the Main Outcome Variables Displayed
Separately for COVID-19 Fear Groups and Experimental Conditions: Average Severity
of Accessed Web Articles, Time Spent on Articles Describing More and Less Severe
Long-Term COVID-19 Health Effects and State Anxiety Post-Search (STAI-S), N = 64.

Table 3. Most Feared Long-Term COVID-19 Health Effects Reported
by Participants With Low and High COVID-19 Fear.

Search Process

Participants accessed an average of 3.14 articles during their search (there were no group differences). Only two participants accessed the same page twice. ANOVA revealed that participants in MLS condition accessed articles with a higher mean severity index than participants in LMS condition, F(1, 63) = 16.75, p < .001, η² = .19—large effect size; M_MLS = 3.71, SD_MLS = 1.01, M_LMS = 22.62, SD_LMS = 1.19. The mean severity of the articles accessed by participants with different levels of COVID-19 fear did not differ, F(1, 63) = 0.028, p = .874; η² < .001, nor was there a significant interaction effect, F(1, 63) = 0.00, p = .987, η² < .001.

Repeated measures 2x2x2 ANOVA revealed a significant two-way interaction between articles with different severity level and experimental group, F(1, 60) = 5.10, p = .028, η² = .08—medium size, and a significant main effect of content type, F(1, 60) = 4.83, p = .032, η² = .08—medium size, meaning that participants in MLS condition spent significantly more time on articles discussing more severe health effects than those depicting less severe health effects (M = 106.59, SD = 105.94; M = 23.08, SD = 36.06 respectively, see Figure 2). However, participants in the LMS condition spent an equal amount of time on more and less severe content (M = 58.24, SD = 85.54; M = 64.05, SD = 70.47, respectively). Main effect of COVID-19 fear was not significant, F(1, 60) = 5.10, p = .028, η² = .08, nor was COVID-19 fear and experimental situation interaction, F(1, 60) = .006, p = .940, η² < .001, neither a 3-way interaction effect, F(1, 60) = 2.33, p = .134, η² = .04.

Figure 2. Time Spent on Pages Describing More and Less Severe COVID-19
Content Depending on Search Results Ranking (Mls-More to Less Severe Content,
Lms-Less to More Severe Content, N = 64).

Search Outcomes

Participants with high COVID-19 fear reported feeling more anxious post-search than those with low COVID-19 fear, multivariate F(2, 59) = 16.76, λ = .64, p < .001, η² = .36; univariate F(1, 60) = 28.21, p < .001, η² = .32—large effect size; M_H = 47.77, SD_H = 11.29; M_L = 32.12, SD_L = 10.86, and expressed a greater wish to continue searching F(1, 60) = 4.06, p = .048, η² = .06—medium effect size; M_H = 20.64, SD_H = 26.21; M_L = 9.48, SD_L = 15.55. No differences between experimental condition groups were found for post-search anxiety or wish to continue searching, multivariate F(2, 59) = 0.27, λ = .99, p = .785, η² = .009, nor was there a significant interaction, multivariate F(2, 59) = 0.05, λ = .99, p = .951, η² = .002.

Hence, our hypotheses were only partially supported. (1) The MLS group, compared to LMS group, did consult articles with a higher average severity index, did spend more time on articles depicting more severe consequences, but did not exhibit more anxiety post-search, nor expressed a greater wish to continue searching. (2) Participants with higher COVID-19 fear, compared to those with lower COVID-19 fear, did exhibit more anxiety post-search, and did express a greater wish to continue searching, but did not consult articles with a higher average severity index, nor did they spend more time on articles depicting more severe consequences. Lastly, (3) we did not find that participants with higher COVID-19 fear were more affected by the severity ranking.

Finally, exploratory analysis revealed there were no differences in the number of different health consequences that participants endorsed as possible long-term health effects of COVID-19 between participants with different levels of COVID-19 fear, F(1, 63) = .44, p = .434, η² = .01, nor between experimental conditions, F(1, 63) = .001, p = .99; η² < .001, nor was there a significant interaction, F(1, 63) = .001, p = .99, η² < .001. Lung damage, fatigue, permanent loss of smell, and concentration and memory problems were the health issues most often endorsed as possible long-term effects of COVID-19. A small number of participants endorsed symptoms not mentioned in any of the articles (e.g., gastritis, muscle twitching).

Discussion

The aim of this study was to explore the influence of COVID-19 fear and ranking of COVID-19 health content based on its severity on users’ internet search process and outcomes. The results suggest that ranking of search results affects the content that will eventually be accessed by the user, where users are more likely to open pages ranked higher by the search engine. This finding is consistent with those from previous studies (Eysenbach & Köhler, 2002; van Deursen & van Dijk, 2009). Furthermore, this study expands upon previous findings suggesting that such ranking-based selection of articles was true regardless of participants’ fear of COVID-19 and regardless of the severity of the content of the article ranked first (i.e., low or high severity).

However, once users have opened a particular online article, their process becomes more deliberate. Namely, if more severe content is presented first, users spent more time on this content, whereas they spent an approximately equal amount of time on both types of content if less severe content is presented first. This suggests that, although users generally elect to open pages ranked higher in the search results, they are more likely to keep reading if these pages are depicting more severe content. This finding is consistent with the proposed reassurance model (Starcevic & Berle, 2013), in which authors present anecdotal data about human preferences for fascinating and serious illnesses. The present study offers experimental support for this assumption and further demonstrates this preference among users with both high and low levels of COVID-19 fear.

It is important to note that the amount of time spent on different types of articles was highly variable among the participants in this study. Such variability has similarly been reported for search queries and session duration in previous research (Buhi et al., 2009; Hansen et al., 2003; White & Horvitz, 2009) and may be indicative of real variability in search strategies across many people or in separate searches conducted by the same individual. At the same time, however, this variability reduces power to detect differences, thus maybe obscuring some findings.

Finally, search outcomes appear to be more affected by participant characteristics (intensity of COVID-19 fear) than by consumed content. Participants more worried about COVID-19 were more distressed post-search and expressed a greater wish to continue searching than those with lower levels of COVID-19 fear, a finding that is consistent with the reassurance model (Starcevic & Berle, 2013; Starcevic et al., 2021) as well as with previous studies examining general health anxiety (Doherty-Torstrick et al., 2016; Muse et al., 2012; Singh & Brown, 2014) and COVID-19 anxiety (Jungmann & Witthöft, 2020). However, this difference cannot be attributed to the content opened by the user, nor to the content on which the user spent more time, because no such differences between users with different level of COVID-19 fear were found in this study. This suggests that the way people attend to, interpret, and recall online health content might be more important in post-search outcomes. For example, health anxious individuals exhibit an attentional and memory bias for threatening health information (Witthöft et al., 2016) and perceive themselves at greater risk of getting ill (Baumgartner & Hartman, 2011).

In general, however, our participants did not express very high interest in continuing the search. This may have been because participants were conducting the search on a topic and at a time determined by the researchers. Since those more fearful of COVID-19 expressed a greater wish to continue searching, it seems reasonable to expect that, if left to continue, they would have continued searching for a longer time and spent more time on more severe health content as a result of a general preference for such content. Consequentially, actual differences in post-search distress might be even more pronounced than those observed in the findings presented here. Further research is necessary to explore the factors that enable less anxious individuals to complete their search even in light of frightening information.

Taken together, results of this study provided support for several elements of Starcevic and Berle’s model of cyberchondria; participants were more likely to open highly rated pages, exhibited a preference for more severe health content, those with higher COVID-19 fear felt more anxiety after searching and wanted to continue searching about COVID-19 long-term consequences more. Furthermore, as suggested by the model, results show that both content ranking and COVID-19 fear play a role in the occurrence of cyberchondria. We did not find a significant interaction between these factors, but instead found that they became important during different stages of OHR. Future studies should experimentally explore other factors included in the model, such as information overload and constantly changing information, and continue to examine the interaction effects of vulnerability factors and online content features on both process and outcomes of OHR. We also hope that this study encourages researchers to study cyberchondria using the same digital medium in which cyberchondria occurs, in new and creative ways.

In is important to note that in this study cyberchondria was operationalized as an occurrence of anxiety post-search and willingness to continue searching, which arises from the reassurance model and is typical in experimental studies and surveys associated with this research stream (Baumgartner & Hartmann, 2011; Norr et al., 2014; Singh & Brown, 2016). In contrast, psychometric research stream operationalizes cyberchondria through questionnaires, mostly Cyberchondria Severity Scale (CSS; McElroy & Shevlin, 2014), and studies it using correlational design. This distinction has important consequences on interpreting our findings; they pertain to the “single occurrence of cyberchondria” and cover Distress and to some degree Excessiveness factors of CSS, but not Reassurance seeking, nor Compulsiveness factors, since they imply long-term consequences of searching. These two research streams complement each other, one exploring under which conditions cyberchondria may arise, and the other exploring its correlates.

The findings of this study imply that ranking valid and balanced health content more highly by the search engine may have only a limited effect on the search process and outcomes. Indeed, it seems that people tend to “correct the search” when less severe content is presented first. Perhaps simply being aware of this tendency can be helpful to a certain degree. Providing education to users about how to choose high quality content and how to appraise and use content might also help avoid escalations in searching. For those already fearful of COVID-19 and who therefore feel more distressed regardless of the content they consume, it is essential that interventions target their heightened fear of COVID-19 (Schimmenti, Starcevic et al., 2020). The context of pandemic encourages OHR (Badell-Grau et al., 2020; Du et al., 2020; Neely et al., 2021) and features of online COVID-19 content probably makes cyberchondria more likely (Soroya et al., 2021). Encouragingly, existing health anxiety treatments also seem to be helpful in the time of pandemic (Sharrock et al., 2021). However, the epidemiological situation needs to be viewed from a process perspective, where numbers of infected individuals and the stringency of imposed restrictions are constantly changing, as do the health topics of public interest—COVID-19 symptoms, long-term health effects or vaccination side effects (Jokić-Begić & Bagarić, 2021). These changes can dramatically influence that which is researched online (Zhao et al., 2020), by whom and with what outcomes.

The findings presented here should be considered alongside several limitations of the study. Firstly, the data is based on experiences of participants who were not researching their own symptoms online, nor were they conducting the search at a time at which they felt the need to do so. However, OHR can be triggering even for people with low health anxiety, presumably when they come across frightening information by chance (te Poel et al., 2016). Secondly, the long-term effects of the online search conducted in this study might be minimal if one assumes that only a large amount of OHR over a prolonged period can lead to psychopathology. Indeed, this study only sheds insight into what occurs during a single search session. Thirdly, the online search conducted by participants in this study was somewhat different from a typical OHR, where participants might, for example, also search for results in English, search for a longer period, and open several pages at the same time. Also, content ranking in this study does not reflect actual ranking by a search engine, which is certainly based on a more complex algorithm, but is only used to test a specific hypothesis. Fourthly, the sample size was smaller than initially planned which could affect the power to detect interaction effects which are usually of a smaller effect size. Furthermore, the decision to dichotomize the COVID-19 fear variable may be controversial since it diminishes power. However, the scale’s authors reported that the score above the cut-off is associated with high level of current psychopathology (Schimmenti, Starcevic et al., 2020) suggesting that the two levels of COVID-19 fear may be qualitatively different. Finally, this study included only young adults and therefore the results cannot be generalized to other populations. For example, it is possible that the COVID-19 health effects deemed more severe would be different for older adults and the elderly. Also, search strategies used by these groups are known to be different from those of young adults in many aspects (Bagarić & Jokić-Begić, 2020; Sanchiz et al., 2017).

To conclude, this study points to several factors with an important role in different stages of online searching about COVID-19. Specifically, the ranking of pages appears to be most important in determining which pages will be accessed by the user, while content, i.e., severity of the described COVID-19 health consequences, appears to determine the amount of time users will devote to a given article. Finally, users’ previous level of COVID-19 fear seems to determine their emotional reaction to the search and their willingness to continue searching.

Conflict of Interest

The authors have no conflicts of interest to declare.

Acknowledgement

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Appendix

Online Articles About COVID-19: Selection Procedure

The authors of the study selected a sample of 20 online articles about long-term COVID-19 health effects (e.g., fatigue, memory problems). To determine whether there was a meaningful range of severity across all articles and to assess level of agreement, the authors and an administrative assistant rated the severity of the health effects discussed in each article. The ICC was .70, CI .36–.87 (model: two-way random; type: consistency; definition: average measure). Discussion following this assessment indicated that, as expected, some of the differences in severity ratings between coders were due to the differing ages of the coders (aged: 33, 43 and 56 years). Therefore, younger participants were subsequently included as both raters and participants in the main study.

Because a period of 3 months elapsed from initial article selection to study preparation, during which time new articles might have been published, the authors conducted a second search for relevant articles. Very short or very long articles (less than 250 or over 600 words), those published more than 6 months prior to the search and those only partially related to this topic were excluded. A total of 18 articles were selected.

These 18 online articles were rated by 35 graduate students of psychology via an online survey. For each article, raters used a slider scale ranging from 0 to 100 to assess severity of long-term COVID-19 health effects, trustworthiness of the article, and distress level. For each article, raters measured their reading time using a stopwatch and were able to write a comment (optional). To avoid the possibility of habituation to the material, the order of article presentation was randomized between participants.

Following data collection, authors checked for problematic raters. While there were no multivariate outliers, we identified 7 univariate outliers, which were all provided by a single rater who was subsequentially excluded from further analyses. The data matrix was transposed (i.e., raters became variables) to check rater reliability. For two raters, item-total correlation (correlation between their codes and all other raters’ codes) was below the recommended value of .30 (Nunnally et al., 1994; .24, .26). Hence, these raters were also excluded, leaving a final total of 32 raters. In this group, the lowest item-total correlation was.47 and the average item-total correlation was .73. To determine raters’ average reliability, we calculated the interclass correlation (model: two-way random; type: consistency; definition: average measure). The results were as follows: severity of COVID-19 health effects ICC = .88 (CI .78–.95) and trustworthiness ICC = .93 (.87–.97), indicating good to excellent reliability (Perinetti, 2018).

In the last stage of article selection, ten online articles with the most suitable characteristics were selected from the group of 18 articles. Three articles were excluded due to low trustworthiness ratings (mean rating below 60). Next, four articles were found to have a notable rank difference between severity and distress ratings (3 and 4 ranks, compared to 0, 1 and 2 ranks for the selected articles). Raters’ comments suggested that these articles were deemed less personally relevant (e.g., the described COVD-19 health effects, although severe, concerned only patients who were hospitalized or had prior mental illness), and therefore had a high severity rating, but caused little distress. Of the remaining 11 articles, one additional article was excluded based on shorter reading time (mean reading time 56.94, compared to 70.34–108.66 for remaining articles).

The characteristics of the final 10 articles used in the main study are presented in Table 1.

List of Potential COVID-19 Long-Term Health Effects Presented to Participants Within the Post-Search Questionnaire

1. fatigue
2. lung damage
3. difficulties concentrating
4. memory problems
5. dermatitis
6. anxiety
7. depression
8. gastritis
9. digestion issues
10. loss of hearing
11. loss of vision
12. muscle pain
13. headache
14. muscle twitching
15. loss of smell
16. stroke
17. osteoporosis
18. hair loss
19. tooth loss

Figure A1: Screenshots of the Foogle Software—Administrator Dashboard

Figure A2: Screenshots of the Foogle Software—User Screen