False consensus in the echo chamber: Exposure to favorably biased social media news feeds leads to increased perception of public support for own opinions

Introduction

There is a growing debate in media, society, and science whether online social networks like Facebook or Twitter facilitate biased information consumption and opinion formation, and, in turn, give rise to negative consequences like misinformation (Del Vicario et al., 2016), radicalization, and societal polarization (Grömping, 2014; Williams et al., 2015). Studies have found users of online social networks showing the same tendencies for confirmation bias (Wason, 1968) and homophily that can also be observed in offline settings: They often prefer information that is consistent to their own attitudes (e.g., Knobloch-Westerwick, 2014) and form ties with like-minded others (Del Vicario et al., 2016; Quattrociocchi et al., 2016; Schmidt et al., 2017; Zollo et al., 2017), especially in regard to highly political topics (Barberá et al., 2015). Online communication showing these tendencies has been characterized as “Echo Chamber” (Gilbert et al., 2009; Quattrociocchi et al., 2016; Sunstein, 2001), as similar attitudes and information supporting these attitudes appear to “echo” between users, and deviant opinions are excluded. The emergence of Echo Chamber-like communication structures comes as no surprise, as humans have long been known to reduce cognitive dissonance by preferring selective exposure to attitudinally congruent information (Cotton & Hieser, 1980; Hart et al., 2009; Knobloch-Westerwick, 2014; Sears & Freedman, 1967) and to preferably connect with similar others (McPherson et al., 2001). Online media may further strengthen these biases by letting users engage with a permanently available stream of information that is more personalized than traditional, asynchronous media, often devoid of editorial judgement and quality control (Allcott & Gentzkow, 2017) and algorithmically tailored to the users’ preferences and usage patterns (Beam, 2014; Pariser, 2011).

However, it must be noted that the prevalence of Echo Chamber-like communication structures is subject to controversial debate (for a short overview see Guess et al., 2019): While some studies, such as a recent analysis of 14 million Facebook users' interactions with news sites over the span of six years (Cinelli et al., 2020), present compelling evidence for online users' preference for selective exposure and their segregation in Echo Chambers, others do not. For example, only a minority of Facebook and Twitter users describe their network as mostly comprised of like-minded others (Duggan & Smith, 2016), a self-report that is backed by an analysis of Australian Twitter users’ likelihood to be exposed to attitudinally congruent and incongruent contents (Bruns, 2017), and a recent study based on Eurobarometer survey data could not link self-reported network homogeneity and political polarization regarding attitudes towards the European Union (Nguyen & Vu, 2019). These findings, nonetheless, are not necessarily in contradiction to evidence of users’ preference for attitudinally congruent information and homophily. Instead, the current state of research demonstrates that online networks differ significantly in regard to their degree of homophily, with both Echo Chambers and more heterogeneous patterns emerging. Whether communication and news consumption take the form of an Echo Chamber appears to be influenced by structural variables like the existence of opinion leaders among whom like-minded crowds gather (Guo et al., 2020) and individual variables like the degree of involvement with a topic (Dubois & Blank, 2018). Different operationalization and measurement of Echo Chambers (e.g., via self-reported news exposure or network composition, via liking and retweeting behavior or via observed website dwell times) may further explain the observed discrepancies.

In summary, while Echo Chambers may not be a necessary consequence of online networking and news consumption, communication structures in which users are selectively exposed to attitudinally congruent information can emerge and exist in online contexts. While further research on the prevalence of Echo Chambers and their connection to factors like societal polarization is important, the current paper focuses on another equally crucial yet still little researched aspect: How does confrontation with artifacts encountered in Echo Chambers affect the individual user? Concretely, two experiments tested whether exposure to attitudinally biased online news feeds influences participants’ False Consensus Effect, that is, their tendency to assume high public support for their own opinions.

Links Between Echo Chambers and Perceived Public Support for Own Opinions

It is known that the perception of public opinion or social norm is an important predictor of actual behavior (Ajzen & Fishbein, 1977; Fishbein, 1976; Liska, 1984): For example, individuals who hold radical political views but perceive the public in disagreement may not act upon them. However, if they perceive the public in support of their views, their inhibitions to show hostile or aggressive behavior may be reduced.

Research has also demonstrated that individuals do not estimate public opinion objectively. Instead, they tend to perceive public opinion as favorably distorted towards their own beliefs. This tendency is called the False Consensus Effect (FCE, Ross et al., 1977): Given two options (e.g., “Yes” or “No” regarding approval to the statement “Marihuana should be legalized.”) the percentage of the population in favor of one option is overestimated by individuals in favor of this option as compared to individuals in favor of the other option, and vice versa. For example, individuals supporting marihuana legalization might estimate that 45% of the population also support it, while individuals strongly opposed to legalization might estimate that only 25% favor legalization. The difference of 20% reflects the extent of the FCE, that is, how strongly the estimate of public opinion depends on individuals' own opinion.

Previous studies suggest that FCE is correlated with participants’ selective exposure to attitudinally congruent information and personal network homogeneity: For example, Bauman and Geher (2002) measured participants’ FCE after exposure to packages of information (e.g., brochures, pamphlets or video-taped discussions). Participants exposed to balanced packages that contained both information supporting and questioning their own opinions displayed a lower FCE than members of the control group without any exposure. This can be interpreted as an effect of selective exposure: In everyday life, individuals prefer congruent information from which they infer high public support for their opinions. However, when exposed to different views on a topic, individuals are forced to also process incongruent information that they would not normally seek out, leading to a lower FCE.

That an effect of selective exposure on FCE might also occur by participating in homogenous online groups is illustrated by Wojcieszak (2008) who examined FCE of members of neo-Nazi and radical environmentalist online forums. In the case of the neo-Nazi forum, the study found FCE positively correlated with the participants’ degree of forum participation as well as their level of extremism (however, in case of the environmentalist forum, only extremism, not forum participation correlated with FCE).

Given these findings, it stands to reason that perceived public support for one’s own opinion should also be influenced by the resemblance of a user’s network to an Echo Chamber: If individuals surround themselves online with others that mostly share their attitudes, they should receive mostly messages with congruent information, and they should experience little interactions with others holding opposing beliefs. This should lead to a stronger FCE. Indeed, a prior correlative questionnaire study (Luzsa & Mayr, 2019) found such a link: The more homogenous participants described their online social network, the stronger their FCE, measured for twenty current political topics. Due to the correlative nature of that study, however, the cause for this stronger FCE could not be unambiguously identified. The assumption that FCE is influenced by selective exposure to attitudinally congruent information shared in networks still requires testing. For this, an experimental approach is necessary that examines how the attributes of messages shared in online social media affect FCE.

Biased News Feeds as Artifacts of Online Echo Chambers

Typical for online social media are news feeds consisting of messages that other connected users have shared. Each message consists of central content (e.g., headlines of news articles or personal commentaries), accompanied by popularity cues, that is, indicators which illustrate how many other users positively evaluated or endorsed the content, for example by “liking” it (Haim et al., 2018; Porten-Cheé et al., 2018). While there are also additional message attributes, such as sender names and images, this study will focus on popularity cues, in addition to message content because effects of popularity cues on outcomes like attention (Dvir-Gvirsman, 2019) and selection and appraisal of messages (Chang et al., 2015; Haim et al., 2018; Messing & Westwood, 2014) are well documented.

It can be argued that message content and other users’ endorsement via “likes” are key attributes that differentiate between biased news feeds originating from Echo Chambers and feeds originating from more balanced, heterogeneous online communication: Firstly, news feeds of users with homogenous networks should express agreement with their own attitudes, that is, consist mostly of attitudinally congruent messages. Secondly, these attitudinally congruent contents should display strong endorsement by users' networks, while occasionally occurring attitudinally incongruent contents should show less endorsement. In contrast, if users’ networks are heterogeneous there should be neither dominance of attitudinally congruent messages nor higher endorsement for congruent than incongruent ones. In fact, depending on topic and network, users might even encounter mostly incongruent messages as well as popularity cues that display low endorsement for their own attitudes – for example in communities that emphasize controversial discussions (Guest, 2018).

Hypotheses

Based on these considerations, two hypotheses regarding the effect of Echo Chamber news feeds on FCE are formulated:

Effects of agreement: Participants that are exposed to a news feed made up of messages mostly congruent to their own attitudes will display a stronger FCE, compared to participants exposed to mostly incongruent messages.

Effects of endorsement: If messages congruent to participants’ own attitudes display higher endorsement by others than incongruent ones, participants will exhibit a stronger FCE, compared to the situation in which incongruent messages display higher endorsement than congruent messages.

Additionally, both agreement expressed in message texts and endorsement expressed by “likes” may vary independently of each other: For example, users might read a feed in which 90% of the messages are congruent to their own attitudes. A positive Echo Chamber effect on FCE should occur. However, what happens if the remaining 10% of incongruent messages display significantly higher numbers of “likes”? Will this reduce the positive effect of message agreement on FCE? In the opposite case, users read a feed with mostly incongruent messages. This should lead to a weaker FCE. However, if the few attitudinally congruent messages display the strongest endorsement, will users interpret this as a “silent majority” agreeing with them, and therefore display a stronger FCE?

Previous research does not allow assuming whether agreement or endorsement is the pivotal factor and whether there will be an interactive effect. Therefore, an open research question is formulated:

Interactive effect: Is there an interactive effect between agreement and endorsement on FCE?

Finally, in tradition of the Elaboration Likelihood Model (Petty & Cacioppo, 1986), popularity cues such as “likes” may be conceived as peripheral, message contents as central cues. The model states that when participants’ involvement is high, they will be mostly affected by central cues, while low involvement leads to stronger effects of peripheral cues.

This might imply that participants’ interest in a topic moderates the effects of agreement as well as endorsement on FCE: Regarding the role of agreement, if participants have little interest in a topic they might put less effort in processing messages related to it, with message content showing little effect on FCE. However, if interest is high, participants might put more effort in reading and evaluating messages, leading to a stronger effect of message content than when interest is low. Regarding the role of endorsement, participants who have little interest in a topic might focus on popularity cues such as “likes” as an effortless way to estimate public opinion. In contrast, the effect of “likes” on FCE should turn out weaker in case of high interest, as higher interest might make participants more skeptical regarding the representativeness of popularity cues.

From this follows the final hypothesis:

Moderation by interest: The effects of agreement as well as of endorsement on FCE are moderated by participants’ interest in a topic. The effect of agreement will turn out stronger in case of high interest than when interest is low. The effect of endorsement will turn out stronger in case of low interest than when interest is high.

To test these hypotheses, controlled experimental approaches are necessary which expose participants to online environments with either high or low Echo Chamber characteristics and then measure their effects on the individual. There is some experimental research which the present study can build upon: For example, Giese et al. (2020) tested how congruent and incongruent information is perceived and shared in attitudinally homogenous and heterogeneous groups of participants. Related approaches are also found in selective exposure research, where participants are asked to indicate attitudes regarding topics and then are exposed to attitudinally congruent and incongruent news while measures of exposure like viewing times or clicking behavior are observed (e.g., Knobloch-Westerwick & Meng 2009). However, to the authors' best knowledge, there are no studies that have specifically focused the effects of agreement and endorsement in Echo Chamber-like online news feeds on perceptions of public opinion.

Therefore, two online experiments were conducted to test the hypothesized effects.

Experiment 1

Experiment 1 adapted selective exposure paradigms (Knobloch-Westerwick & Meng, 2009; Lee & Jang, 2010; Messing & Westwood, 2014; Peter et al., 2014) in order to confront participants with biased Echo Chamber news feeds: Participants were exposed to nine news feeds, each consisting of four simultaneously presented messages, that is, short news headlines regarding one topic. A number of “likes” accompanied each message. The participants' task was to select the message whose linked full article they preferred to read. No full articles were displayed afterwards. The task was merely given to ensure that participants read and processed all the messages.

The attributes of biased Echo Chamber news feeds – agreement expressed in messages and endorsement expressed by “likes” – were independently manipulated: Participants were exposed to either three messages congruent to their own attitudes and one incongruent (condition high agreement) or to three incongruent and one congruent (condition low agreement). Similarly, either the congruent messages had high and the incongruent messages low numbers of likes (condition high endorsement) or vice versa (condition low endorsement). The news feeds reflecting these conditions were created during the runtime of the experiment based on the initially assessed own attitudes of the participants.

An example: If a participant favors the legalization of marihuana, Figure 1 illustrates a news feed that reflects low agreement (most messages are incongruent as three of four highlight the dangers of marihuana) and low endorsement for his or her own attitude (the three anti-marihuana messages have higher numbers of “likes” than the one in favor of legalization).

Figure 1. Example of a News Feed Used in Experiment 1.

Method

Sample

A self-administered online experiment was conducted with 388 German participants, recruited on the campus of the University of Passau and from the authors’ volunteer database. Data collection took place in May 2018. This and the following experiment were conducted in accordance with the ethical guidelines of the German Psychological Association and the Professional Association of German Psychologists (BDP & DGPs, 2016) and with the 1964 Declaration of Helsinki (WMA, 2013). Participation was voluntary and participants were fully informed and debriefed after the experimental manipulation.

Implausible cases were excluded based on completion times: First, participants who took less than 5 or longer than 60 minutes to complete the experiment were dropped. Then, only cases within 2 SD of the resulting mean completion time were kept. The final mean completion time was 12.03 minutes (SD = 4.57, Min = 5.15, Max = 28.52). The remaining sample comprised 331 participants (231 female; age between 18 and 35 years with M = 22.36 and SD = 0.27). An a-priori power analysis with G*Power (Faul et al., 2007) determined that at least 195 participants were necessary to achieve a power of 1–β = .95, given α = .05 and a medium sized effect of f² = 0.15 in the later described model¹.

Design

A 2×2 between-subjects design with the factors agreement and endorsement was used: For nine topics, participants were exposed to either mostly attitudinally congruent (high agreement) or incongruent messages (low agreement), and congruent messages displayed either higher (high endorsement) or lower (low endorsement) numbers of “likes” than incongruent ones. Participants were exposed to the same condition over all topics. Estimates of public opinion for each topic were measured to calculate FCE as dependent variable. The experimental conditions and sizes of experimental groups are illustrated in Table 1.

Table 1. Attributes of the Displayed News Feeds in the Experimental Conditions in Experiment 1.

Materials

News feed topics were selected based on a prior study (Luzsa & Mayr, 2019) which had measured attitudes and FCE regarding twenty current topics. Nine topics that had elicited a strong FCE yet had also displayed some variance of participants’ own attitudes, that is, which had not evoked unanimous assent or dissent were selected. The topics are listed in Table 2.

The news feed messages were based on headlines and teaser texts found on social media accounts and websites of German news outlets (e.g., “Der Spiegel”, “Die Welt”). Sixteen texts per topic were used, eight expressing consent regarding the topic’s statement (e.g., highlighting advantages of legalization of marihuana), eight expressing dissent (e.g., emphasizing the dangers of marihuana). Some example messages are presented in Figure 1. To eliminate confounding variables, the assenting and dissenting messages’ characteristics (e.g., word count, use of exclamation marks or citations) were – If possible – balanced by rephrasing messages without altering content and meaning. In a pre-test, 15 participants rated how strongly they perceived each message as assenting to or dissenting from the topic and how likely they would click on it (6-point Likert scales). Four assenting and four dissenting messages per topic were selected which had received strong opinion ratings and were moderately likely to be clicked on. The number of words in assenting (M = 29.33) and dissenting messages (M = 28.47) was approximately equal.

The numbers of “likes” indicating low or high endorsement were similar to those used by Messing and Westwood (2014), which were based on actual numbers of “likes” of American Facebook messages: Low endorsement was expressed by 100–500 “likes”, high endorsement by 6000–19000. This approach was chosen because these numbers have already been successfully used in the cited study. An alternative approach would be to use numbers of “likes” observed in German social media. This strategy was employed in the second experiment.

Measures

Prior to news feed presentation, participants indicated their own stance towards each topic (e.g., agreement to the statement “Marihuana should be legalized.”) on a 6-point Likert scale (“Strongly agree” to “Strongly disagree”). To ensure that the measurements were stable and valid for FCE calculation, attitudes were measured again at the end of the experiment. Due to the short re-test interval, the initial statements were not reused. Instead, two additional items per topic were formulated (based on existing publicly available questionnaires, e.g., “Personal possession of marihuana should not be criminalized.”) and confirmed in a pre-test to be consistent with the initial item (all Cronbach’s α > .80). The means of the two items were then used for post-exposure attitude measurement. In addition to their own stance towards the topics, participants indicated how interested they were in each topic on a 6-point Likert scale (“very interested” to “not at all interested”).

During news feed presentation, the frequency of participants selecting congruent, incongruent, high and low endorsement messages was recorded.

Finally, to calculate FCE, the perceived public opinion was assessed after news feed presentation: For each topic, participants were shown the statements previously used for attitude measurement (e.g., “Marihuana should be legalized”) and were asked to estimate the percentage of the population in favor of the statement via numerical input (0 to 100%).

Procedure

First, participants were informed about voluntariness of participation and the possibility to cancel at any time. Then, they gave their consent regarding data privacy. After stating their own attitudes and interest regarding the topics, they were presented with the nine news feeds in accordance with their randomly assigned experimental condition. The order of the presentation of feeds as well as the order of messages in each feed were randomized. Each feed was displayed until the participant selected a message. Afterwards, participants estimated the percentage of the population with a positive stance regarding each topic. Then, they answered the post-exposure attitude items and gave basic demographic data. Finally, participants were debriefed and informed about the experimental manipulation.

Analysis

Data analysis was conducted with GNU R 3.5.2. All data, stimuli and code are available on github (https://github.com/RobertLuzsa/false_consensus_2020 ) and upon request to the corresponding author.

Stability of Attitudes. To ensure that the initially measured attitudes were stable and could be used for the further analysis, correlations with the post-exposure attitude items were calculated.

Overall False Consensus Effect. First, it was tested whether the topics used in this experiment successfully elicited an overall FCE (independent of experimental manipulation). As the participants’ attitude towards topics was measured with scales and not with traditional dichotomous questions, FCE was conceptualized as the correlation between participants’ own stance towards a topic and their estimated percentage of the public with positive stance towards the topic. Positive correlations were expected (i.e., the more positive participants’ attitude towards a topic, the larger the estimated percentage of the public with a positive attitude towards the topic).

Effects on False Consensus. For hypothesis testing, an approach that has already been successfully employed in a previous study (Luzsa & Mayr, 2019) was used: Traditionally, FCE is operationalized by between-groups comparison (Ross et al., 1977). However, for the current experiment, an individual-level measurement appeared suitable, as individual-level factors such as participants’ own interest in a topic needed to be considered. Several approaches for an individual-level measurement of FCE exist but are debated controversially (de la Haye, 2000; Galesic et al., 2013). Therefore, an alternative linear mixed effects modelling approach was employed, using the “lme4” and “lmertest” R-packages (Kuznetsova et al., 2017) for model estimation and significance testing².

The model predicted participants’ estimate of population percentage that has a positive stance towards a topic. First, this outcome was predicted by two random intercepts of the crossed random factors (Baayen et al., 2008) participant and topic. By this, baseline differences of estimates between participants (participants may display idiosyncratic tendencies to give high or low estimates, independent of topic) and topics (topics may generally lead to higher or lower estimates) were taken into account.

Then, FCE was modeled as the effect of participants’ own attitude towards a topic (6-point interval scaled from fully negative to fully positive) on the estimated population percentage with positive stance towards the topic. The regression coefficient of attitude indicates how strongly own attitudes bias estimates of public opinion: A coefficient of 2, for example, would indicate that participants with strong positive attitude (answer 6 on a scale from 1 to 6) estimate 12% more of the population having a positive attitude than participants with strong negative attitude (answer 1) and 4% more than participants with a weak positive attitude (answer 4). A larger coefficient (e.g., 5) would reflect a larger bias of own attitude (e.g., a difference of 30% in estimation between participants with strong positive and strong negative attitudes). Thus, the regression weight of attitude is a measure for the strength of FCE.

The effects of agreement and endorsement on FCE were then operationalized as interactive terms of the dummy-coded factors (with -1 indicating low agreement/endorsement and 1 high agreement/endorsement, respectively) and the participants’ own interval-scaled attitudes. For example, the participants were hypothesized to display stronger FCE in the high agreement than in the low agreement condition. Therefore, an interaction between the factor agreement and participants' attitude on FCE should be found, meaning that in the high agreement condition the estimate of public opinion should be more strongly biased in favor of participants’ own attitudes.

Finally, the participants’ age and gender as well as the news feed presentation order were included as control variables. All predictors entered were centered on population means in order to reduce variance inflation due to the included interactive terms.

Message Selection. The task to click on messages was primarily given to ensure reading of messages, and no hypotheses regarding message selection were formulated. Nonetheless, effects of the experimental manipulation on message selection were explored. For this, first, the number of attitudinally congruent message choices of each participant (e.g., for 3 of 9 topics) was determined. Then, the number expected assuming random selection (e.g., 9*3/4 = 6.75 if three of four presented messages were congruent) was subtracted. The resulting value (e.g., -3.75) was compared between experimental conditions via 2×2 ANOVA.

Results

Stability of Attitudes

Attitude values are reported in Table 2. For 8 of 9 topics, attitudes before and after experimental manipulation were strongly correlated (r from .66 to .89, p < .001). Only the topic “EU integration” displayed a moderately positive correlation (r = .42, p < .001). The attitudes were therefore considered stable and were used to calculate FCE. An additional analysis that excluded the moderately stable topic "EU integration" led to identical results as an analysis of all topics. Therefore, results of the latter are reported.

Table 2. Descriptive Statistics of Participants' Own Attitudes and False Consensus Indicators in Experiment 1 and 2.

Overall False Consensus Effect

An overall FCE was found for all topics: The more positive the participants’ own attitude towards a topic, the higher they estimated the percentage of the population with positive attitude towards it, with correlations from .14 to .41. Additionally, to compare overall FCE with studies that employ a group-based FCE measure, the estimated percentage of population with positive attitude was compared between participants with own negative (answer 1–3 on a 6-point scale) vs. own positive (answer 4–6) attitude (see Table 2). Positive FCE values were found for all topics, ranging from 14.82% for the topic “Dual citizenship” (52.35% vs. 37.53%) to 5.34% for “Ban on diesel vehicles” (42.54% vs 37.20%). FCE values were similar to those reported in literature (e.g., Bauman & Geher, 2002; Ross et al., 1977).

Effects on False Consensus

Table 3. Linear Mixed Effects Regression on Estimated Percentage of Population in Favor of a Statement.

The left side of Table 3 displays the results of the linear mixed effects modelling for Experiment 1, with model 0 as random-intercept-only reference model, model 1 including only control variables, and model 2 as full model with all predictors. Fit of the full model was significantly better than fit of model 1 (χ²(14) = 40, p < .001), therefore results of the full model are stated.

Firstly, the model confirmed a general FCE by finding estimated public opinion positively correlated with participants’ own attitude (β = 3.95, t[2909] = 10.44, p < .001). Figure 2a visualizes FCE by plotting this correlation as regression lines for all experimental conditions.

Figures 2a and 2b. Strength of the False Consensus Effect in Experiments 1 and 2
as a Function of the Factors Agreement and Endorsement, Illustrated as Regression Lines.


Note. Values on the vertical axis are values predicted by the regression model, not observed values.
Therefore, no error bars are depicted. Standard Errors for the illustrated regression coefficients are given in Table 2.

The positive correlation between own attitude and estimated public opinion, that is, the FCE, was stronger when participants were exposed to mostly congruent messages (high agreement) than when exposed to mostly incongruent ones (low agreement, β = 0.63, t[2865] = 2.94, p = .004). In Figure 2a, this is indicated by the solid lines, representing high agreement, being steeper than the dashed lines that represent low agreement.

The extent of this effect was moderated by endorsement (β = -0.47, t[2865] = -2.16, p = .031): If congruent messages displayed low endorsement, the effect of agreement was evident, and participants who saw mostly congruent messages (high agreement) displayed higher FCE than participants who saw mostly incongruent messages (low agreement). However, if congruent messages displayed high endorsement, the FCE was not affected by the number of congruent messages, that is, the factor agreement (see the difference in steepness between solid and dashed lines in case of high and low agreement, represented by black and grey lines, respectively, in Figure 2a).

Additionally, an interactive effect of endorsement and participants’ interest in a topic was found (β = -0.47, t[2908] = -2.93, p = .003). To understand this interactive effect, the differences in FCE between the two endorsement conditions when the participants’ interest was lowest vs. highest are visualized in Figures 3a and 3b, respectively: If participants had low interest in a topic, high endorsement, that is, higher numbers of “likes” for congruent than incongruent messages, led to a higher FCE than low endorsement. However, if they showed high interest in a topic, this effect reversed, and high endorsement led to a weaker FCE than low endorsement. In contrast to the interactive effect of endorsement and participants’ interest on FCE, there was no interactive effect of agreement and interest.

Figures 3a and 3b. Strength of the False Consensus Effect in Experiment 1
as a Function of the Factor Endorsement in Case of Lowest (3a) and Highest (3b) Interest Values.


Note. The factor agreement did not enter into an interactive effect with interest and is therefore not shown
(i.e., graphs reflect the effects of interest and endorsement). Values on the vertical axis are values predicted by the regression model, not observed values.

Message Selection

Regarding message selection, participants displayed a novelty or oddity effect (Berlyne & Ditkofksy, 1976) by preferentially clicking on the one message that voices a deviant opinion (F[1,327] = 370.91, p < .001).

Discussion

Overall, the experiment confirmed the assumption that agreement and endorsement encountered in social media news feeds influence participants’ perception of public opinion. Moreover, it sheds light upon the interplay of message contents, numbers of “likes”, and interest in a topic.

It was assumed that exposure to news feeds with mostly attitudinally congruent messages (high agreement) would lead participants to estimate a higher percentage of the population to share their views than exposure to mostly incongruent messages (low agreement). Indeed, participants displayed a stronger FCE in the high agreement condition. This confirms that participants’ estimate of public opinion is influenced by the level of agreement they encounter in online messages.

However, the factor agreement played a major role mostly when the congruent messages had less “likes” than the incongruent ones (condition low endorsement). In contrast, when congruent messages displayed high endorsement, the factor agreement, that is, the number of congruent messages, had no effect on FCE. This result might be explained as an effect of resistance and reactance due to participants suspecting a persuasive intention (van Noort et al., 2012): When attitudinally congruent messages consistently display high numbers of “likes”, participants might get skeptical of the validity of the numbers and suspect that the news feed is manipulated and biased. This might lead them to be more critical towards the overall news feed and the message contents. Thus, the effect of the number of congruent messages is reduced or disappears. It can be assumed that the simultaneous presentation of all four news items per topic has contributed to this effect, as there was a repeating pattern of three messages with similar and one with dissimilar “likes” on each page. If this was the case, the observed interactive pattern should be eliminated or at least weakened if messages are not presented parallelly but sequentially, one per screen, making patterns in “likes” less overt. This was tested and addressed in experiment 2.

While there was no main effect of endorsement on FCE per se, the experiment found a second interactive effect of endorsement and the participants’ interest in a topic: Based on Elaboration Likelihood Theory (Petty & Cacioppo, 1986), “likes” were conceived as peripheral cues which should have a stronger effect when involvement/interest was low. Thus, in case of low interest in a topic, participants were expected to take the numbers of “likes” as indicators of public opinion, resulting in a positive effect of endorsement on FCE. This effect was confirmed.

In case of high interest in a topic, however, participants were expected to pay less attention to “likes”, with the effect of endorsement turning out weaker or disappearing. In fact, the experiment found that the effect did not merely disappear but even reversed: As hypothesized, when interest in a topic was low, high endorsement led to a stronger FCE than low endorsement. However, when interest was high, high endorsement led to a significantly weaker FCE. Thus, the moderating effect of interest was even stronger than expected. A possible explanation for this could be seen in an interplay of participants' elaboration style and the numbers of "likes" used in the experiment: In case of low interest, participants put little effort into the processing of messages and accept the numbers of “likes” as valid indicators of public opinion. However, when they have high interest, they elaborate messages and numbers of “likes” more thoroughly and more critically and perhaps question the objectivity and representativeness of the seen numbers, therefore adjusting their estimate of public opinion in the opposite direction. This effect might have been amplified by the high numbers of “likes” adapted from Messing and Westwood (2014) and based on American Facebook profiles. These numbers may have appeared unrealistic to German participants who are accustomed to lower numbers in German social media. This might have contributed to the assumed perception of a persuasive intent. Therefore, the follow-up experiment will use the alternative strategy described above, with numbers of “likes” based on those observed on German news outlets' social media pages.

Two more methodological aspects of the paradigm need to be addressed:

The paradigm implemented an Echo Chamber in which there is a majority view on the topic and one deviant message. However, it is well known that exposure to a mostly homogenous group in which one member states a deviant opinion has less impact on judgements than exposure to a completely unanimous group (Asch, 1961). A similar effect might have occurred in the present experiment. Therefore, it appears worthwhile to examine the effects on FCE when there is unanimity in the Echo Chamber, that is, no message expresses a deviant point of view.

Moreover, the experiment used a forced-choice paradigm that required participants to explicitly click on one message. While this was based on existing paradigms, it can be criticized for having low ecological validity (Knobloch-Westerwick, 2014): When browsing real social media news feeds, users are not forced to follow only one link, but might open several links one after another. Thus, it cannot be ruled out that the current experiments’ specific task might have led participants to process and evaluate the messages and popularity cues differently than in a more naturalistic setting. A replication could avoid this by instead letting participants indicate for each message how likely they are to click upon it.

All mentioned issues were addressed in Experiment 2.

Experiment 2

Experiment 2 aimed at replicating the findings of Experiment 1 regarding effects of agreement and endorsement on FCE³ while testing the robustness of the findings under changed modes of presentation. The paradigm was altered as follows: Participants were again exposed to congruent and incongruent messages that again displayed either high or low endorsement. However, participants were exposed to only one message per screen and were required to indicate how likely they would click on and read this message if it appeared in a social media news feed (see Figure 4). Four messages per topic were shown in sequence. In contrast to Experiment 1, no attitudinally deviant message was included: In the agreement condition, all messages were in accordance with participants’ point of view, in the disagreement condition all messages disagreed with participants’ opinion. Endorsement was again manipulated via numbers of “likes”: In case of low endorsement, congruent messages had low or incongruent high numbers of “likes”, and vice versa for high endorsement. As all messages were either congruent or incongruent, participants always saw either high or low numbers of “likes”.

As no deviant message with differing numbers of “likes” was shown, participants needed an anchor (Furnham & Boo, 2011) to allow them to judge whether numbers were high or low. For this, four irrelevant messages (topic “housing costs”) were presented in the beginning, two of them displaying high, two low numbers of “likes”.

Figure 4. Example of a Message Presented in Experiment 2.

Method

Sample

A sample of 236 participants was recruited via student social media groups and on University of Passau campus in December 2018 and January 2019. After data cleaning similar to Experiment 1 and exclusion of participants who already took part in Experiment 1, 207 valid cases remained (146 female; age between 18 and 35 years with M = 21.69 and SD = 2.75). Again, the criterion of 195 cases for a power of 1–β = .95, f² = 0.15 and α = .05 was met.

Materials. Topics and news feed texts were identical to Experiment 1. The numbers of “likes” to express high or low endorsement were now based on actual numbers observed for Facebook messages of several major German news outlets (e.g., “Der Spiegel”, “Tagesschau”) over a two-day period. Observed values mostly ranged between 10 and 1000. Therefore, low endorsement was expressed by 10 to 50 “likes” and high endorsement by 300 to 999 “likes”.

The four irrelevant messages were taken from news articles on the unrelated topic “housing costs”. Two displayed low endorsement numbers, two high endorsement numbers.

Measures

Attitudes and interest regarding the topics and estimates of public opinion were measured as in Experiment 1. Message selection was operationalized as the likelihood to click on messages, measured via a 6-point Likert scale (“Would certainly click” to “Would certainly not click”).

Design

Again a 2×2 between-subjects design with factors agreement and endorsement was used: Participants either saw only congruent (agreement) or only incongruent messages (disagreement), and endorsement was either high (congruent messages displaying high or incongruent low numbers of “likes”) or low (incongruent messages displaying low or congruent high numbers of “likes”). FCE as dependent variable was calculated as in experiment 1. The sizes of the experimental groups are given in Table 4.

Table 4. Attributes of the Displayed News Feeds in the Experimental Conditions in Experiment 2.

Procedure

As in Experiment 1, participants were first informed about voluntariness of participation, gave consent and then stated their own attitudes and interest in topics. Afterwards, the four neutral messages were displayed, two of them with high, two with low numbers of “likes”. Then, participants were exposed to the biased news feed, with four messages for each of the nine topics displayed sequentially, one per screen. Presentation orders of topics and of messages for each topic were randomized, but all messages for one topic were displayed blocked. After exposure, participants gave their estimates of public opinion, then answered demographic questions and were debriefed.

Analysis

The existence of a general FCE was examined by correlating own attitude and estimated public opinion as in Experiment 1. Similarly, the effects of experimental conditions on FCE were analyzed with the same model as used in Experiment 1. In addition, an exploratory evaluation of participants’ message selection was conducted. As message selection was now indicated as the likelihood to click on an article on a 6-point Likert-scale, a linear mixed effect regression model identical to that used for FCE analysis was employed to evaluate effects on message selection.

Results

Overall False Consensus

Overall FCE patterns similar to Experiment 1 were observed (see Table 2).

Effects on False Consensus

The effects of agreement and endorsement on FCE are illustrated on the right side of Table 3 and in Figure 2b. Results are similar to Experiment 1 but do not reveal the same complex interactive structure: The overall FCE was again reflected by the effect of own attitude on estimated public opinion (β = 4.24, t[1819] = 12.39, p < .001). Also comparable to Experiment 1, FCE turned out stronger in the agreement condition, that is, when only congruent messages were shown than when participants only saw incongruent messages (i.e., disagreement condition, β = 0.88, t[1787] = 3.08, p = .002). High endorsement, however, led to a weaker FCE compared to low endorsement (β = -0.82, t[1783] = -2.83, p = .005), independent of interest in topic and other factors. No further effects were significant.

Message Selection

Participants indicated a higher likelihood to click on messages when the opinion voiced in the message was congruent to their own (β = 0.19, t[203] = 4.30, p < .001), when the message had a high number of likes (β = 0.10, t[203] = 2.26, p = .025), and when their interest in the messages topic was high (β = 0.04, t[1614] = 2.48, p = .013).

Discussion

Experiment 2 successfully replicated the effect of agreement on FCE: Similar to Experiment 1, if participants saw congruent messages, they displayed higher FCE than when exposed to incongruent messages. The size of this effect was independent of endorsement, that is, the numbers of “likes” displayed, while in Experiment 1 it was stronger when “likes” expressed low endorsement for the participants’ attitude.

The effect of endorsement on FCE, however, could only be partially replicated: In Experiment 1, endorsement interacted with participants’ interest in a topic, with high endorsement increasing FCE when interest was low, but decreasing FCE when interest was high. In Experiment 2, participants’ FCE was weaker in the high endorsement condition than in the low endorsement condition, independent of interest in topic.

Thus, in Experiment 2, the assumption of higher endorsement leading to higher FCE has to be discarded, and instead, an inversed effect occurred. An explanation for the reversal of the effect could be that in the second experiment, participants might have always been skeptical of “likes” and suspected a persuasive, manipulative intention. This more critical view of endorsement might have been caused by the unanimity in numbers of “likes” while in Experiment 1 there was always one deviant message with a strongly different number of “likes”. For participants with low interest, this observable variance in “likes” might have been enough to accept them as a valid indicator of popular opinion, while highly involved participants perceived them as biased. In Experiment 2, “likes” were continuously high or low in comparison to the initially displayed anchor values. Therefore, even participants with low interest might have realized that the numbers show little variance and are biased, leading them to also react with skepticism and lower their estimates of public support, thus weakening FCE. This explanation appears in accordance with prior research illustrating that both context of presentation and participants' previous experiences affect interpretation of popularity (Haim et al., 2018).

General Discussion

Evidence for Effects of Online Echo Chambers

Both experiments tested whether online Echo Chambers can influence False Consensus Effect (FCE), that is, how strongly individuals overestimate public support for their own opinions. Participants were exposed to biased news feeds in which most/all messages were either congruent (high agreement) or incongruent (low agreement) to their attitudes and in which higher numbers of “likes” were displayed for congruent (high endorsement) or incongruent messages (low endorsement).

Both experiments confirm that exposure to biased online news feeds leads to a more favorably distorted perception of public opinions: When participants encounter high agreement, that is, (mostly) attitudinally congruent messages in news feeds, they will display a stronger FCE. This occurs independent of participants’ interest for a topic, and regardless of mode of presentation and whether there is one message with a deviant opinion or complete unanimity. Such an effect was expected, given previous findings regarding links between availability of attitudinally (in)congruent information and FCE (Bauman & Geher, 2002; Wojcieszak, 2008). However, the current experiments also verify this effect for information in shape of short social media news messages. They demonstrate that participants’ perception of public opinion can be distorted by exposure to a biased news feed, even if no further interaction (e.g., clicking on links and reading longer texts) occurs. Moreover, the effect of agreement suggests that participants tend to accept information expressed in message contents as basis for their estimate of public opinion, and that they are less suspicious of biases in message contents than of biases in numbers of “likes”.

Regarding the size of effects, the overall dichotomized False Consensus values (see Table 2) are close to the 10%–20% traditionally found (Ross et al., 1977). Due to the newness of the paradigm, the effects of agreement and endorsement on FCE are more difficult to judge: For example, the interactive effect of 0.88 of agreement in experiment 2 means that the effect of own attitude on estimated population opinion increases from 3.36 (4.24–0.88) in the disagreement to 5.12 (4.24 + 0.88) in the agreement condition. To compare to previous research, a dichotomized FCE value can be calculated, by subtracting the sum of effects for pro attitude (answers 4–6 on the Likert-scale) from the sum of contra attitude (answers 1–3). The obtained FCE is 5.28% higher in the agreement (15.36%) than the disagreement condition (10.08%). When compared to traditional overall FCE values of 10%–20%, this appears as a medium-sized and practically relevant effect of agreement.

Overall, these findings highlight that warnings of negative effects of online Echo Chambers are at least somewhat substantiated: Users of online social networks often tend to surround themselves preferably with like-minded others (Del Vicario et al., 2016; Quattrociocchi et al., 2016; Schmidt et al., 2017; Zollo et al., 2017) and thus receive mostly messages that agree with their own attitudes. The current study suggests that they will most likely conclude from this selective exposure to attitudinally congruent messages to high public support for their own points of view. This perception of public support and favorably distorted social norm might, in turn, make them more willing to act upon their attitudes, giving rise to possibly harmful consequences. For example, persons who hold racist or otherwise hostile attitudes might conclude from the approval by their biased social network that large parts of the population agree with them, and they might therefore be more likely to put their beliefs into (violent) action.

However, the present study also allows for more optimistic conclusions. While online users appear not to suspect biases in agreement expressed in message texts, they appear critical of endorsement indicated by the numbers of “likes”: They do not universally interpret high numbers of “likes” for messages congruent to their own attitudes as valid evidence for the public agreeing with them, especially if their interest in a topic is high. Instead, they lower their estimate of public agreement. Thus, users appear to be wary of biases in numbers of “likes” and should be somewhat resistant towards attempts to influence their perception of public opinion via manipulated numbers. Moreover, in experiment 1, this critical view of “likes” was stronger when participants reported more interest in a topic, and a previous correlative study (Luzsa & Mayr, 2019) found that higher ambiguity tolerance, that is, more openness towards ambiguous and contradicting information, leads to weaker FCE. Both factors – interest/involvement and ambiguity tolerance – thus appear to increase resilience towards negative effects of Echo Chambers. Further research might therefore examine whether and how these resilience factors could be strengthened, for example by “nudging” users towards a more critical evaluation of information through website design (Kim & Dennis, 2019).

Methodological Considerations

Some limitations of both reported experiments must be addressed:

It is unclear how stable the effects of Echo Chamber induction on FCE are, as FCE was measured directly after news feed exposure. A long-term effect appears plausible, given that in real life individuals are exposed to news feeds whenever they visit a social networking platform. This could be tested with a repeated measurement design. For example, participants could repeatedly take part in discussions in a dedicated online community, where they are confronted with news feeds with high or low agreement and endorsement, but can also interact with the website in other ways. By measuring FCE repeatedly, the stability of effects and their robustness in a more realistic online environment could be tested. Moreover, if additionally other outcomes (e.g., attitudinal strength and extremity as more direct indicators of polarization) are measured and users' message selection behavior is observed, the interplay between False Consensus, polarization and Echo Chamber formation could be more closely examined.

Moreover, in this environment, other operationalizations of participants' involvement with a topic instead of self-reported interest could be used, for example by giving tasks with offered rewards. Similarly, such studies would also allow for the testing of interventions that try to reduce individuals' biases in online news perception and processing, for example via online games (Roozenbeek & van der Linden, 2019).

The second main limitation of the current study is that, due to the experimental approach, only the message attributes agreement and endorsement were manipulated. Real social media messages, however, also show additional attributes that might influence users, such as sender names and images. Their effects should be addressed in future experiments. Attention could also be paid to other types of popularity cues (e.g., how often messages were shared or commented on) and whether these display similar effects as “likes”.

Another limitation can be seen in the use of news messages as stimuli. These were employed because FCE was measured towards news topics and because news play an import role in social media (Bastos, 2015; Swart et al., 2019). However, interaction in social networks is also made up of personal messages and personal narrations. Thus, it should be tested whether the current findings can be replicated with more subjectively phrased personal messages that express agreement or disagreement regarding a topic. This could also reduce possible confoundation that arises from using actual news texts, which, despite being homogenized and pre-tested, can still display some unavoidable variance (e.g., differences in linguistic and argumentation style, some texts citing public figures).

Furthermore, the experimental conditions can be discussed: Both experiments compared conditions in which the majority of messages either voiced opinions congruent or incongruent to subjects’ opinions. This decision was made to maximize the power of the experimental manipulation. However, future studies might include a balanced condition with equal numbers of congruent and incongruent messages to establish a baseline of FCE values. Parallel to this, by using an uneven numbered attitude scale that also allows for a neutral answer, further research might also test effects of biased news feeds on participants that indicate no clear pro or contra stances.

Lastly, the current study used a non-representative sample. This, however, does not need to be seen as a weakness, as the expected effects were found even in this well-educated and young sample which can be assumed to be experienced and critical regarding online contents. Nonetheless, examination in other, more representative and international samples seems reasonable as well as additional examination of personal-level variables that might influence how strongly biased news feeds affect individuals, for example overall internet skills (van Deursen et al., 2016) or information literacy (Çoklar et al., 2017). In the same vein, a bias check might be conducted by examining which participants report to have perceived the news feeds as biased, and how this perception influences the effects of the news feeds on the participants.

Despite these limitations, however, the present study offers valuable insights into the effects of online Echo Chambers on individuals’ actual perceptions and behavior. Furthermore, the study demonstrates that experimental approaches that modify existing paradigms can be successfully used to investigate Echo Chambers and uncover their causal effects on outcomes like FCE.

Footnotes

¹ The power refers to an F-test for difference in explained variance R² between the full model with 21 predictors and the control variables only model with seven predictors (models 1 and 2 in Table 3). As the estimated linear mixed effects model uses random intercepts but only fixed slopes, power is approximated via power calculation for traditional OLS regression. This serves as conservative lower-boundary approximation of power, as in case of fixed slopes, mixed effect models generally have more power than OLS (Snijders, 2005).

² In “lmertest”, for significance testing, degrees of freedom are approximated via Satterthwaite's approximation.

³ Experiment 2 examined participants’ willingness to share attitudinally congruent messages with their social network contacts as an additional dependent variable. However, no effects were found. The measurement took place at the end of the experiment and did not interfere with the other measurements. As the findings are not informative with respect to the central purpose of Experiment 2, they are not discussed in further detail.