Are online users influenced by what other users say? Meta-analyzing the cognitive, emotional, and behavioral impact of online comment valence

Vol.18,No.5(2024)

Abstract

Online comments have become an essential component of online media consumption. A meta-analysis was conducted to understand how online comment valence affects message perception, issue-relevant beliefs and attitudes, issue-relevant behaviors and behavioral intentions, communication behaviors and intentions, and emotions. Comment valence is defined as the distinction between positive comments, which align with, support, or favor the opinions expressed in the original message, and negative comments, which oppose, criticize, or disagree with the opinions expressed in the original message. After a comprehensive search and systematic screening and coding of existing studies, we identified 44 studies that are eligible to be included in the meta-analysis. We found that positive (vs. negative) comments led to significantly more positive evaluations of original messages (r = .22), stronger beliefs and attitudes that align with the positive comments (r = .29), higher likelihood to engage in behaviors that align with the positive comments (r = .09), higher likelihood to express opinions that align with the positive comments (r = .26), and more positive emotions (r = .16). Moreover, the number of comments, whether comment valence was mixed or not, and whether the original message was news or non-news moderated the effects of online comment valence on several outcomes. The findings suggest integrating these outcomes and moderators to develop a media effect theory and guide media practices in light of comment valence effects.


Keywords:
online comments; news comments; social media; meta-analysis; social influence
Author biographies

Junhan Chen

Journalism and Media Studies Centre, University of Hong Kong, Hong Kong

Junhan Chen is an Assistant Professor at the Journalism and Media Studies Centre, the University of Hong Kong. Her research focuses on how social norms influence behavior and how these influences can be leveraged in communication strategies to promote health behavior change. Her work also delves into the innovative use of virtual and augmented reality technologies within the fields of health and science communication.

Shilin Xia

Department of Communication, University of Maryland, College Park, USA

Shilin Xia is a Ph.D. candidate in Communication Science at the University of Maryland-College Park. Her research focuses on the science of persuasion in the health and risk contexts. Her research primarily examines the effects and processing of persuasive messages, and the roles of emerging technologies (e.g., generative AI, real-time comments) in persuasive message design.

References

Ahmed, S., Jaidka, K., & Cho, J. (2017). Tweeting India’s Nirbhaya protest: A study of emotional dynamics in an online social movement. Social Movement Studies, 16(4), 447–465. https://doi.org/10.1080/14742837.2016.1192457

Ahn, H. (2011). The effect of online news story comments on other readers’ attitudes: Focusing on the case of incongruence between news tone and comments [Master’s thesis, The University of Alabama]. University of Alabama Libraries. https://ir.ua.edu/handle/123456789/1118

Andrighetto, G., & Vriens, E. (2022). A research agenda for the study of social norm change. Philosophical Transactions. Series A, Mathematical, Physical, and Engineering Sciences, 380(2227), Article 20200411. https://doi.org/10.1098/rsta.2020.0411

Ballantine, P. W., Lin, Y., & Veer, E. (2015). The influence of user comments on perceptions of Facebook relationship status updates. Computers in Human Behavior, 49, 50–55. https://doi.org/10.1016/j.chb.2015.02.055

Cacioppo, J. T., Petty, R. E., Kao, C. E, & Rodriguez, R. (1986). Central and peripheral routes to persuasion: An individual difference perspective. Journal of Personality and Social Psychology, 51(5), 1032–1043. https://doi.org/10.1037/0022-3514.51.5.1032

Cameron, J., & Geidner, N. (2014). Something old, something new, something borrowed from something blue: Experiments on dual viewing TV and Twitter. Journal of Broadcasting & Electronic Media, 58(3), 400–419. https://doi.org/10.1080/08838151.2014.935852

Chen, G., & Lu, S. (2017). Online Political Discourse: Exploring Differences in Effects of Civil and Uncivil Disagreement in News Website Comments. Journal of Broadcasting & Electronic Media, 61(1), 108-125. https://doi.org/10.1080/08838151.2016.1273922

Chen, J., Yan, Y., & Leach, J. (2022). Are emotion-expressing messages more shared on social media? A meta-analytic review. Review of Communication Research, 10, 59–79. https://www.rcommunicationr.org/index.php/rcr/article/view/9/2

Cho, H., Cannon, J., Lopez, R., & Li, W. (2024). Social media literacy: A conceptual framework. New Media & Society, 26(2), 941–960. https://doi.org/10.1177/14614448211068530

Chung, J. E. (2019). Peer influence of online comments in newspapers: Applying social norms and the social identification model of deindividuation effects (SIDE). Social Science Computer Review, 37(4), 551–567. https://doi.org/10.1177/0894439318779000

Cialdini, R. B., Reno, R. R., & Kallgren, C. A. (1990). A focus theory of normative conduct: Recycling the concept of norms to reduce littering in public places. Journal of Personality and Social Psychology, 58(6), 1015–1026. https://doi.org/10.1037/0022-3514.58.6.1015

Cohen, J. (1969). Statistical power analysis for the behavioral sciences. Academic Press.

Colliander, J. (2019). “This is fake news”: Investigating the role of conformity to other users’ views when commenting on and spreading disinformation in social media. Computers in Human Behavior, 97, 202–215. https://doi.org/10.1016/j.chb.2019.03.032

Couture Bue, A. C., & Harrison, K. (2020). Visual and cognitive processing of thin-ideal Instagram images containing idealized or disclaimer comments. Body Image, 33, 152–163. https://doi.org/10.1016/j.bodyim.2020.02.014

Dempsey, B., Looney, K., McNamara, R., Michalek, S., & Hennessy, E. (2022). An experimental investigation of adolescent and young adult responses to stigmatizing and supportive social media posts in response to a depressed peer. Computers in Human Behavior, 131, Article 107229. https://doi.org/10.1016/j.chb.2022.107229

Duncan, M., & Coppini, D. (2019). Party v. the people: Testing corrective action and supportive engagement in a partisan political context. Journal of Information Technology & Politics, 16(3), 265–289. https://doi.org/10.1080/19331681.2019.1644266

Duncan, M., Pelled, A., Wise, D., Ghosh, S., Shan, Y., Zheng, M., & McLeod, D. (2020). Staying silent and speaking out in online comment sections: The influence of spiral of silence and corrective action in reaction to news. Computers in Human Behavior, 102, 192–205. https://doi.org/10.1016/j.chb.2019.08.026

Duong, H. T., Van Nguyen, L. T., Vu, H. T., & Trinh, A. T. (2020). Association between online social influence and corporal punishment: An experimental study. Child & Adolescent Social Work Journal, 37(2), 163–177. https://doi.org/10.1007/s10560-019-00632-9

Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997). Bias in meta-analysis detected by a simple, graphical test. BMJ, 315(7109), 629–634. https://doi.org/10.1136/bmj.315.7109.629

Fletcher, R., & Park, S. (2017). The impact of trust in the news media on online news consumption and participation. Digital Journalism, 5(10), 1281–1299. https://doi.org/10.1080/21670811.2017.1279979

Flynn, M. A. (2012). The effects of body ideal profile pictures and friends’ comments on social network site users’ body image: A side model approach [Doctoral dissertation, Bowling Green State University]. ProQuest Dissertations & Theses Global. https://www.proquest.com/dissertations-theses/effects-body-ideal-profile-pictures-friends/docview/1264911894/se-2?accountid=14696

Fu, R., Gartlehner, G., Grant, M., Shamliyan, T., Sedrakyan, A., Wilt, T. J., Griffith, L., Oremus, M., Raina, P., Ismaila, A., Santaguida, P., Lau, J., & Trikalinos, T. A. (2011). Conducting quantitative synthesis when comparing medical interventions: AHRQ and the effective health care program. Journal of Clinical Epidemiology, 64(11), 1187–1197. https://doi.org/10.1016/j.jclinepi.2010.08.010

Gellerstedt, M., & Arvemo, T. (2019). The impact of word of mouth when booking a hotel: Could a good friend’s opinion outweigh the online majority? Information Technology & Tourism, 21(3), 289–311. https://doi.org/10.1007/s40558-019-00143-4

Hatfield, E., Cacioppo, J. T., & Rapson, R. L. (1993). Emotional contagion. Current Directions in Psychological Science, 2(3), 96–99. https://doi.org/10.1111/1467-8721.ep10770953

Hayes, J. L., Shan, Y., & King, K. W. (2018). The interconnected role of strength of brand and interpersonal relationships and user comment valence on brand video sharing behaviour. International Journal of Advertising, 37(1), 142–164. https://doi.org/10.1080/02650487.2017.1360576

Higgins, J. P. T., & Thompson, S. G. (2002). Quantifying heterogeneity in a meta-analysis. Statistics in Medicine, 21(11), 1539–1558. https://doi.org/10.1002/sim.1186

Hilverda, F., Kuttschreuter, M., & Giebels, E. (2018). The effect of online social proof regarding organic food: Comments and likes on Facebook. Frontiers in Communication, 3, Article 30. https://doi.org/10.3389/fcomm.2018.00030

Houston, J. B., Hansen, G. J., & Nisbett, G. S. (2011). Influence of user comments on perceptions of media bias and third-person effect in online news. Electronic News, 5(2), 79–92. https://doi.org/10.1177/1931243111407618

Hsueh, M., Yogeeswaran, K., & Malinen, S. (2015). “Leave your comment below”: Can biased online comments influence our own prejudicial attitudes and behaviors? Human Communication Research, 41(4), 557–576. https://doi.org/10.1111/hcre.12059

Hunter, J., & Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd ed.). Sage.

Hwang, H., Kim, Y., & Huh, C. U. (2014). Seeing is Believing: Effects of Uncivil Online Debate on Political Polarization and Expectations of Deliberation. Journal of Broadcasting & Electronic Media, 58(4), 621-633. https://doi.org/10.1080/08838151.2014.966365

Kim, H., Han, J. Y., & Seo, Y. (2020). Effects of Facebook comments on attitude toward vaccines: The roles of perceived distributions of public opinion and perceived vaccine efficacy. Journal of Health Communication, 25(2), 159–169. https://doi.org/10.1080/10810730.2020.1723039

Kim, H. M. (2021). What do others’ reactions to body posting on Instagram tell us? The effects of social media comments on viewers’ body image perception. New Media & Society, 23(12), 3448–3465. https://doi.org/10.1177/1461444820956368

Kohout, S., Kruikemeier, S., & Bakker, B. N. (2023). May I have your attention, please? An eye tracking study on emotional social media comments. Computers in Human Behavior, 139, Article 107495. https://doi.org/10.1016/j.chb.2022.107495

Koo, A. Z.-X., Su, M.-H., Lee, S., Ahn, S.-Y., & Rojas, H. (2021). What motivates people to correct misinformation? Examining the effects of third-person perceptions and perceived norms. Journal of Broadcasting & Electronic Media, 65(1), 111–134. https://doi.org/10.1080/08838151.2021.1903896

Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014). Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences, 111(24), 8788–8790. https://doi.org/10.1073/pnas.1320040111

Krämer, N. C., Neubaum, G., Winter, S., Schaewitz, L., Eimler, S., & Oliver, M. B. (2021). I feel what they say: The effect of social media comments on viewers’ affective reactions toward elevating online videos. Media Psychology, 24(3), 332–358. https://doi.org/10.1080/15213269.2019.1692669

Krippendorff, K. (2004). Reliability in content analysis: Some common misconceptions and recommendations. Human Communication Research, 30(3), 411–433. https://doi.org/10.1111/j.1468-2958.2004.tb00738.x

Ksiazek, T. B., & Springer, N. (2019). User comments in digital journalism: Current research and future directions. In E. Scott & B. Franklin (Eds.), The Routledge handbook of developments in digital journalism studies (pp. 475–486). Routledge.

Kümpel, A. S., & Unkel, J. (2020). Negativity wins at last: How presentation order and valence of user comments affect perceptions of journalistic quality. Journal of Media Psychology: Theories, Methods, and Applications, 32(2), 89–99. https://doi.org/10.1027/1864-1105/a000261

Kuttschreuter, M., & Hilverda, F. (2019). Risk and benefit perceptions of human enhancement technologies: The effects of Facebook comments on the acceptance of nanodesigned food. Human Behavior and Emerging Technologies, 1(4), 341–360. https://doi.org/10.1002/hbe2.177

Kwon, K. H., & Gruzd, A. (2017). Is offensive commenting contagious online? Examining public vs interpersonal swearing in response to Donald Trump’s YouTube campaign videos. Internet Research, 27(4), 991–10101. https://doi.org/10.1108/IntR-02-2017-0072

Landreville, K. D., & Niles, C. (2019). “And that’s a fact!”: The roles of political ideology, PSRs, and perceived source credibility in estimating factual content in partisan news. Journal of Broadcasting & Electronic Media, 63(2), 177–194. https://doi.org/10.1080/08838151.2019.1622339

Lands, M., Carpenter, E., Valley, T., Jacques, L., & Higgins, J. (2023). “Am I the only one who feels like this?”: Needs expressed online by abortion seekers. Social Work, 68(2), 103–111. https://doi.org/10.1093/sw/swad011

Lee, E.-J., & Tandoc, E. C. (2017). When news meets the audience: How audience feedback online affects news production and consumption. Human Communication Research, 43(4), 436–449. https://doi.org/10.1111/hcre.12123

Lee, J., & Lim, Y.-s. (2014). Who says what about whom: Young voters’ impression formation of political candidates on social networking sites. Mass Communication & Society, 17(4), 553–572. https://doi.org/10.1080/15205436.2013.816743

Lee, S., Atkinson, L., & Sung, Y. H. (2022). Online bandwagon effects: Quantitative versus qualitative cues in online comments sections. New Media & Society, 24(3), 580-599. https://doi.org/10.1177/1461444820965187

Lee, Y.-I., Phua, J., & Wu, T.-Y. (2020). Marketing a health brand on Facebook: Effects of reaction icons and user comments on brand attitude, trust, purchase intention, and eWOM intention. Health Marketing Quarterly, 37(2), 138–154. https://doi.org/10.1080/07359683.2020.1754049

Lee, Y.-J., Yoon, H. J., & O’Donnell, N. H. (2018). The effects of information cues on perceived legitimacy of companies that promote corporate social responsibility initiatives on social networking sites. Journal of Business Research, 83, 202–214. https://doi.org/10.1016/j.jbusres.2017.09.039

Lenhard, W., & Lenhard, A. (2016). Computation of effect sizes. Psychometrica. https://www.psychometrica.de/effect_size.html

Liao, M.-Q., & Mak, A. K. Y. (2019). “Comments are disabled for this video”: A technological affordances approach to understanding source credibility assessment of CSR information on YouTube. Public Relations Review, 45(5), Article 101840. https://doi.org/10.1016/j.pubrev.2019.101840

Lim, Y.-s., & Van Der Heide, B. (2015). Evaluating the wisdom of strangers: The perceived credibility of online consumer reviews on yelp. Journal of Computer-Mediated Communication, 20(1), 67–82. https://doi.org/10.1111/jcc4.12093

Liu, J. (2017). Effects of media exposure on descriptive social norm perception formation: Experimental and observational studies of why and how repeated exposure matters [Doctoral dissertation, University of Pennsylvania]. ProQuest Dissertations & Theses Global; Public Health Database. https://www.proquest.com/dissertations-theses/effects-media-exposure-on-descriptive-socialnorm/docview/1952165355/se-2?accountid=14696

Lu, F., & Sun, Y. (2022). COVID-19 vaccine hesitancy: The effects of combining direct and indirect online opinion cues on psychological reactance to health campaigns. Computers in Human Behavior, 127, Article 107057. https://doi.org/10.1016/j.chb.2021.107057

McIntyre, K. (2019). Solutions journalism: The effects of including solution information in news stories about social problems. Journalism Practice, 13(1), 16–34. https://doi.org/10.1080/17512786.2019.1640632

Möller, A. M., & Boukes, M. (2023). Online social environments and their impact on video viewers: The effects of user comments on entertainment experiences and knowledge gain during political satire consumption. New Media & Society, 25(5), 999–1022. https://doi.org/10.1177/14614448211015984

Noel, J. K. (2021). Using social media comments to reduce alcohol purchase intentions: An online experiment. Drug and Alcohol Review, 40(6), 1047–1055. https://doi.org/10.1111/dar.13262

Noelle-Neumann, E. (1993). The spiral of silence: Public opinion, our social skin (2nd ed.). The University of Chicago Press.

Oh, S.-H., Lee, S. Y., & Han, C. (2021). The effects of social media use on preventive behaviors during infectious disease outbreaks: The mediating role of self-relevant emotions and public risk perception. Health Communication, 36(8), 972–981. https://doi.org/10.1080/10410236.2020.1724639

Park, N., & Lee, K. M. (2007). Effects of online news forum on corporate reputation. Public Relations Review, 33(3), 346–348. https://doi.org/10.1016/j.pubrev.2007.05.018

Peter, C., Rossmann, C., & Keyling, T. (2014). Exemplification 2.0: Roles of direct and indirect social information in conveying health messages through social network sites. Journal of Media Psychology, 26(1), 19–28. https://doi.org/10.1027/1864-1105/a000103

Pjesivac, I., Geidner, N., & Cameron, J. (2018). Social credibility online: The role of online comments in assessing news article credibility. Newspaper Research Journal, 39(1), 18–31. https://doi.org/10.1177/0739532918761065

Rivis, A., & Sheeran, P. (2003). Descriptive norms as an additional predictor in the theory of planned behaviour: A meta-analysis. Current Psychology: A Journal for Diverse Perspectives on Diverse Psychological Issues, 22(3), 218–233. https://doi.org/10.1007/s12144-003-1018-2

Rösner, L., Winter, S., & Krämer, N. C. (2016). Dangerous minds? Effects of uncivil online comments on aggressive cognitions, emotions, and behavior. Computers in Human Behavior, 58, 461-470. https://doi.org/10.1016/j.chb.2016.01.022

Sadri, S. (2012). Perceived credibility of sports articles and attitudes toward sports sources and media: The role of sport fan identification [Doctoral dissertation, University of Florida]. ProQuest Dissertations & Theses Global. https://www.proquest.com/dissertations-theses/perceived-credibility-sports-articles-attitudes/docview/1654461837/se-2?accountid=14696

Scheufle, D. A., & Moy, P. (2000). Twenty-five years of the spiral of silence: A conceptual review and empirical outlook. International Journal of Public Opinion Research, 12(1), 3–28. https://doi.org/10.1093/ijpor/12.1.3

Sedgwick, P. (2015). What is publication bias in a meta-analysis? BMJ, 351, Article h4419. https://doi.org/10.1136/bmj.h4419

Shen, F., Sheer, V. C., & Li, R. (2015). Impact of narratives on persuasion in health communication: A meta-analysis. Journal of Advertising, 44(2), 105–113. https://doi.org/10.1080/00913367.2015.1018467

Shi, R., Messaris, P., & Cappella, J. N. (2014). Effects of online comments on smokers’ perception of antismoking public service announcements. Journal of Computer-Mediated Communication, 19(4), 975–990. https://doi.org/10.1111/jcc4.12057

Stok, F. M., Verkooijen, K. T., de Ridder, D. T. D., de Wit, J. B. F., & De Vet, E. (2014). How norms work: Self‐identification, attitude, and self‐efficacy mediate the relation between descriptive social norms and vegetable intake. Applied Psychology: Health and Well‐Being, 6(2), 230–250. https://doi.org/10.1111/aphw.12026

Stroud, N. J., Duyn, E., & Peacock, C. (2016). Survey of commenters and comment readers. Center for Media Engagement. https://mediaengagement.org/research/survey-of-commenters-and-comment-readers/

Stylianou, S., & Sofokleous, R. (2019). An online experiment on the influence of online user comments on attitudes toward a minority group. Communication & Society, 32(4), 125–142. https://doi.org/10.15581/003.32.28955

Sundar, S. S., Jia, H., Waddell, T. F., & Huang, Y. (2015). Toward a theory of interactive media effects (TIME) four models for explaining how interface features affect user psychology. In S. S. Sundar (Ed.), The handbook of the psychology of communication technology (pp. 47–86). https://doi.org/10.1002/9781118426456.ch3

Trumbo, C. W., & McComas, K. A. (2003). The function of credibility in information processing for risk perception. Risk Analysis: An International Journal, 23(2), 343–353. https://doi.org/10.1111/1539-6924.00313

Tsfati, Y., & Cappella, J. N. (2003). Do people watch what they do not trust? Exploring the association between news media skepticism and exposure. Communication Research, 30(5), 504–529. https://doi.org/10.1177/0093650203253371

Viechtbauer, W. (2010). Conducting meta-analyses in R with the metafor package. Journal of Statistical Software, 36(3), 1–48. https://doi.org/10.18637/jss.v036.i03

von Sikorski, C., & Hänelt, M. (2016). Scandal 2.0: How valenced reader comments affect recipients’ perception of scandalized individuals and the journalistic quality of online news. Journalism & Mass Communication Quarterly, 93(3), 551–571. https://doi.org/10.1177/1077699016628822

Waddell, T. F. (2018). What does the crowd think? How online comments and popularity metrics affect news credibility and issue importance. New Media & Society, 20(8), 3068–3083. https://doi.org/10.1177/1461444817742905

Waddell, T. F. (2019). When comments and quotes collide: How exemplars and prior attitudes affect news credibility. Journalism Studies, 20(11), 1598–1616. https://doi.org/10.1080/1461670x.2018.1533415

Waddell, T. F., & Sundar, S. S. (2017). #thisshowsucks! The overpowering influence of negative social media comments on television viewers. Journal of Broadcasting & Electronic Media, 61(2), 393–409. https://doi.org/10.1080/08838151.2017.1309414

Walther, J. B., DeAndrea, D., Kim, J., & Anthony, J. C. (2010). The influence of online comments on perceptions of antimarijuana public service announcements on YouTube. Human Communication Research, 36(4), 469–492. https://doi.org/10.1111/j.1468-2958.2010.01384.x

Walther, J. B., Neubaum, G., Rösner, L., Winter, S., & Krämer, N. C. (2018). The effect of bilingual congruence on the persuasive influence of videos and comments on YouTube. Journal of Language & Social Psychology, 37(3), 310–329. https://doi.org/10.1177/0261927x17724552

Wang, R., & Sundar, S. S. (2018). How does parallax scrolling influence user experience? A test of TIME (theory of interactive media effects). International Journal of Human–Computer Interaction, 34(6), 533–543. https://doi.org/10.1080/10447318.2017.1373457

Weber, P. (2014). Discussions in the comments section: Factors influencing participation and interactivity in online newspapers’ reader comments. New Media & Society, 16(6), 941–957. https://doi.org/10.1177/1461444813495165

Wiencierz, C., Pöppel, K. G., & Röttger, U. (2015). Where does my money go? How online comments on a donation campaign influence the perceived trustworthiness of a nonprofit organization. International Journal of Strategic Communication, 9(2), 102–117. https://doi.org/10.1080/1553118X.2015.1008634

Winter, S., Krämer, N. C., Benninghoff, B., & Gallus, C. (2018). Shared entertainment, shared opinions: The influence of social TV comments on the evaluation of talent shows. Journal of Broadcasting & Electronic Media, 62(1), 21–37. https://doi.org/10.1080/08838151.2017.1402903

Wise, D., & McLaughlin, B. (2016). In media we distrust: The interplay of message, context, and media trust on campaign message effects. Electronic News, 10(2), 105–120. https://doi.org/10.1177/1931243116647769

Witteman, H. O., Fagerlin, A., Exe, N., Trottier, M.-E., & Zikmund-Fisher, B. J. (2016). One-sided social media comments influenced opinions and intentions about home birth: An experimental study. Health Affairs, 35(4), 726–733. https://doi.org/10.1377/hlthaff.2015.1382

Wu, T.-Y., & Atkin, D. J. (2018). To comment or not to comment: Examining the influences of anonymity and social support on one’s willingness to express in online news discussions. New Media & Society, 20(12), 4512–4532. https://doi.org/10.1177/1461444818776629

Yu, H. (2014). The effect of social media comment on Chinese consumers’ attitude toward the brand [Mater’s thesis, Iowa State University]. ProQuest Dissertations & Theses Global. https://www.proquest.com/dissertations-theses/effect-social-media-comment-on-chinese-consumers/docview/1620682318/se-2?accountid=14696

Zerback, T., & Fawzi, N. (2017). Can online exemplars trigger a spiral of silence? Examining the effects of exemplar opinions on perceptions of public opinion and speaking out. New Media & Society, 19(7), 1034–1051. https://doi.org/10.1177/1461444815625942

Zhao, D., Wang, F., Wei, J., & Liang, L. (2013). Public reaction to information release for crisis discourse by organization: Integration of online comments. International Journal of Information Management, 33(3), 485–495. https://doi.org/10.1016/j.ijinfomgt.2013.01.003

Zhang, W., & Wang, Q. (2019). The failure of news coverage supportive of human papillomavirus vaccination: The investigation of the effects of online comments on female college students’ vaccination intention. Vaccine, 37(38), 5681–5687. https://doi.org/10.1016/j.vaccine.2019.08.007

Zillmann, D. (1999). Exemplification theory: Judging the whole by some of its parts. Media Psychology, 1(1), 69–94. https://doi.org/10.1207/s1532785xmep0101_5

Zillmann, D. (2002). Exemplification theory of media influence. In J. Bryant & D. Zillmann (Eds.), Media effects: Advances in theory and research (pp. 19–41). Lawrence Erlbaum Associates.

Additional information

Authors’ Contribution

Junhan Chen: conceptualization, data curation, formal analysis, investigation, writing—original draft, writing—review & editing. Shilin Xia: conceptualization, data curation, formal analysis, writing—original draft, writing—review & editing.

 

Editorial Record

First submission received:
April 18, 2023

Revisions received:
January 1, 2024
June 15, 2024

Accepted for publication:
September 20, 2024

Editor in charge:
Michel Walrave

Full text

Introduction

Numerous online media such as social media and news websites provide users with the affordance of posting and reading comments. This affordance has been well-utilized by online users. Research shows that 77.9% of online users have read comments at some point (Stroud et al., 2016). Among people who read comments but do not post comments, the majority read comments at least once a week (Stroud et al., 2016).

As the act of reading online comments has become an integral part of people’s daily online media consumption, there has been a growing focus in research on understanding how these comments impact users’ cognition, emotion, and behavior. Within this body of research, one of the primary emphases has been on the influence of comment valence—distinguishing between positive comments that convey opinions that align with, support, or favor the opinions expressed in the original message, and negative comments that convey opinions that oppose, criticize, or disagree with the opinions expressed in the original message. Research has shown that online comment valence influences various psychological and behavioral outcomes such as perceived credibility of the original article (e.g., Waddell, 2018), likelihood to express opinion (e.g., Duncan & Coppini, 2019), emotion (e.g., Kuttschreuter & Hilverda, 2019), attitudes toward the focal issue (H. Kim et al., 2020), and intentions to engage in the focal behavior (e.g., Krämer et al., 2021). However, researchers have found diverse effect sizes, ranging from small (e.g., Dempsey et al., 2022) to large (e.g., Ballantine et al., 2015). In some cases, research yielded inconsistent results regarding whether there was a significant effect or not. For example, H. Kim et al. (2020) found that reading more favorable comments about the flu vaccine led to a more favorable attitudes toward the vaccine. However, Walther et al. (2010) did not find significant differences in attitudes toward marijuana use between people who read comments that supported an anti-marijuana public service announcement and those who read derisive comments.

Considering the mixed findings, conducting a meta-analytical synthesis of the existing evidence will aid in understanding the extent to which online comment valence influences people’s cognition, emotion, and behavior. Additionally, it will offer insights into the reasons for the observed variations in effect sizes. We begin by offering an overview of existing theories related to the impact of online comments, based on which we proposed a framework that predicts the cognitive, emotional, and behavioral outcomes resulting from online comment valence. Then, we used a meta-analysis approach to gauge the strengths of the predicted effects. Given the considerable variation in the effects observed in empirical studies, we also explored factors that may enhance or diminish the influence of online comment valence on these outcomes.

Theoretical Basis for the Effects of Online Comment Valence

A variety of existing theories have shed light on and guided empirical research on the influence of online comment valence, including a theory of interactive media effects (TIME; Sundar et al., 2015), exemplification theory (Zillmann, 1999), social norm influence (Cialdini et al., 1990), spiral of silence theory (Noelle-Neumann, 1993), and emotional contagion (Hatfield et al., 1993). Although to different extents, these theories all imply the power of social influence in shaping individuals’ emotional, cognitive, and/or behavioral outcomes. In other words, online comments of either positive or negative valence indicate a majority opinion and/or behavior, which modifies audiences’ emotional, cognitive, and/or behavioral outcomes to be consistent with the majority because individuals have the tendency to conform to and follow majority opinions and behaviors.

TIME

TIME (Sundar et al., 2015) posits that interactive media provide affordances, which refers to the action possibilities offered by the interactive media. For example, a news website with a comments section provides the affordance of commenting on the news articles. Affordances affect users’ perceptions, knowledge, and behaviors through two routes. First, affordances can trigger users’ actions. For example, the affordance of parallax scrolling on a product presentation website triggers user engagement, which in turn results in positive attitudes and behavioral intentions toward the product (Wang & Sundar, 2018). Second, affordances can serve as cues on the media interface. For example, the number of likes on a social media post may activate bandwagon heuristics about the nature of the post and in turn affect users’ evaluation of the post’s content and source (Sundar et al., 2015).

Within TIME, the Modality-Agency-Interactivity-Navigability (MAIN) model further specifies that four technological affordances—modality, agency, interactivity, and navigability—can cue a range of heuristics that affect users’ perceptions of media content (Sundar et al., 2015). Of these, the agency affordance is relevant to the multilayers of media content sources and its influence on users’ judgement on message quality and credibility. One of the layers of media content sources is the users themselves; in online media, users have the ability to be a source through activities such as commenting, liking, and sharing. MAIN posits that metrics representing the collective opinion of other users (e.g., number of views and likes) can cue the bandwagon heuristic and affect users’ evaluation of the original message content (Sundar, 2015). For example, an online video frequency metric that indicates a higher number of viewers may lead to more media enjoyment through influencing perceptions of audience size (Waddell & Sundar, 2017).

In a similar vein, online comments that are predominantly positive or negative can also serve as an agency cue which triggers bandwagon heuristic and influences users’ evaluation of the original message. In general, more positive (vs. negative) comments affect perceptions of majority opinions and result in more positive evaluation of the original message content (Lu & Sun, 2022; Waddell, 2018; Waddell & Sundar, 2017). Also, studies found that positive (vs. negative) comments led to more positive evaluation of companies’ Facebook pages (Y.-J. Lee et al., 2018), news articles (Ahn, 2011; von Sikorski & Hänelt, 2016), political satire shows (Möller & Boukes, 2023), and public service announcements (Shi et al., 2014; Walther et al., 2010, 2018).

Exemplification Theory and Social Norm Influence

Exemplification theory posits that people draw inferences and generalize about a larger group of people, objects, or phenomena based on a small number of examples that can represent broader phenomena of the same kind (Zillmann, 1999). Online comments can be exemplars from which people infer majority public opinions (E.-J. Lee & Tandoc, 2017; Waddell, 2019). When the comments are predominantly positive or negative, individuals may infer that the majority of the public hold a positive or negative opinion on the issue or the message content (Waddell, 2018). As predicted by the social norm literature (e.g., Cialdini et al., 1990), the perception that most people endorse an opinion or engage in a behavior (i.e., descriptive norm) may influence individuals’ judgment on the appropriate opinion and behavior for themselves. In other words, individuals who are informed of a majority norm tend to have attitudes that are consistent with the norm and subsequently stronger intentions to engage in the behavior suggested by it (Stok et al., 2014). Moreover, a meta-analysis has shown that a descriptive norm has a medium to strong association with behavioral intentions (Rivis & Sheeran, 2003). In short, predominantly more positive or negative online comments may lead to beliefs, attitudes, and behavioral intentions that are consistent with the majority opinions indicated by the comments.

Consistent with the prediction, a number of studies have shown that individuals tend to report beliefs, attitudes, and/or behavioral intentions that are consistent with the majority opinion suggested by the comments (Ahn, 2011; Ballantine et al., 2015; Cameron & Geidner, 2014; Duncan & Coppini, 2019; Hsueh et al., 2015; H. Kim et al., 2020; Kim, 2021; Kuttschreuter & Hilverda, 2019). For example, reading comments that are supportive (vs. unsupportive) of the application of nanotechnology in foods led to lower risk perception, higher benefit perception, and more positive attitude toward nanodesigned food (Kuttschreuter & Hilverda, 2019). In addition, individuals who read comments in favor of (vs. against) vaccination expressed slightly stronger intention to get vaccinated (Peter et al., 2014).

Spiral of Silence Theory

The spiral of silence theory (Noelle-Neumann, 1993) posits that individuals’ willingness to express their opinions depends on how they perceive public opinion and the extent to which their own opinions align with the perceived public opinion. The theory assumes that individuals fear that deviating from social consensus will result in isolation from society (Scheufle & Moy, 2000). Because of this fear of social isolation, individuals consistently monitor public opinions, and they are inclined to publicly express their opinions when they perceive that their viewpoint is dominant or on the rise; conversely, they tend to remain silent when they perceive that their opinion is in the minority (Scheufle & Moy, 2000).

In light of spiral of silence theory, comment valence influences individual willingness to express their opinions; individuals are more likely to post comments when their views align with the majority’s opinion, and are less likely to post comments when their views diverge from the majority. Consistent with this prediction, empirical studies have shown that individuals are more likely to express opinions consistent with the majority comments in various contexts, such as prejudice toward racial minorities (Hsueh et al., 2015), ideal body norm (Flynn, 2012), abortion (Wu & Atkin, 2018), and fake news (Colliander, 2019).

Emotional Contagion

Emotional contagion theory suggests that people catch emotions through social interactions (Hatfield et al., 1993). Emotional contagion theory originally focused on face-to-face interactions, positing that individuals pick up emotions and automatically mimic and synchronize others’ facial expressions, voices, postures, and movements during a conversation (Hatfield et al., 1993). As an extension of the original focus, Kramer et al. (2014) argued that emotional contagion can happen through interactions on social media as well: On social media, individuals can observe each other’s experiences by reading emotion-ladened content that expresses positive or negative emotions, which can trigger emotional contagion (Kramer et al., 2014; Kwon & Gruzd, 2017). Consistent with this prediction, studies on online comments have shown that online comments pass along positive (e.g., joy, glad) or negative emotions (e.g., sadness, disgust, anger) when the original messages are about organizational crisis (Zhao et al., 2013) and vaccination (Lu & Sun, 2022).

Current Study: Framework Constructed and Terms Defined

Synthesizing the relevant theories reviewed above, we constructed a framework for the effects of online comment valence (Figure 1). In summary, TIME and MAIN suggest that comment valence influences message perceptions. Exemplification theory and social norm influence suggest that comment valence influences issue-relevant beliefs, attitudes, and behavioral intentions. Spiral of silence theory suggests that comment valence influences communication intentions. And emotional contagion theory suggests that comment valence influences emotions.

Figure 1. The Framework Constructed for the Current Study.

Here we define positive comments as comments conveying opinions that align with, support, or favor the opinions expressed in the original message (e.g., pro-vaccine comments following a pro-vaccine social media post; Zhang & Wang, 2019). In the same vein, negative comments refer to comments conveying opinions that oppose, criticize, or disagree with the opinions expressed in the original message (e.g., pro-smoking comments following an anti-smoking public service announcement; Shi et al., 2014). Regarding the outcomes, message perception refers to the evaluation (e.g., credibility; Waddell, 2019) of the original message to which users respond (e.g., social media post, news articles, YouTube videos), rather than the subsequent comments. We define issue-relevant beliefs and attitudes as the beliefs or attitudes toward the issue being discussed in the original post and/or the comments (e.g., attitudes toward water conservation; Walther et al., 2018). And we define issue-relevant behaviors and behavioral intentions as the actual behaviors or the intentions to engage in behaviors that are relevant to the issue being discussed in the original post and/or the comments (e.g., intention to quit smoking; Shi et al., 2014). Communication behaviors and intentions refer to the actual behaviors or intentions to express opinions that are consistent with the comment valence (e.g., likelihood to make positive comments after reading positive versus negative comments; Colliander, 2019). Lastly, emotions refer to any affective feelings that are elicited after reading online comments (e.g., anxiety, happiness; Hilverda et al., 2018).

Given the theoretical basis and the definitions, we proposed the following hypotheses. Through a meta-analysis approach, we also assessed the sizes of the hypothesized effects (RQ1).

H1–5: Individuals exposed to predominantly positive (compared to negative) comments will exhibit the following: a more positive evaluation of the original message (H1), stronger issue-relevant beliefs and attitudes aligned with the positive comments (H2), increased likelihood of engaging in behaviors advocated by the positive comments (H3), increased likelihood to express opinions consistent with the positive comments (H4), and stronger positive emotions (H5).

RQ1: What are the sizes of the effects predicted in H1–5?

Factors Moderating the Effects of Online Comment Valence

The sizes of the proposed comment valence effects may vary depending on the features of the comments and the original messages. In particular, we discuss two comment features (i.e., number of comments and whether the comment valence is mixed or not) and one message feature (i.e., news versus non-news) in this section.

Comment Features

Exemplification theory posits that more frequently activated exemplars are more likely to create an endurable influence on individuals’ judgment about the frequency of the event (Zillmann, 2002). Because comments are exemplars of public opinion, individuals may hold a stronger perception of the majority opinion as they read more comments supporting/opposing a position, and thus adjust their judgment to align with the majority opinion. Consistent with this prediction, empirical studies found that when there were 10 exemplar comments about the eviction of violent immigrants, individuals who read favorable (vs. unfavorable) comments had a more favorable opinion about eviction, whereas this difference was not significant when they read only two exemplar comments (Zerback & Fawzi, 2017). However, the effect of comment valence seems diminished as the number of exemplars is higher than 10. In e-cigarette and genetically modified food contexts, individuals who read 10 (vs. 20) supportive or unsupportive comments did not differ in their perceptions of e-cigarette and genetically modified food (Liu, 2017).

In real-life situations, the valence of comments is often mixed. Predominantly negative comments may be accompanied by a small portion of positive or neutral comments and vice versa (e.g., Liao & Mak, 2019). Because individuals’ perceived frequency of an event is based on the relative exemplar frequencies (Zillmann, 2002), individuals exposed to comments uniformly expressing the same valence (i.e., all positive or all negative) may perceive a stronger majority opinion compared to individuals reading comments with mixed valence (i.e., mostly positive or mostly negative). Empirical studies supported this prediction. It was found that unanimously positive (negative) comments about a political candidate led to more favorable (unfavorable) opinions about the candidate, whereas mixed-valence comments had no effect in changing opinions about the candidate (Duncan & Coppini, 2019). Similarly, Kuttschreuter and Hilverda (2019) compared positive, mixed, and negative comments about nanodesigned food. They found that more positive comments led to lower risk perception, higher benefit perception and more positive attitude toward nanodesigned food than perceptions and attitudes following mixed and negative comments. Based on this evidence, it is likely that comments with mixed valence have a smaller effect size compared to comments with non-mixed valence. Therefore, we examined moderating roles of comment quantity and the mixture of valences in comments on the effects of comment valence.

RQ2–3: How are the hypothesized effects moderated by the number of comments (RQ2) and whether the comment valence is mixed or not (RQ3).

Original Message Feature

Individuals may comment on original messages involving various content types such as news, entertainment stories, public service announcements, and product advertisements. We are particularly interested in the influence of comments following news articles because such influence has significant social implications. Unlike non-news content, such as promotional material from various organizations and individual experiences or opinions, news stands out as the principal platform reporting matters of societal importance. In addition, to enhance news objectivity and credibility, news pieces often incorporate perspectives from multiple sources within society (Lee & Tandoc, 2017). As such, individuals typically acknowledge the wider societal implications of the topics under discussion while reading news (e.g., McIntyre, 2017). Comments accompanying news pieces are integral components of the discourse. Therefore, reading other people’s comments following a news article may induce a feeling of collective relevance and involvement with the news event and a more favorable perceived public opinion climate (Ksiazek & Springer, 2019; Weber, 2014). These feelings and perceptions may increase the willingness to leave a comment, which creates a greater potential for public discourse on important social issues (Ksiazek & Springer, 2019; Weber, 2014). In addition, news comment valence affects news readers’ evaluation of the news articles and journalistic quality (Ksiazek & Springer, 2019), which may influence individual trust in journalists as an information source and shape the ways individuals seek out and process news reports about important social issues in the future (Kümpel & Unkel, 2020; Trumbo & McComas, 2003; Tsfati & Cappella, 2003). Specifically, we investigated 1) if the valence of comments following a news article had significant impact on the outcomes, and 2) if the valence of comments following a news article was more or less impactful on the outcomes than the valence of comments following a non-news message.

RQ4: How are the hypothesized effects moderated by whether the original message is news or non-news?

Methods

We used a meta-analysis approach to examine the hypotheses and the research questions. In addition, we investigated how the effect sizes are influenced by the number of comments, whether the comment valence is mixed or non-mixed, and the original message feature.

Search Strategy

To systematically synthesize the existing evidence on the effects of comment valence, we aimed to conduct a comprehensive search of the literature to capture all existing empirical studies that examined the causal impact of positive (vs. negative) online comments on one or more of the outcomes listed in H1–5. Given that the effects of comment valence can be examined across different contexts, we conducted searches across 12 databases1 relevant to the domains of communication, psychology, public health, information science, political science, and the broader social sciences. The search was conducted through EBSCOhost and ProQuest in December 2021 and August 2022. To begin broadly, we did not specify outcomes in the search string and included only keywords pertinent to online comments. The final search string is “online comment*” OR “news comment*” OR “newspaper comment*” OR ((TikTok OR YouTube OR Facebook OR Twitter OR Instagram OR “social media”) AND “comment*”). The wildcard searching technique was used to include possible variations of keywords (e.g., comment, comments, commenting). Title and abstract were searched. We limited the search to scholarly reviewed journal articles, conference papers, dissertations, and theses written in English. There was no restriction on the year of publication. The initial search resulted in 4,729 items. Figure 2 shows the searching and screening steps.

Figure 2. Process of Searching and Screening.

Next, 1,078 duplicate items were removed. The abstracts and full texts of the remaining 3,651 items were screened by two researchers. The articles are screened based on the following inclusion criterion: the article must report at least one experiment that examined the causal impact of positive (vs. negative) online comments on one or more of the following outcomes: (a) message perception (b) issue-relevant beliefs and attitudes, (c) issue-relevant behavior and behavioral intentions, (d) emotions, (e) communication intentions. After the screening, there were 42 articles remaining in the sample. Then, we searched the reference lists of the included articles. There were 15 articles that met the inclusion criteria and were added to the sample. Finally, 57 articles (64 studies) were coded and assessed for meta-analysis eligibility.

Coding Category and Intercoder Reliability

For publications that presented more than one study, each study was coded individually. For each study, we coded study context, number of participants, the number of comments, comment valence (i.e., mixed or not mixed), and media platform that comments were posted: Twitter, Facebook, News website, YouTube, Instagram, or other. We also coded the type of original post (i.e., news article or non-news article) reported by each study. Specifically, we referred news article as news reports posted on a webpage or on social media. Studies in which the original post was not a news report and studies that did not have an original post were coded as non-news article. Finally, we coded the effect size for each study.

Two coders independently coded all included studies. All discrepancies were resolved based on discussion. Krippendorff’s alpha showed good intercoder reliability, ranging from a low of .85 to a high of .97, which met the .8 criterion (Krippendorff, 2004).

Effect Size Calculation

The correlation r was used as the effect size indicator. The correlation rs were computed from other statistics including means and standard deviations, F statistics, and Chi-squares. We did the computation using an online application developed by Lenhard and Lenhard (Lenhard & Lenhard, 2016).

For most studies, only one correlation r for one dependent variable could be computed from a study. Other studies measured more than one outcome variable that can represent one of the outcomes we are interested in (i.e., message perception, beliefs and attitudes, behavior intentions, emotions, and communication intentions). That means, more than one Cohen’s ds (Cohen, 1969) could be computed from the study. In such cases, we computed the effect sizes of each outcome variable and used the average as the effect size indicator. Our treatment of the dependent variables was consistent with the practices in prior research (e.g., Shen et al., 2015).

Number of Studies for Each Outcome

A total of 101 studies examined message perception (19 studies), issue-relevant beliefs and attitudes (47 studies), issue-relevant behaviors and behavioral intentions (21 studies), communication behaviors and intentions (8 studies), and emotions (6 studies) as the outcome variables. For studies that did not provide the number of participants in positive and negative comment conditions, we assumed equal sample size across conditions to calculate the effect sizes. Also, for studies with inadequate data for effect size calculation, we contacted the authors to request the data. If the authors did not provide the data after being contacted twice, the study was excluded from the analysis. Finally, the analysis included 10, 31, 13, 5, and 4 studies for message perception, issue-relevant beliefs and attitudes, issue-relevant behaviors and behavioral intentions, communication behaviors and intentions, and emotions, respectively.

Meta-Analytic Approach

We used metafor R package (Viechtbauer, 2010) for the analysis. The effect sizes were weighted by inverse variances. Random effect models were fitted using the Hunter and Schmidt method because this method corrects for variance due to artifacts such as sampling error, reliability of measurement and range restriction (Hunter & Schmidt, 2004). The I2 was used to test heterogeneity among the effect sizes as recommended by Higgins and Thompson (2002). Publication bias was examined using funnel plots and using Egger’s regression test to assess the asymmetry of the funnel plots (Egger et al., 1997). In moderation analysis, we followed the recommendation by Fu et al. (2011): to conduct subgroup meta-analysis, there should be at least 6–10 studies for a continuous study-level variable and at least 4 studies for each categorical subgroup when the sizes of the included studies are moderate or large. Therefore, for the continuous moderator (i.e., number of comments), if the total number of studies was smaller than 6, then moderation analysis was not conducted. For categorical moderators, categories that had fewer than 4 studies were excluded from the moderation analysis. If none or only one of the categories had 4 or more studies, then the moderation analysis was not conducted.

Result

Description of Studies

Included studies (k = 48) were published between 2007 to 2022, in journals across diverse areas such as communication (e.g., New Media and Society and Human Communication Research), marketing (e.g., Journal of Business Research and International Journal of Advertising), and public health (e.g., Vaccine and Drug and Alcohol Review). Studies were conducted in various contexts such as COVID-19 vaccination, political elections, water conservation, and brand marketing. Table 1 shows characteristics and effect sizes for each study by outcome type.

Table 1. Summary of Study Characteristics and Effect Sizes by Outcome Type.

Study

Sample Size

Correlation

r

Original Message

Platform

# of Comments

Mixed-valence

Context

Outcome Variables

Outcome: Message Perception

Ahn (2011)

80

.23

News

Unreported

4

No

University-related news

Perception of news story

Colliander (2019) Study 2

400

.27

News

Facebook

4

No

Fake news

Attitude towards the fake news

Houston et al. (2011)

134

.05

News

News website

unreported

unreported

Political election

Media bias

Y.-J. Lee et al. (2018)

254

.28

Non news

Twitter and Facebook

4

No

Corporate social responsibility (CSR)

Attitude towards the social media page

Lu and Sun (2022)

465

.30

Non news

Facebook

5

No

COVID-19 vaccination

Negative cognition of the post

Pjesivac et al. (2018)

196

−.01

News

News website

3

No

Genetically modified organisms (GMO) food

Perceived credibility of the article

Sadri (2012)

252

.15

News

News website

3

No

Sports

Perceived credibility of the article

Von Sikorski and Hänelt (2016)

55

.24

News

News website

7

No

Financial scandal

Perceived journalistic quality of the article

Walther et al. (2010)

152

.32

Non news

YouTube

5

No

Marijuana

Evaluation of the public service announcement

Walther et al. (2018)

191

.21

Non news

YouTube

10

Yes

Water conservation

Evaluation of the public service announcement

Outcome: Issue-relevant Beliefs and Attitudes

Ballantine et al. (2015)

453

.64

Non news

Facebook

10

No

Romantic relationship

Attitude towards Facebook relationship status update

Cameron and Geidner (2014) Study 1

54

.30

Non news

Twitter

unreported

Yes

Entertainment TV program

Opinion toward the singer’s performance; perceived likelihood of the singer winning

Cameron and Geidner (2014) Study 2

96

.37

Non news

Twitter

unreported

Yes

Political TV program

Opinion toward the speaker; perceived likelihood of the speaker winning

Chung (2019)

50

−.09

News

Unreported

7

No

E-cigarettes

Attitude towards a) e-cigarettes, b) the use of e-cigarettes, and c) the regulation of e-cigarettes.

Dempsey et al. (2022) Study 1

136

−.02

Non news

Facebook

8

Yes

Depression

Personal stigma

Dempsey et al. (2022) Study 2

463

.04

Non news

Facebook

8

Yes

Depression

Personal stigma

Duncan et al. (2019)

231

.19

Non news

Other

6

Yes

Political election

Opinion toward the candidate

Duong et al. (2020)

366

.25

News

News website

10

Yes

Child corporal punishment

Attitude towards child corporal punishment

Flynn (2012)

501

.03

Non news

Facebook

3

unreported

Body image

Perceptions about body image

Hilverda et al. (2018)

124

.06

Non news

Facebook

4

No

Organic food

Risk perception

Hsueh et al. (2015)

137

.21

News

Other

12

No

University funding

Explicit and implicit prejudice responses

Kim (2021)

182

.26

Non news

Instagram

4

No

Body image

Perception of the posted body image

H. Kim et al. (2020)

271

.07

Non news

Facebook

4

unreported

Flu vaccination

Attitude towards flu vaccine

Kuttschreuter and Hilverda (2019)

209

.18

Non news

Facebook

4

No

Nanotechnology food

Risk perception; benefit perception; perceived retail safety; attitude towards nano-designed food

J. Lee and Lim (2014)

520

.62

Non news

Facebook

2

No

Political election

Perceived trustworthiness of the political candidate

Y.-J. Lee et al. (2018)

254

.24

Non news

Twitter and Facebook

4

No

CSR

Perceived legitimacy

Y.-I. Lee et al. (2020)

206

.68

Non news

Facebook

2

No

Marketing

Brand trust; brand attitude

Lee et al. (2022) Study 1

140

.21

News

News website

5

No

GMO food

Attitude towards GMO food

Lee et al. (2022) Study 2

146

.34

News

News website

5

No

GMO food

Attitude towards GMO food

Liao and Mak (2019)

102

.04

Non news

YouTube

4

Yes

CSR

Perceived credibility of the company; attitude towards the company’s social responsibility

Lu and Sun (2022)

465

.03

Non news

Facebook

5

No

COVID-19 vaccination

Vaccine hesitancy

Park and Lee (2007)

80

.56

News

Other

1 or 5

No

Consumer/brand

Perception of the company’s social responsiveness and employee treatment

Shi et al. (2014)

404

.11

Non news

Other

10

Yes

Smoking

Attitude and perceived risk of smoking

Stylianou and Sofokleous (2019)

58

.19

Non news

Other

9

No

Refugee

Attitude toward refugees

Von Sikorski and Hänelt (2016)

55

.33

News

News website

7

No

Financial scandal

Perceived responsibility; attitudes toward scandalized individual

Walther et al. (2010)

152

.08

Non news

YouTube

5

No

Marijuana

Marijuana attitude

Wiencierz et al. (2015) Study 1

219

.31

Non news

Facebook

4

No

Nonprofit organization (NPO)

Perceived trustworthiness of the NPO

Wiencierz et al. (2015) Study 2

89

.23

Non news

Facebook

4

No

Nonprofit organization

Perceived trustworthiness of the NPO

Winter et al. (2018)

77

.21

Non news

Other

9

No

Antisocial behavior and conventional performance

Evaluation of a) the show, b) the judge, and c) the candidate

Witteman et al. (2016)

1,128

.41

News article

News website

10

No

Home birth

Opinion of home birth

Yu (2014)

70

.72

Non news

Other

unreported

unreported

Marketing

Attitude towards the brand

Outcome: Issue-relevant Behaviors and Behavioral Intentions

Dempsey et al. (2022) Study 1

136

.03

Non news

Facebook

8

Yes

Depression

Intention to support the character

Dempsey et al. (2022) Study 2

390

−.15

Non news

Facebook

8

Yes

Depression

Intention to support the character

Hayes et al. (2018)

405

.13

Non news

Facebook

1

No

Online video ads

Referral acceptance (click the ad shared by others)

Hilverda et al. (2018)

124

.05

Non news

Facebook

4

No

Organic food

Willingness to buy

Kuttschreuter and Hilverda (2019)

209

.09

Non news

Facebook

4

No

Nanotechnology food

Willingness to buy

Y.-J. Lee et al. (2018)

254

.14

Non news

Twitter and Facebook

4

No

CSR

Purchase intention

Y.-I. Lee et al. (2020)

206

.15

Non news

Facebook

2

No

Marketing

Purchase intention

Liu (2017) Study 1

392

.08

News

News website

10 or 30

Yes

E-cigarettes

Intention to use e-cigarettes

Liu (2017) Study 2

387

.03

News

News website

10 or 30

Yes

GMO food

Intention to check GMO labels

Noel (2021)

286

.15

Non news

Facebook

2

No

Alcohol consumption

Purchase intention

Peter et al. (2014)

575

.08

Non news

Facebook

5

Yes

Flu vaccination

Intention to take flu vaccines

Shi et al. (2014)

404

.02

Non news

Other

10

Yes

Smoking

Intention to quit smoking

Witteman et al. (2016)

1,128

.17

News

News website

10

No

Home birth

Intention of home birth

Outcome: Communication Behaviors and Intentions

Colliander (2019) Study 2

1,164

.18

News

Facebook

4

No

Fake news

Likelihood to make positive or negative comments

Colliander (2019) Study 2

400

.36

News

Facebook

4

No

Fake news

Likelihood to make positive or negative comments

Hsueh et al. (2015)

137

.39

News

Other

12

No

University funding

Prejudice expression

Flynn (2012)

402

.42

Non news

Facebook

3

unreported

Body image

Behavioral adherence to social norm

Wu and Atkin (2018)

339

.17

News

News website

3

No

Abortion

Willingness to express opinions consistent with the comments

Outcome: Emotions

Lu and Sun (2022)

465

.10

Non news

Facebook

5

No

COVID-19 vaccination

Anger

Couture Bue and Harrison (2020)

186

.03

Non news

Instagram

unreported

No

Body image

Body anxiety

Hilverda et al. (2018)

124

.11

Non news

Facebook

4

No

Organic food

Anxiety and positive emotions

Kuttschreuter and Hilverda (2019)

209

.22

Non news

Facebook

4

No

Nanotechnology food

Anxiety and positive emotions

Effects of Comment Valence on Emotional, Cognitive, and Behavioral Outcomes

Hypotheses 1–5 predicted that individuals exposed to predominantly positive (compared to negative) comments will have a more positive evaluation of the original message (H1), stronger issue-relevant beliefs and attitudes aligned with the positive comments (H2), increased likelihood of engaging in behaviors advocated by the positive comments (H3), increased likelihood to express opinions consistent with the positive comments (H4), and stronger positive emotions (H5).

Supporting all hypotheses, our results showed that comment valence significantly influenced all five outcomes. Answering RQ1, the inversed-variance-weighted effect sizes for each outcome were rmessage perception = .22, 95% CI [.16, .29] (p < .001, k = 10, N = 2,179, I2 = 54.09%), rissue-relevant beliefs and attitudes = .29, 95% CI [.17, .39] (p < .001, k = 31, N = 7,438, I2 = 93.30%), rissue-relevant behaviors and behavioral intentions = .09, 95% CI [.03, .14] (p < .01, k = 13, N = 4,896, I2 = 63.45%), rcommunication behaviors and intentions = .26, 95% CI [.04, .13] (p < .001, k = 5, N = 2,883, I2 = 79.99%), and remotions = .16, 95% CI [.06, .25] (p < .01, k = 4, N = 984, I2 = 51.16%). Figures 3–7 are the forest plots for each outcome.

 Figure 3. Forest Plot—Message Perception Outcome.

Note. The forest plot shows the result of the random effect model. The weighted effect size and the effect size of each study is presented as correlational r with corresponding 95% confidence intervals. The same approach applies to all forest plots in this paper.

Figure 4. Forest Plot—Beliefs and Attitudes Outcome.

 

Figure 5. Forest Plot—Behaviors and Behavioral Intentions Outcome.

 

Figure 6. Forest Plot—Communication Behaviors and Intentions Outcome.

 

Figure 7. Forest Plot—Emotions Outcome.

Moderation Analysis

Number of Comments

To answer RQ2, we tested a number of comments as a moderator of comment valence impact on message perception, issue-relevant beliefs and attitudes, and issue-relevant behaviors and behavioral intentions. We did not conduct moderation analyses regarding emotions and communication behaviors and intentions outcomes because the number of studies for each outcome did not meet the moderation test requirement (Fu et al., 2011).

We found that number of comments significantly moderated the effects of comment valence on message perception (QM = 6.91, df = 1, p = .009), issue-relevant beliefs and attitudes (QM = 6.65, df = 1, p = .010), and issue-relevant behaviors and behavioral intentions (QM = 7.95, df = 1, p = .005). Specifically, as individuals read more comments, comment valence had a stronger effect on message perception (b = 0.06, p = .009) and weaker effects on issue-relevant beliefs and attitudes (b = −0.05, p = .010) and issue-relevant behaviors and behavioral intentions (b = −0.04, p = .005).

Mixed-Valence

To answer RQ3, we tested whether the comment valence was mixed or not as a moderator of comment valence impact on issue-relevant beliefs and attitudes and issue-relevant behaviors or behavioral intentions. We did not conduct moderation analyses regarding message perception, emotions, and communication behaviors and intentions outcomes because the number of studies for each of the mixed versus nonmixed subcategories did not meet the moderation test requirement (Fu et al., 2011).

We found that whether the comment valence was mixed or not significantly moderated the effects of comment valence on issue-relevant beliefs and attitudes (QM = 4.77, df = 1, p= .029) and issue-relevant behaviors or behavioral intentions (QM = 14.40, df = 1, p < .001). Specifically, when the comments included only positive or only negative comments, comment valence had a significant effect on issue-relevant beliefs and attitudes (r = .38, p < .001) and issue-relevant behaviors and behavioral intentions (r = .15, p < .001); when the comments were mixed-valence, comment valence did not affect issue-relevant beliefs and attitudes (r = .14, p = .127) and issue-relevant behaviors and behavioral intentions (r = .02, p = .478). Table 2 shows the moderation results.

Table 2. Moderation Analysis Results.

Moderator/Category

N

k

r

95% CI

Message Perception

Original Message

     

News

1,117

6

.17***a

[.09, .24]

Non-news

1,062

4

.29***b

[.21, .37]

Issue-relevant Beliefs and Attitudes

Mixed-valence

 

 

 

 

Mixed

1,852

8

.14a

[−.04, .32]

Nonmixed

4,744

20

.38***b

[.25, .52]

Original Message

 

 

 

 

News

2,102

8

.36*

[.08, .64]

Non-news

5,336

23

.27***

[.15, .39]

Issue-relevant Behaviors and Behavioral Intentions

Mixed-valence

 

 

 

 

Mixed

2,284

6

.02a

[−.03, .06]

Nonmixed

2,612

7

.15***b

[.10, .19]

Note. *p < .05, **p < .01, ***p < .001. a, bDifferent subscripts show that the effect sizes under the same outcome and moderator variables were significantly different from each another at p < .05 level or better.

Original Message Feature

To investigate RQ4, we tested whether the original message was news or non-news as a moderator of comment valence impact on message perception and issues-relevant beliefs and attitudes. We did not conduct moderation analyses regarding issue-relevant behaviors and behavioral intentions, emotions, and communication behaviors and intentions outcomes because the number of studies for each of the news versus non-news subcategories did not meet the moderation test requirement (Fu et al., 2011).

We found that whether the original message is news or non-news significantly moderated the effects of comment valence on message perception (QM = 5.17, df = 1, p = .023). Specifically, when the original message was a news article, comment valence had a smaller effect on message perception (r = .17, p < .001) than when the original message was not a news article (r = .29, p < .001). Whether the original message is news or non-news did not significantly moderate the effect of comment valence on issue-relevant beliefs and attitudes (QM = 0.32, df = 1, p = .572).

Publication Bias

Publication bias refers to the phenomenon that a study is more likely to be published if the findings are statistically significant (Sedgwick, 2015). Figures A1–A5 show the funnel plots for each outcome. Although visual examination of the funnel plots suggest asymmetry, the Egger’s tests indicated that there was no publication bias regarding message perception (z = −1.15, p = .250), issue-relevant beliefs and attitudes (z = −0.61, p = .545), and issue-relevant behaviors and behavioral intentions (z = −0.92, p = .359). However, there was publication bias regarding communication behaviors and intentions (z = 2.01, p = .045) and emotions (z = −2.80, p = .005).

Discussion

Result Summary

The results show that compared to negative comments, positive comments led to more positive message perception (r = .22), stronger issue-relevant beliefs and attitudes that align the positive comments (r = .29), higher likelihood of engaging in behaviors advocated by the positive comments (r = .09), higher likelihood to express opinions consistent with the positive comments (r = .26), and stronger positive emotions (r = .16).

This meta-analysis also reveals moderators for the effects of comment valence on different outcomes. First, the number of online comments moderated the effects of comment valence on 1) message perception, 2) issue-relevant belief and attitudes, and 3) issue-relevant behaviors and behavioral intentions. The effect of comment valence on message perception was stronger when the number of comments was greater, whereas the effects of comment valence on both issue-relevant beliefs and attitudes and issue-relevant behaviors and behavioral intentions were weaker when the number of comments was fewer. Second, whether comment valence was mixed or not significantly moderated the effects of comment valence on issue-relevant behaviors or behavioral intentions and issue-relevant beliefs and attitudes, such that comment valence effects were larger when comments were univalent compared to mixed-valent. Comments that were all positive vs. comments that were all negative led to stronger issue-relevant behaviors or behavioral intentions and issue-relevant beliefs and attitudes. However, comments where the majority were positive vs. those where the majority were negative showed no impacts on these two outcomes. Lastly, type of original post (i.e., news article vs. non-news posts) significantly moderated the effects of comment valence on message perception, although such a moderation effect was not found for the outcome of issues-relevant beliefs and attitudes. Comment valence exerted greater effects on message perceptions when the original post was a non-news post (vs. news article).

Impact of Online Comment Valence on Psychological and Behavioral Outcomes

Message Perception

The result that positive (vs. negative) comments lead to more positive perception toward the media content supports the TIME model, which postulates that those online comments, as heuristic cues, shape individuals’ evaluation of the message. The results showed that comment valence has a stronger effect on message perception when individuals read more comments. This is not surprising given that the number of comments itself may serve as a heuristic cue that allows audiences to gauge the reliability of the comment valence in representing the public opinions on the message (Lim & Van Der Heide, 2015). Compared to a few comments, a considerable number of comments can be perceived as a stronger exemplar of public opinion and thus has stronger effects of comment valence on message perceptions. Results also showed that comment valence has a smaller impact on message perception when the original message is news (vs. non-news). It is likely because individuals have relatively stable pre-existing perceptions of news articles given the characteristics of the affiliated news agency (e.g., party affiliation; Landreville & Niles, 2019) and relatively stable levels of trust in news media in general (Fletcher & Park, 2017). Therefore, individuals may rely less on other people’s opinions when reading news articles compared to other types of media content.

Issue-Relevant Beliefs and Attitudes

The results suggest that positive (vs. negative) comments lead to issue-relevant beliefs and attitudes that more align with the stances taken by the positive comments. This shows that online comments, serving as exemplar for majority public opinions or cues for social norms, are powerful to push individuals to opinion conformity (Duncan et al., 2020). The results also showed two moderators of the effects of comment valence on issue-relevant beliefs and attitudes. First, the moderating role of the number of comments was intriguing: comment valence has a stronger effect on issue-relevant beliefs and attitudes when fewer comments are presented. This is possibly because compared to a fewer number of comments, a larger number of comments are often loaded with more information, which can be harder for readers to process and gauge the opinion climate, especially during an issue evaluation process which requires intensive cognitive efforts (Cacioppo et al., 1986).

In addition, whether or not the comments are mixed-valence moderated the effects of comment valence on issue-relevant beliefs and attitudes. First, comment valence significantly affects issue-relevant beliefs and attitudes when the comment was univalent. This is probably because unanimous valence, compared to mixed valence, demonstrates a higher level of consistency of the commenters’ opinions, which yields stronger normative influence in changing issue-relevant beliefs and attitudes (Andrighetto & Vriens, 2022). On the other hand, comment valence has no effect on issue-relevant beliefs and attitudes when the comments are mixed-valanced. This is probably because the small portion of comments with certain valence opposite to the majority comments stand out to the readers as a divergent opinion cluster (Kohout et al., 2023), which may attenuate, offset, and even outweigh the effects of majority comments (Gellerstedt & Arvemo, 2019). The finding suggests a need for future researchers to consider the potential impacts of operationalizations of comment valence (e.g., proportions of comments with varying valences) when investigating and interpreting the effects of comment valence.

The non-significant moderation effect of original message type (i.e., news vs. non-news) suggests that online comment valence influences online users’ issue-relevant beliefs and attitudes, regardless of the original message type. As previous research suggested, overt message cues may diminish the role of message source (Wise & McLaughlin, 2016). In our context, user comments with predominant valence may serve as strong message cues, which may be of paramount importance in the online users’ issue evaluation process.

Behaviors and Behavioral Intentions

In line with predictions from the exemplification theory and existing literature on social norm influence, our results show that positive (vs. negative) comments are effective in promoting behaviors and intentions to perform the behaviors advocated by the positive comments. It should be noted that the effect size of comment valence on behavior and behavioral intentions was much smaller than the effect size for issue-relevant beliefs and attitudes. This may be due to the fact that changes in behaviors and behavioral intentions depend on changes in antecedent variables such as attitudes, subjective norms, and perceived behavioral control (Rivis & Sheeran, 2003), which may render behaviors and behavioral intentions more challenging to change. However, for individuals who habitually read online comments daily, the small effect size on behaviors and behavioral intentions may grow stronger due to repeated exposure to similar online comments over time (Chen et al., 2022).

Communication Behaviors and Intentions

In line with the spiral of silence theory, our finding shows that predominantly positive (vs. negative) comments lead to increased likelihood that the users will express opinions consistent with the positive comments. However, it should be noted that the topics investigated in these studies—such as fake news, body image, and abortion—may be frequently discussed by online users. For instance, individuals often share personal experiences or opinions on issues like abortion or body image to seek or offer social support within online communities (e.g., Lands et al., 2023). Furthermore, people tend to correct misinformation online, driven by the perception that others might be vulnerable to misinformation (e.g., Koo et al., 2021). Thus, more research efforts are needed to test the effects of comment valence on a wider range of topics for more accurate conclusions.

Emotions

The significant effect of positive (vs. negative) comments on emotions supports that emotion contagion could happen in cyberspace through user-generated comments (Kwon & Gruzd, 2017). Studies found that emotions aroused by online media messages may further affect cognition such as perceived health risk (Oh et al., 2021) and motivate behaviors such as engaging in social movement (Ahmed et al., 2017). In this sense, emotion change may be considered a mediator when theorizing the effect of online comments on beliefs, attitudes, and behaviors in future research.

Theoretical Implications

This meta-analysis yields important theoretical implications. First, are online users influenced by what other users say? The evidence we meta-analyzed suggests that they are. Drawing from TIME, exemplification theory, social norm influence, spiral of silence theory, and emotional contagion theory, this meta-analysis provides cohesive support for the proposition that online comment valence is powerful in shaping cognitive, emotional, and behavioral responses.

Second, this analysis demonstrates the efficacy of existing theories in predicting the effects of online comment valence on various outcomes among online users. Specifically, our findings support TIME and MAIN’s postulates that online comment valence, as a heuristic cue, affect users’ perceptions of media content. In line with exemplification theory and the concept of social norm, the results confirm that online comment valence, as exemplars or cues of prevailing opinions, steer online users towards adopting the majority’s issue-relevant beliefs, attitudes, and behaviors. Additionally, the findings align with the fundamental premises of spiral of silence theory, showing that online comment valence prompts individuals to express opinions aligned with the majority’s opinions in online environments. Lastly, the finding that online comments valence evokes corresponding emotions supports emotion contagion theory by expanding its purview from face-to-face interactions to digital spaces. 

Moving forward, this meta-analysis suggests avenues for theoretical refinement. Firstly, instead of focusing on singular psychological or behavioral outcomes evoked by the interactive media (e.g., comment valence), theoretical models should include a wider array of psychological and behavioral outcomes based on empirical evidence. For example, the TIME model may extend its propositions, delineating how bandwagon cues influence not just message perceptions but also issue-related outcomes and subsequent behaviors. Move beyond merely identifying effects, theories should delve into the mechanisms behind these effects across various outcomes. For instance, research might explore if there is a sequence in these outcomes when people are exposed to comment valence. It could be that comment valence initially alters how messages are perceived, which then triggers changes on issue-relevant perceptions that ultimately result in behavior changes. In addition, research could explain how different outcomes interact to produce other effects. For example, message-related perceptions and emotions might interact, leading to shifts in issue-related perceptions. Answering these questions is beyond the scope of our meta-analysis; more research efforts are needed to explore the possibilities presented here.

With respect to findings regarding moderators of comment valence effects, our results showed that the types of original message, number of comments, and compositions of comment valence (i.e., mixed-valence or univalence) should be considered when predicting the effects of online comments. It should be noted that our meta-analysis may only scratch the surface regarding the moderators of comment valence effects. For further theoretical refinement, future meta-analyses and systematic reviews should examine the moderating roles of a broader range of media and message characteristics, such as the type of comments (e.g., real-time vs. retrospective), bandwagon metrics (e.g., emoji reactions), and the comment source (e.g., human vs. machine).

Practical Implications

The current meta-analysis showed that online comments can be a double-edged sword with potential positive and negative impacts on individuals who read the comments. Comments expressing positive opinions toward socially desirable phenomena or expressing negative opinions on socially undesirable phenomena hold promise in fostering desirable social impacts, such as promoting health outcomes (Lu & Sun, 2022) and reducing prejudice toward disadvantaged social groups (Stylianou & Sofokleous, 2019). Conversely, comments conveying positive opinions on socially undesirable phenomena or conveying negative opinions on socially desirable phenomena may exacerbate undesirable social outcomes, such as promoting undesirable health behaviors (Lu & Sun, 2022) and violence and prejudice against minorities (Stylianou & Sofokleous, 2019). 

The finding that comment valence influences cognitive, emotional, and behavioral outcomes highlights the importance of early and continuous monitoring of comments as an integral part of media campaigns. Media practitioners need to recognize that the valence of the comments on their communication material holds substantial power to shape message perceptions, issue-related beliefs, attitudes, behavioral intentions, and emotions. While the presence of only positive comments may imply effective communication efforts, predominantly negative comments may signal a need to adjust communication strategies before the negative comment valence influences more viewers. Therefore, effective media campaigns aimed at public welfare, like those addressing public health concerns, empowering minoritized groups, or advocating for positive public policies, should vigilantly track the online comments’ valence and adjust their communication strategies accordingly for optimal communication effects. In addition, to empower online users in generating and consuming online comments, government agencies, educational organizations, and the media platforms can implement media literacy education and interventions (Cho et al., 2024). As such, online users are expected to be aware of a) the influences of online opinion climate on themselves as media consumers, and b) the impacts of opinion they expressed through online comments on other individuals.

The results of the moderators of comment valence effects also have implications for media practitioners. The insights about the moderating role of the number of comments indicate that, to alter message perceptions, media practitioners can amplify the impact of positive comments by encouraging online comments through compelling and interactive communication methods. When the objective is to influence issue-relevant beliefs, attitudes, behavioral intentions, and behaviors, practitioners may control the number of comments by, for example, limiting the volume of comments or implementing moderation/filtering strategies. The moderating effect of mixed valence suggests that fostering a consistent comment valence that aligns with campaign objectives (through approaches like active user engagement) should be beneficial for changing issue-relevant beliefs, attitudes, behaviors, and behavioral intentions. Lastly, the moderating role of original message frames (news vs. non-news) demonstrates that those disseminating non-news content ought to be particularly vigilant regarding comment valence. Unlike news writers, who experience a comparatively diminished impact of comment valence on message perception, practitioners sharing non-news content should actively monitor comment sentiments and make strategic adjustments in response to comment valence to maintain or alter the desired perception among audiences.

Limitations and Future Research

The study’s limitations offer some insights for interpreting the results and guiding future research efforts. First, because we did not receive replies from some authors after we requested data, we had to exclude a number of studies from analysis. This may lead to less precise and possibly biased effect size estimates of online comments on the outcomes. We would like to call for collective efforts of future researchers, whether actively requesting or responsively providing necessary data, to ensure precise meta-analysis results. The list of articles that were excluded due to the lack of data to calculate effect sizes can be requested from the corresponding author.

Second, the identification of publication bias in communication behaviors and intentions and emotions poses a limitation to our study. This bias, potentially inflating effect sizes, demands caution when interpreting and generalizing the findings regarding these two outcomes. To mitigate bias, future research should report comprehensive data regardless of significance of the effects. Continuous synthesis of updated evidence is crucial for a more accurate understanding of how comment valence impacts communication behaviors, intentions, and emotions. In addition, the high heterogeneity issue was remains unaddressed by our moderation test. Future studies might explore alternative moderators to account for this ongoing high heterogeneity.

Third, we calculated average effect sizes for studies that reported multiple outcome variables under a single outcome category. For example, the effect sizes of brand trust and brand attitudes were averaged to represent the effects on beliefs and attitudes (Y.-I. Lee et al., 2020). Although this method is an effective approach to synthesize studies that measured different outcome variables, the method may overlook valuable information about the effect sizes of specific outcome variables. When there are enough studies examining each specific outcome variable in the future, we would recommend conducting meta-analyses for each outcome variable separately.

Fourth, a number of studies in this review examined potential moderating roles of individual characteristics, such as issue familiarity (Shi et al., 2014) and pre-existing attitudes toward the issue (Lu & Sun, 2022), in comment valence effects. However, only partial studies included in this review investigated the moderation effects of individual characteristics. Also, the wide range of topics (e.g., smoking, fake news, child corporal punishment) investigated across the included studies may result in varying levels of individual attributes, such as issue familiarity, issue involvement, and strengths of pre-existing attitudes, which cannot be objectively coded. Thus, it was not feasible to analyze the extent to which individual traits moderate the overall effect sizes of comment valence across all included studies.

Lastly, despite our comprehensive investigation into the effects of comment valence on cognitive, emotional, and behavioral outcomes, an unexplored facet remains pertinent to the landscape of online comments—the interplay between comment valence and comment civility/incivility. Prior research has convincingly demonstrated that the civility or incivility of comments significantly influences a wide range of cognitive and emotional outcomes (Chen & Lu, 2017; Hwang et al., 2014; Rösner et al., 2016). Thus, it is possible that uncivil comments, relative to civil comments, attenuate the effects of positive vs. negative comments on issue-related attitudes and intentions due to stronger negative emotions (Chen & Lu, 2017), stronger perceived intergroup difference in issue positions (Hwang et al., 2014), and stronger aggressive feelings and hostile cognitions toward the commenters (Rösner et al., 2016). Given little empirical research in this field, continue research efforts are needed to explore the potential interactions between comment valence and comment civility/incivility on various outcomes.

Footnotes

1 The 12 databases included Academic Search Ultimate, Communication & Mass Media Complete, APA PsycInfo, SocINDEX with Full Text, MEDLINE, Library & Information Science Source, Psychology and Behavioral Sciences

Conflict of Interest

The authors have no conflicts of interest to declare.

Acknowledgement

The data and R code can be found on OSF via the link: https://osf.io/34krz/?view_only=7f2ff649d4e743179de1f15fd963a20d.

Appendix

Figure A1. Funnel Plot—Message Perception Outcome.

Note. The dots represent the studies included in the meta-analysis. The white region corresponds to p-values greater than .10. The red region corresponds to p-values between .10 and .05. The orange region corresponds to p-values between .05 and .01. The grey region corresponds to p-values smaller than .01. The same illustration approach applies for all funnel plots in this paper.

 

Figure A2. Funnel Plot—Beliefs and Attitudes Outcome.

 

Figure A3. Funnel Plot—Behaviors and Behavioral Intentions Outcome.

Figure A4. Funnel Plot—Communication Behaviors and Intentions Outcome.

 

Figure A5. Funnel Plot—Emotions Outcome.

Metrics

0

Crossref logo

0


63

Views

8

PDF views