Assessing Aggressiveness via Reaction Times Online


Aggressive tendencies can be assessed either commonly by explicit measures (self-report questionnaires), or by implicit measures that require the speeded classification of quickly presented stimuli and the recording and analysis of the reaction-times. We explored the psychometric properties of implicit measures assessing aggressiveness objectively: the Implicit Association Test (IAT) and its derivate, the Single-Target IAT. While the IAT focused on the automatic attitude towards aggressiveness, the ST-IAT focused on the self-concept. This feasibility study describes in methodological detail how a diversity of game players can be recruited to take these measures with common web-browser technology, even though reaction-time measurement in the range of a few hundred milliseconds is at stake. Self-reported and objective characteristics of users of violent, less violent, and no games differed. The results are partly in line with what can be expected on the basis of psychological theorizing, but structural-equation modelling shows that implicit measures on attitudes and self-concept differ in quality. Pitfalls and challenges for internet studies on computer players involving reaction-time measures are pointed out.

computer games, implicit association test, single-target IAT, aggressiveness, internet
Author biographies

Matthias Bluemke

Author photo Dr. Matthias Bluemke is an assistant professor of social psychology at the Psychological Institute, University of Heidelberg, D-69117 Heidelberg, Germany. After completing his undergraduate studies at the University of Trier, he obtained his diploma in psychology and his doctorate in psychology at the University of Heidelberg. As a member of the social psychology department, his interests lie in group dynamics and cognitive factors of social phenomena with a focus on methods and psychometrics of explicit and implicit measures. He contributed several book chapters and conference papers. He also has published articles in journals of interest to social psychologists and researchers on interindividual differences, among them Journal of Personality and Social Psychology, Journal of Experimental Social Psychology, European Review of Social Psychology, Memory and Cognition, Cognition and Emotion, Aggressive Behavior, and other journals. He served as a peer reviewer to most of these journals. In 2003 he received the Best German Diploma Thesis Award in Social Psychology (in collaboration with Malte Friese) by the Deutsche Gesellschaft für Psychologie (DGPs). Later he received the prestigious Hengstberger Award to hold a fruitful and stimulating conference on the future of implicit measures at the University of Heidelberg, and two Postdoctoral Fellowships by the German Research Foundation for further research on the validity of implicit measures at the University of Otago, Dunedin, New Zealand.

Joerg Zumbach

Author photo Prof. Dr. Joerg Zumbach is a full professor and, since 2007, Head of the Department of Science Education and Teacher Training, University of Salzburg, A-5020 Salzburg, Austria. He obtained his diploma in psychology and doctorate from the University of Heidelberg, Germany. His main interests are educational and instructional psychology, media-based learning, and the media-aggression link. He works at the intersection of theory development and applied research in relation to cognitive processing in computer-based learning environments and blended learning approaches. He is author and editor of several books on web based teaching and problem-based learning. His articles have been published, among others, in Computers in Human Behavior, Journal of Educational Multimedia and Hypermedia, Journal of Educational Computing Research, Aggressive Behavior, and Journal of Nonverbal Behavior.

Anderson, C. A. (2002). Aggression. In E. Borgatta (Ed.), The Encyclopedia of Sociology. New York: MacMillan.

Anderson, C. A., & Bushman, B. J. (2001). Effects of violent video games on aggressive behavior, aggressive cognition, aggressive affect, physiological arousal, and prosocial behavior: A meta-analytic review of the scientific literature. Psychological Science, 12, 353-359.

Anderson, C. A., & Bushman, B. J. (2002). Human aggression. Annual Review of Psychology, 53, 27-51.

Anderson, C. A., & Carnagey, N. L. (2009). Causal effects of violent sports video games on aggression: Is it competitiveness or violent content? Journal of Experimental Social Psychology, 45, 731-739.

Anderson, C. A., & Murphy, C. R. (2003). Violent video games and aggressive behavior in young women. Aggressive Behavior, 29, 423-429.

Anderson, C. A., Shibuya, A., Ihori, N., Swing, E. L., Bushman, B. J., Sakamoto, A., Rothstein, H. R., & Saleem, M. (2010). Violent video game effects on aggression, empathy, and prosocial behavior in Eastern and Western countries: A meta-analytic review. Psychological Bulletin, 136, 151-173.

Bandura, A. (1973). Aggression: A social learning analysis. Englewood Cliffs, N.J.: Prentice-Hall.

Banse, R., Clarbour, J., & Fischer, I. (2005). Implicit and explicit aggressiveness, social desirability, and the prediction of aggressive behaviour. Unpublished research.

Banse, R., & Fischer, I. (2002, July). Implicit and explicit aggressiveness and the prediction of aggressive behavior. Poster presented at the 11th European Conference on Personality, European Society for Personality Psychology, Jena, Germany.

Bartholow, B. D., & Anderson, C. A. (2002). Effects of violent video games on aggressive behavior: Potential sex differences. Journal of Experimental Social Psychology, 38, 283-290.

Bensley, L., & Van Eenwyk, J. (2001). Video games and real-life aggression: Review of the literature. Journal of Adolescent Health, 29, 244-257.

Björkqvist, K., Österman, K., & Lagerspetz, K. M. J. (1994). Sex differences in covert aggression among adults. Aggressive Behavior, 20, 27-33.<27::AID-AB2480200105>3.0.CO;2-Q

Bluemke, M., Friedrich, M., & Zumbach, J. (2010). The influence of violent and nonviolent computer games on implicit measures of aggressiveness. Aggressive Behavior, 36, 1-13.

Bluemke, M., & Friese, M. (2006). Do features of stimuli influence IAT effects? Journal of Experimental Social Psychology, 42, 163-176.

Bluemke, M., & Friese, M. (2008). Reliability and validity of the Single-Target IAT (ST-IAT): Assessing automatic affect towards multiple attitude objects. European Journal of Social Psychology, 38, 977-997.

Bluemke, M., & Zumbach, J. (2007). Implicit and explicit measures for analyzing the aggression of computer gamers. In G. Steffgen & M. Gollwitzer (Eds.), Emotions and aggressive behavior (pp. 38-57). Göttingen: Hogrefe & Huber Publishers.

Bösche, W. (2010). Violent video games prime both aggressive and positive cognitions. Journal of Media Psychology: Theories, Methods, and Applications, 22, 139-146.

Bosnjak, M. & Tuten, T. L. (2001). Classifying response behaviors in web-based surveys. Journal of Computer-Mediated Communication, 6(3).

Bushman, B. J., & Anderson, C. A. (2002). Violent video games and hostile expectations: A test of the general aggression model. Personality and Social Psychology Bulletin, 28, 1679-1686.

Bushman, B. J., Baumeister, R. F., Stack, A. D. (1999). Catharsis, aggression, and persuasive influence: Self-fulfilling or self-defeating prophecies? Journal of Personality and Social Psychology, 76, 367-376.

Bushman, B. J., Rothstein, H. R., & Anderson, C. A. (2010). Much ado about something: Violent video game effects and a school of red herring: Reply to Ferguson and Kilburn (2010). Psychological Bulletin, 136, 182-187.

Buss, A. H., & Perry, M. (1992). The aggression questionnaire. Journal of Personality and Social Psychology, 63, 452-459.

Carnagey, N. L., & Anderson, C. A. (2004). Violent video game exposure and aggression: A literature review. Minerva Psychiatrica, 45, 1-18.

Ceranoglu, T. A. (2010). Video games in psychotherapy. Review of General Psychology, 14, 141-146.

Christopherson, K. M. (2007). The positive and negative implications of anonymity in Internet social interactions: 'On the Internet, Nobody Knows You're a Dog.' Computers in Human Behavior, 23, 3038-3056.

Cohen, J. (1973). Eta-squared and partical eta-squared in fixed factor ANOVA designs. Educational and Psychological Measurement, 33, 107-112.

Colwell, J., & Kato, M. (2003). An investigation of the relationship between social isolation, aggression, and computer game play in Japanese adolescents. Asian Journal of Social Psychology, 6, 149-158.

Durkin, K., & Barber, D. E. (2002). Not so doomed: Computer game play and positive adolescent development. Applied Developmental Psychology, 23, 373-392.

Durkin, K. (2010). Video games and young people with developmental disorders. Review of General Psychology, 14, 122-140.

Eichstaedt, J. (2001). An inaccurate-timing filter for reaction-time measurement by JAVA-applets implementing internet-based experiments. Behavior Research Methods, Instruments, and Computers, 33, 179-186.

Fazio, R. H. (2001). On the automatic activation of associated evaluations: An overview. Cognition and Emotion, 15, 115-141.

Fazio, R. H., & Olson, M. A. (2003). Implicit measures in social cognition: Their meaning and use. Annual Review of Psychology, 54, 297-327.

Ferguson, C. J. (2010). Blazing angels or resident evil? Can violent video games be a force for good? Review of General Psychology, 14, 68-81.

Ferguson, C. J. (2007a). Evidence for publication bias in video game violence effects literature: A meta-analytic review. Aggression and Violent Behavior, 12, 470-482.

Ferguson, C. J. (2007b). The good, the bad and the ugly: A meta-analytic review of positive and negative effects of violent video games. Psychiatric Quarterly, 78, 309-316.

Ferguson, C. J., Cruz, A. M., Martinez, D., Rueda, S. M., Ferguson, D. E., & Negy, C. (2008). Personality, parental, and media influences on aggressive personality and violent crime in young adults. Journal of Aggression, Maltreatment and Trauma, 17, 395-414.

Ferguson, C. J., & Rueda, S. M. (2010). The Hitman study: Violent video game exposure effects on aggressive behavior, hostile feelings, and depression. European Psychologist, 15, 99-108.

Ferguson, C .J., Rueda, S., Cruz, A., Ferguson, D., Fritz, S., & Smith, S. (2008). Violent video games and aggression: Causal relationship or byproduct of family violence and intrinsic violence motivation? Criminal Justice and Behavior, 35, 311-332.

Ferguson, C. J., & Kilburn, J. (2009). The Public health risks of media violence: A meta-analytic review. Journal of Pediatrics 154, 759-763.

Ferguson, C. J., & Kilburn, J. (2010). Much ado about nothing: The misestimation and overinterpretation of violent video game effects in Eastern and Western nations: Comment on Anderson et al. (2010). Psychological Bulletin, 136, 174-178.

Ferguson, C. J., San Miguel, C., & Hartley, R. D. (2009). A multivariate analysis of youth violence and aggression: The influence of family, peers, depression and media violence. Journal of Pediatrics, 155, 904-908.

Feshbach, S. (1961). The stimulating versus cathartic effect of vicarious aggressive activity. Journal of Abnormal and Social Psychology, 63, 381-385.

Fiedler, K., & Bluemke, M. (2005). Faking the IAT: Aided and unaided response control on the Implicit Association Test. Basic and Applied Social Psychology, 27, 307-316.

Fiedler, K., Messner, C., & Bluemke, M. (2006). Unresolved problems with the "I", the "A" and the "T": Logical and psychometric critique of the Implicit Association Test (IAT). European Review of Social Psychology, 17, 74-147.

Frindte, W., & Obwexer, I. (2003). Ego-Shooter – Gewalthaltige Computerspiele und aggressive Neigungen. Zeitschrift für Medienpsychologie, 15, 140-148.

Furr, F. M. (2004). Interpreting effect sizes in contrast analysis. Understanding Statistics, 3, 1-25.

Gentile, D. A., & Anderson, C. A. (2003). Violent video games: The newest media violence hazard. In D. Gentile (Ed.), Media Violence and Children (pp. 131-152). Westport, CT: Praeger.

Gawronski, B., & Bodenhausen, G. V. (2006). Associative and propositional processes in evaluation: An integrative review of implicit and explicit attitude change. Psychological Bulletin, 132, 692-731.

Green, C. S., & Bavelier, D. (2006). Effects of video game playing on the spatial distribution of visual selective attention. Journal of Experimental Psychology: Human Perception and Performance, 32, 465-478.

Goldberg, L. R. (2001). International Personality Item Pool. A scientific collaboratory for the development of advanced measures of personality traits and other individual differences [WWW document]. Retrieved from:

Greenwald, A. G., & Farnham, S. D. (2000). Using the implicit association test to measure self-esteem and self-concept. Journal of Personality and Social Psychology, 79, 1022-1038.

Greenwald, A. G., McGhee, D. E., & Schwartz, J. K. L. (1998). Measuring individual differences in implicit cognition: The Implicit Association Test. Journal of Personality and Social Psychology, 74, 1464-1480.

Greenwald, A. G., Nosek, B. A., & Banaji, M. R. (2003). Understanding and using the Implicit Association Test: I. An improved scoring algorithm. Journal of Personality and Social Psychology, 85, 197-216.

Greenwald, A. G., Poehlman, T. A., Uhlmann, E. L., & Banaji, M. R. (2009). Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity. Journal of Personality and Social Psychology, 97, 17-41.

Griffiths, M. (1999). Violent video games and aggression: A review of the literature. Aggression and Violent Behavior, 4, 203-212.

Hartig, J., Jude, N., & Rauch, W. (2003). Entwicklung und Erprobung eines deutschen Big-Five-Fragebogens auf Basis des International Personality Item Pools (IPIP40). (Arbeiten aus dem Institut für Psychologie der Johann Wolfgang Goethe-Universität, Heft 1, 2003). Frankfurt: Institut für Psychologie an der Johann Wolfgang Goethe-Universität.

Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Brain and Behavioral Sciences, 33, 61-135.

Hofmann, W., Friese, M., & Strack, F. (2009). Impulse and self-control from a dual-systems perspective. Perspectives on Psychological Science, 4, 162-176.

Im, E.-O., & Chee, W. (2006). An online forum as a qualitative research method: practical issues. Nursing Research, 55, 267-273.

Jackson, L. A., Witt, E. A., Games, A. I., Fitzgerald, H. E., van Eye, A., & Zhao, Y. (2011). Information technology use and creativity: Findings from the Children and Technology Project. Computers in Human Behavior, 28, 370-376.

Jin, S.-A. A. (2011). “I feel present. Therefore, I experience flow:” A structural equation modeling approach to flow and presence in video games. Journal of Broadcasting and Electronic Media, 55, 114-136.

Karpinski, A. (2004). Measuring self-esteem using the Implicit Association Test: The role of the other. Personality and Social Psychology Bulletin, 30, 22-34.

Kato, P. M. (2010). Video games in health care: Closing the gap. Review of General Psychology,14, 113-121.

Keller, F., Gunasekharan, S., & Mayo, N. (2009). Timing accuracy of web experiments: A case study using the WebExp software package. Behavior Research Methods, 41, 1-12.

Kestenbaum, G. I., & Weinstein, L. (1985). Personality, psychopathology and developmental issues in male adolescent video game use. Journal of the American Academy of Child Psychiatry, 24, 329-337.

Krahé, B., & Möller, I. (2004). Playing violent electronic games, hostile attribution bias, and aggression-related norms in German adolescents. Journal of Adolescence, 27, 53-69.

Krantz, J. H., & Dalal, R. S. (2000). Validity of Web-based psychological research. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 35-60). San Diego, CA: Academic Press.

LeBel, E. P., & Paunonen, S. V. (2011). Sexy but often unreliable: Impact of unreliability on the replicability of experimental findings involving implicit measures. Personality and Social Psychology Bulletin, 37, 570-583.

Lee, K. M., & Peng, W. (2006). What do we know about social and psychological effects of computer games? In P. Vorderer, & J. Bryant (Eds.) Playing video games: Motives, responses, and consequences (pp. 325-345). Mahwah, NJ: Lawrence Erlbaum Associates.

Lemmens, J. S. & Bushman, B. J. (2006). The appeal of violent video games to lower educated aggressive adolescent boys from two countries. CyberPsychology and Behavior, 9, 638-641.

Levine, T. R., & Hullett, C. R. (2002). Eta squared, partial eta squared, and misreporting of effect size in communication research. Human Communication Research, 28, 612-625.

Lindsay, L. L., & Anderson, C. A. (2000). From antecedent conditions to violent actions: A general affective aggression model. Personality and Social Psychology Bulletin, 26, 533-547.

Nosek, B. A. (2007). Implicit-explicit relations. Current Directions in Psychological Science, 16, 65-69.

Nosek, B. A., & Smyth, F. L. (2007). A multitrait-multimethod validation of the Implicit Association Test: Implicit and explicit attitudes are related but distinct constructs. Experimental Psychology, 54, 14-29.

Olson, C. K., Kutner, L. A., Warner, D. E., Almerigi, J. B., Baer, L., Nicholi, A. M. II, & Beresin, E. V. (2007). Factors correlated with violent video game use by adolescent boys and girls. Journal of Adolescent Health, 41, 77-83.

Reips, U.-D. (2000). The Web Experiment Method: Advantages, disadvantages, and solutions. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 89-118). San Diego, CA: Academic Press.

Reips, U.-D. (2002). Internet-based psychological experimenting: Five dos and five don’ts. Social Science Computer Review, 20, 241-249.

Reips, U.-D. (2007). The methodology of Internet-based experiments. In A. Joinson, K. McKenna, T. Postmes, & U.-D. Reips (Eds.), The Oxford Handbook of Internet Psychology (pp. 373-390). Oxford University Press.

Rosnow, R. L., & Rosenthal, R. (2002). Contrasts and correlations in theory assessment. Journal of Pediatric Psychology, 27, 59-66.

Rothermund, K., & Wentura, D. (2004). Underlying processes in the Implicit Association Test: Dissociating salience from associations. Journal of Experimental Psychology: General, 133, 139-165.

Schmidt, W. C. (2000). The server-side of psychology Web experiments. In M. H. Birnbaum (Ed.), Psychological experiments on the Internet (pp. 285-310). San Diego, CA: Academic Press.

Sherry, J. (2007). Violent video games and aggression: Why can’t we find links? In R. Preiss, B. Gayle, N. Burrell, M. Allen, & J. Bryant (Eds.), Mass Media effects research: Advances through meta-analysis (pp. 231-248). Mahwah, NJ: L. Erlbaum.

Sparks, G. G., Sparks, C. W., & Sparks, E. A. (2009). Media Violence. In J. Bryant & M. B. Oliver (Eds.), Media Effects: Advances in Theory and Research (3rd ed., pp. 269-286). New York: Routledge.

Strack, F., & Deutsch, R. (2004). Reflective and impulsive determinants of social behavior. Personality and Social Psychology Review, 8, 220-247.

Teubel, T., Banse, R., Asendorpf, J. B., & Schnabel, K. (2011). Implicit but not explicit aggressiveness predicts performance outcome in basketball players. International Journal of Sport Psychology, 42, 390-400.

Uhlmann, E., & Swanson, J. (2004). Exposure to violent video games increases automatic aggressiveness. Journal of Adolescence, 27, 41-52.

Unsworth, G., Devilly, G., & Ward, T. (2007). The effect of playing violent videogames on adolescents: Should parents be quaking in their boots? Psychology, Crime and Law, 13, 383-394.

von Collani, G., & Werner, R. (2005). Self-related and motivational constructs as determinants of aggression: An analysis and validation of a German version of the Buss-Perry aggression questionnaire. Personality and Individual Differences, 38, 1631-1643.

Wells, G. L., & Windschitl, P. D. (1999). Stimulus sampling and social psychological experimentation. Personality and Social Psychology Bulletin, 25, 1115-1125.

Wiegman, O., & van Schie, E.G.M. (1998). Video game playing and its relations with aggressive and prosocial behavior. British Journal of Social Psychology, 37, 367-378.

Wigboldus, D. H. J., Holland, R. W., & van Knippenberg, A. (2004). Single target implicit associations. Unpublished manuscript.

Williams, D., & Skoric, M. (2005). Internet fantasy violence: A test of aggression in an online game. Communication Monographs, 72, 217-233.

Winkel, M., Novak, D. M., & Hopson, H. (1987). Personality factors, subject gender, and the effects of aggressive video games on aggression in adolescents. Journal of Research in Personality, 21, 211-223.

Ybarra, M., Diener-West, M., Markow, D., Leaf, P., Hamburger, M., & Boxer, P. (2008). Linkages between internet and other media violence with seriously violent behavior by youth. Pediatrics, 122, 929–937.





HTML views