A conceptual affective design framework for the use of emotions in computer game design

Special issue: Experience and Benefits of Game Playing

The purpose of this strategy of inquiry is to understand how emotions influence gameplay and to review contemporary techniques to design for them in the aim of devising a model that brings current disparate parts of the game design process together. Emotions sit at the heart of a game player’s level of engagement. They are evoked across many of the components that facilitate gameplay including the interface, the player’s avatar, non-player characters and narrative. Understanding the role of emotion in creating truly immersive and believable environments is critical for game designers. After discussing a taxonomy of emotion, this paper will present a systematic literature review of designing for emotion in computer games. Following this, a conceptual framework for affective design is offered as a guide for the future of computer game design.

affective computing; emotion; framework; computer games; game design
Author biography

Penny de Byl

Author photo Professor Penny de Byl teaches and researches in Games Development and Interactive Multimedia at Bond University, Australia. Prior to this she taught serious games theory in Breda, The Netherlands and computer science at the University of Southern Queensland. In 2007 she won a national award for her work in Virtual Worlds and in 2008 a national teaching excellence and research fellow award. She has published widely in peer reviewed journals and is the author of four books on game development including the acclaimed “Holistic Game Development”. Penny’s areas of expertise include e-learning, serious games, augmented reality and affective computing.

Baillie, P. (2002). The synthesis of emotions in artificial intelligences (Doctoral dissertation). University of Southern Queensland.

Beatson, O. (2015). Csikszentmihalyi's flow model [SVG]. Wikipedia. Retrieved from http://en.wikipedia.org/wiki/Mihaly_Csikszentmihalyi

Beskow, J., & Nordenberg, M. (2005, January). Data-driven synthesis of expressive visual speech using an MPEG-4 talking head. Paper presented at INTERSPEECH, Lisbon. (pp. 793-796). Retrieved from www.speech.kth.se/prod/publications/files/1281.pdf

Brown, C., Yannakakis, G., & Colton, S. (2012) Guest editorial: Special issues on computational aesthetics in games. IEEE Transactions on Computational Intelligence and AI in Games, 4, 149-151. https://doi.org/10.1109/TCIAIG.2012.2212900

Collins, K. (2009). An introduction to procedural music in video games. Contemporary Music Review, 28, 5-15. https://doi.org/10.1080/07494460802663983

Collins, K., Kapralos, B., & Kanev, K. (2014). Smart table computer interaction interfaces with integrated sound. Retrieved from http://www.researchgate.net/profile/Karen_Collins6/ publication/260750965_Smart_Table_Computer_Interaction_Interfaces_with_Integrated_Sound/links/0c960537b45602df94000000.pdf

Crawford, C. (1984). The art of computer game design. Retrieved from http://www-rohan.sdsu.edu/~stewart/cs583/ACGD_ArtComputerGameDesign_ChrisCrawford_1982.pdf

Csikszentmihalyi, M. (1997). Flow and the psychology of discovery and invention. Harper Perennial, New York.

Cunningham, S., Grout, V., & Picking, R. (2010). Emotion, content, and context in sound and music. In M. Grimshaw (Ed.), Game sound technology and player interaction: Concepts and developments, (pp. 235–263). https://doi.org/10.4018/978-1-61692-828-5.ch012

Damasio, A. R. (1999). How the brain creates the mind. Scientific American-American Edition, 281, 112-117. https://doi.org/10.1038/scientificamerican1299-112

de Peuter, A. (2014). Development of an adaptive game by runtime adjustment of game parameters to EEG measurements. Retrieved from http://www.few.vu.nl/~apr350/bachelor_project/Scriptie_Auke_de_Peuter_2512078.pdf

Denton, D. A. (2006). The primordial emotions: The dawning of consciousness. Oxford University Press.

Desurvire, H., & Wiberg, C. (2009). Game usability heuristics (PLAY) for evaluating and designing better games: The next iteration. In A. Ozok & Zaphiris P. (Eds.), Online communities and social computing (pp. 557-566). Berlin Heidelberg: Springer.

Dickey, M. D. (2006). Game design narrative for learning: Appropriating adventure game design narrative devices and techniques for the design of interactive learning environments. Educational Technology Research and Development, 54, 245-263. https://doi.org/10.1007/s11423-006-8806-y

Drossos, K., Floros, A., & Kanellopoulos, N. G. (2012). Affective acoustic ecology: Towards emotionally enhanced sound events. In A. Floros & I. Zannos (Eds.), Proceedings of the 7th Audio Mostly Conference: A Conference on Interaction with Sound (pp. 109-116), ACM, New York. Retrieved from http://dl.acm.org/citation.cfm?doid=2371456.2371474

Dupire, J., Gal, V., & Topol, A. (2009). Physiological player sensing: New interaction devices for video games. In S. Stevens & S. Saldamarco (Eds.), Entertainment computing - ICEC 2008 (pp. 203-208). Berlin Heidelberg: Springer. https://doi.org/10.1007/978-3-540-89222-9_25

Fairclough, S. (2008). BCI and physiological computing for computer games: Differences, similarities & intuitive control. Proceedings of CHI’08. Retrieved from http://shfairclough.com/Stephen_Fairclough_Research/Publications_physiological_computing_mental_effort_stephen_fairclough_files/chi08_final.pdf

Fanselow, M. S. (1994). Neural organization of the defensive behaviour system responsible for fear. Psychonomic Bulletin Review, 1, 429-438. https://doi.org/10.3758/BF03210947

Freeman, D. (2004). Creating emotion in games: The craft and art of emotioneering™. Computers in Entertainment (CIE), 2(3), article 15. https://doi.org/10.1145/1027154.1027179

Garner, T., Grimshaw, M., & Nabi, D. A. (2010). A preliminary experiment to assess the fear value of preselected sound parameters in a survival horror game. Proceedings of the 5th Audio Mostly Conference: A Conference on Interaction with Sound. (pp. 10). ACM. https://doi.org/10.1145/1859799.1859809

Garner, T., & Grimshaw, M. (2011, September). A climate of fear: Considerations for designing a virtual acoustic ecology of fear. In Proceedings of the 6th Audio Mostly Conference: A Conference on Interaction with Sound (pp. 31-38). ACM. https://doi.org/10.1145/2095667.2095672

Gebhard, P., Schröder, M., Charfuelan, M., Endres, C., Kipp, M., Pammi, S., . . . Türk, O. (2008). IDEAS4Games: Building expressive virtual characters for computer games. In H. Prendinger, J. Lester & M. Ishizuka (Eds.), Intelligent virtual agents (pp. 426-440). Berlin Heidelberg: Springer. https://doi.org/10.1007/978-3-540-85483-8_43

Gemrot, J., Kadlec, R., Bída, M., Burkert, O., Píbil, R., Havlíček, J., . . . Brom, C. (2009). Pogamut 3 can assist developers in building AI (not only) for their videogame agents. In F. Dignum, J. Bradshaw, B. Silverman, & W. van Doesburg (Eds.), Agents for games and simulations (pp. 1-15). Berlin Heidelberg: Springer. https://doi.org/10.1007/978-3-642-11198-3_1

Gilroy, S. W., Cavazza, M., & Benayoun, M. (2009). Using affective trajectories to describe states of flow in interactive art. Proceedings of the International Conference on Advances in Computer Entertainment Technology (pp. 165-172). https://doi.org/10.1145/1690388.1690416

Hefner, D., Klimmt, C., & Vorderer, P. (2007). Identification with the player character as determinant of video game enjoyment. In L. Ma, M. Rauterberg, & R. Nakatsu (Eds.), Entertainment computing–ICEC 2007 (pp. 39-48). Berlin Heidelberg: Springer. https://doi.org/10.1007/978-3-540-74873-1_6

Hoeberechts, M., & Shantz, J. (2009). Realtime emotional adaptation in automated composition. Proceedings of the 4th Audio Mostly Conference (pp. 1-8). Retrieved from http://mmm.csd.uwo.ca/People/gradstudents/jshantz4/pubs/hoeberechts_shantz_2009.pdf

Hubbard, J. A. (2001). Emotion expression processes in children's peer interaction: The role of peer rejection, aggression, and gender. Child Development, 72, 1426-1438. https://doi.org/10.1111/1467-8624.00357

Hudlicka, E. (2008). Affective computing for game design. In Proceedings of the 4th Intl. North American Conference on Intelligent Games and Simulation (GAMEON-NA) (pp. 5-12), Montreal, Canada.

Ip, B. (2011). Narrative structures in computer and video games. Part 1: Context, definitions, and initial findings. Games and Culture, 6, 103-134. https://doi.org/10.1177/1555412010364982

Isbister, K., & DiMauro, C. (2011). Waggling the form baton: Analyzing body-movement-based design patterns in Nintendo Wii games, toward innovation of new possibilities for social and emotional experience. In D. England (Ed.), Whole body interaction (pp. 63-73). London: Springer. https://doi.org/10.1007/978-0-85729-433-3_6

Jabareen, Y. R. (2009). Building a conceptual framework: Philosophy, definitions, and procedure. International Journal of Qualitative Methods, 8(4), 49-62. https://doi.org/10.1177/160940690900800406

Johnson, D., & Wiles, J. (2003). Effective affective user interface design in games. Ergonomics, 46, 1332-1345. https://doi.org/10.1080/00140130310001610865

Koestler, A. (1967). The ghost in the machine. Hutchinson: London.

Kovács, G., Ruttkay, Z., & Fazekas, A. (2007). Virtual chess player with emotions. In Proceedings of the 4th Hungarian Conference on Computer Graphics and Geometry. Retrieved from http://www.inf.unideb.hu/~fattila/CV/Downloads/Proc33.pdf

Kromand, D. (2007). Avatar categorization. In Proceedings from DiGRA 2007: Situated Play (pp. 400-406).

Larsen, R., & Diener, E. (1992). Promises and problems with the circumplex model of emotion. Emotion, 13, 25–59.

Laureano-Cruces, A. L., Acevedo-Moreno, D. A., Mora-Torres, M., & Ramírez-Rodríguez, J. (2012). A reactive behavior agent: Including emotions into a video game. Journal of Applied Research and Technology, 10, 651-672.

Lefton, L. A. (1994). Psychology (5th ed.), Allyn and Bacon: Boston.

Liljedahl, M. (2011). Sound for fantasy and freedom. In Grimshaw, M. (Ed.), Game sound technology and player interaction: Concepts and developments (pp. 22-44). IGI Global.

Livingstone, S. R., & Brown, A. R. (2005). Dynamic response: Real-time adaptation for music emotion. In Proceedings of the second Australasian conference on Interactive entertainment (pp. 105-111). Creativity & Cognition Studios Press.

Machine Elf 1735. (2011). Plutchik’s Wheel of Emotions [SVG]. Wikipedia. Retrieved from http://en.wikipedia.org/wiki/Contrasting_and_categorization_of_emotions#Plutchik.27s_wheel_of_emotions

MacLean, P. D. (1958). Contrasting functions of limbic and neocortical systems of the brain and their relevance to psychophysiological aspects of medicine. The American Journal of Medicine, 25, 611-626. https://doi.org/10.1016/0002-9343(58)90050-0

Mansilla, W. A. (2006). Interactive dramaturgy by generating acousmêtre in a virtual environment. In R. Harper, M. Rauterberg, & M. Combetto (Ed.), Entertainment computing-ICEC 2006 (pp. 90-95). Berlin Heidelberg: Springer. https://doi.org/10.1007/11872320_11

Mehrabian, A. (1995). Framework for a comprehensive description and measurement of emotional states. Genetic, social, and general psychology monographs, 121, 339-361.

Merckx, P. P. A. B., Truong, K. P., & Neerincx, M. A. (2007). Inducing and measuring emotion through a multiplayer first-person shooter computer game. In Proceedings of the Computer Games Workshop Amsterdam (pp. 7-6).

Miles, M. B., & Huberman, A. M. (1994) Qualitative data analysis: An expanded sourcebook. SAGE.

Mori, M. (1970). The uncanny valley. Energy, 7(4), 33–35.

Murray, J. (2005). Did it make you cry? Creating dramatic agency in immersive environments. In G. Subsol (Ed.), Virtual storytelling. Using virtual reality technologies for storytelling (pp. 83-94). Springer Berlin Heidelberg.

Nacke, L. E., Grimshaw, M. N., & Lindley, C. A. (2010). More than a feeling: Measurement of sonic user experience and psychophysiology in a first-person shooter game. Interacting with Computers, 22, 336-343. https://doi.org/10.1016/j.intcom.2010.04.005

Ochs, M., Sabouret, N., & Corruble, V. (2008). Modeling the dynamics of non-player characters' social relations in video games. In Proceedings of the Fourth Artificial Intelligence and Interactive Digital Entertainment Conference. Retrieved from https://www.researchgate.net/publication/220978562_Modeling_the_Dynamics_of_Non-Player_Characters%27_Social_Relations_in_Video_Games

Ortony, A., Clore, G. L., & Collins, A. (1990). The cognitive structure of emotions. Cambridge university press.

Ortony, A., & Turner, T. J. (1990). What's basic about basic emotions? Psychological Review, 97, 315-331. https://doi.org/10.1037/0033-295X.97.3.315

Peña, L., Peña, J. M., & Ossowski, S. (2011). Representing emotion and mood states for virtual agents. In F. Klugl & S. Ossowski (Eds.), Multiagent system technologies (pp. 181-188). Berlin Heidelberg: Springer. https://doi.org/10.1007/978-3-642-24603-6_19

Pert, C. B. (1997). Molecules of emotion: Why you feel the way you feel. Simon and Schuster.

Picard, R. W. (1997). Affective computing. MIT press.

Plutchik, R. (1980). Emotion: A psychoevolutionary synthesis. Harper Collins College Division.

Popescu, A., Broekens, J., & van Someren, M. (2014). Gamygdala: An emotion engine for games. Affective Computing, IEEE Transactions on, 5, 32-44.

Przybylski, A. K., Weinstein, N., Murayama, K., Lynch, M. F., & Ryan, R. M. (2011). The ideal self at play the appeal of video games that let you be all you can be. Psychological Science, 23, 69-76. https://doi.org/10.1177/0956797611418676

Robison, J., McQuiggan, S., & Lester, J. (2009). Evaluating the consequences of affective feedback in intelligent tutoring systems. In Affective Computing and Intelligent Interaction and Workshops, 2009. ACII 2009. 3rd International Conference on (pp. 1-6). IEEE.

Rusch, D. C., & König, N. (2007). Barthes revisited: Perspectives on emotion strategies in computer games. Computer Philology Yearbook.

Russell, J. A. (2003). Core affect and the psychological construction of emotion. Psychological Review, 110, 145-172. https://doi.org/10.1037/0033-295X.110.1.145

Sandercock, J., Padgham, L., & Zambetta, F. (2006). Creating adaptive and individual personalities in many characters without hand-crafting behaviors. In J. Gratch, M. Young, R. Aylett, D. Baillin, & P. Olivier (Eds.), Intelligent virtual agents (pp. 357-368). Berlin Heidelberg: Springer. https://doi.org/10.1007/11821830_29

Scherer, K. R., & Ekman, P. (Eds.). (2014). Approaches to emotion. Psychology Press.

Schönbrodt, F. D., & Asendorpf, J. B. (2011). The challenge of constructing psychologically believable agents. Journal of Media Psychology: Theories, Methods, and Applications, 23, 100-107. https://doi.org/10.1027/1864-1105/a000040

Shaver, P., Schwartz, J., Kirson, D., & O'connor, C. (1987). Emotion knowledge: Further exploration of a prototype approach. Journal of Personality and Social Psychology, 52, 1061-1086. https://doi.org/10.1037/0022-3514.52.6.1061

Shilling, R., Zyda, M., & Wardynski, E. C. (2002, November). Introducing emotion into military simulation and video game design America's army: Operations and VIRTE. In Proceedings of GAME-ON 2002.

Shinkle, E. (2008). Video games, emotion and the six senses. Media, Culture, and Society, 30, 907-915. https://doi.org/10.1177/0163443708096810

Silva, D. R., Siebra, C. A., Valadares, J. L., Almeida, A. L., Frery, A. C., da Rocha Falcão, J., & Ramalho, G. L. (2000). A synthetic actor model for long-term computer games. Virtual Reality, 5, 107-116. https://doi.org/10.1007/BF01424341

Skalski, P., & Whitbred, R. (2010). Image versus sound: A comparison of formal feature effects on presence and video game enjoyment. PsychNology Journal, 8, 67-84.

Smith, C. A., & Kirby, L. D. (2009). Putting appraisal in context: Toward a relational model of appraisal and emotion. Cognition and Emotion, 23, 1352-1372. https://doi.org/10.1080/02699930902860386

Strauss, A. L. (1987). Qualitative analysis for social scientists. Cambridge University Press.

Sweetser, P., & Wyeth, P. (2005). GameFlow: A model for evaluating player enjoyment in games. Computers in Entertainment (CIE), 3(3), article 3.

Takatalo, J., Häkkinen, J., Kaistinen, J., & Nyman, G. (2010). Presence, involvement, and flow in digital games. In R. Bernhaupt (Ed.), Evaluating user experience in games (pp. 23-46). London: Springer. https://doi.org/10.1007/978-1-84882-963-3_3

Tinwell, A. (2009). Uncanny as usability obstacle. In A. Ozok & P. Zaphiris (Ed.), Online communities and social computing (pp. 622-631). Berlin Heidelberg: Springer. https://doi.org/10.1007/978-3-642-02774-1_67

Tinwell, A., Grimshaw, M., & Abdel-Nabi, D. (2011). Effect of emotion and articulation of speech on the uncanny valley in virtual characters. In S. D’Mello, A. Graesser, B. Schuller, & J. Martin (Eds.), Affective computing and intelligent interaction (pp. 557-566). Berlin Heidelberg: Springer. https://doi.org/10.1007/978-3-642-24571-8_69

Tinwell, A., Grimshaw, M., Nabi, D. A., & Williams, A. (2011). Facial expression of emotion and perception of the uncanny valley in virtual characters. Computers in Human Behavior, 27, 741-749. https://doi.org/10.1016/j.chb.2010.10.018

Tomlinson, B., & Blumberg, B. (2003). Alphawolf: Social learning, emotion and development in autonomous virtual agents. In W. Truszkowski, M. Hinchey, & C. Rouff (Eds.), Innovative concepts for agent-based systems (pp. 35-45). Berlin Heidelberg: Springer. https://doi.org/10.1007/978-3-540-45173-0_2

Truong, K. P., van Leeuwen, D. A., & Neerincx, M. A. (2007). Unobtrusive multimodal emotion detection in adaptive interfaces: Speech and facial expressions. In D. Schmorrow & L. Reeves (Eds.), Foundations of augmented cognition (pp. 354-363). Berlin Heidelberg: Springer. https://doi.org/10.1007/978-3-540-73216-7_40

van Tol, R., & Huiberts, S. (2008). IEZA: A framework for game audio. Retrieved from http://www.gamasutra.com/view/feature/131915/ieza_a_framework_for_game_audio.php

Weiner, B. (1986). An attributional theory of motivation and emotion. Springer Science & Business Media.

Weiner, B. (1992). Human motivation: Metaphors, theories, and research. Sage.

Yun, C., Shastri, D., Pavlidis, I., & Deng, Z. (2009). O'game, can you feel my frustration? Improving user's gaming experience via stresscam. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 2195-2204). Retrieved from www.cpl.uh.edu/publication_files/C49.pdf

Zagalo, N., Torres, A., & Branco, V. (2006). Passive interactivity, an answer to interactive emotion. In R. Harper, M. Rauterber, & M. Combetto (Eds.), Entertainment computing-ICEC 2006 (pp. 43-52). Berlin Heidelberg: Springer. https://doi.org/10.1007/11872320_6

Zammitto, V., DiPaola, S., & Arya, A. (2008). A methodology for incorporating personality modeling in believable game characters. In Proceedings of Game Research and Development – Cybergames (pp. 24-31).


web of science logo




PDF views