Effects of online multitasking on reading comprehension of expository text

Introduction

Technology is ubiquitous in the lives of collegians. Most U.S. college students use the computer daily, reportedly spending 26.2 hr (undergraduate) and 33.7 hr (graduate) per week on it (Noack-Cooper, Sommerich, & Mirka, 2009). In 16 countries, significant majorities of college students reported that they were Internet users (WIP, 2012), and in the U.S., nearly 100% of college students access the Internet (Smith, Rainie, & Zickuhr, 2011). In one study, 76% of male and 58% of female U.S. college students reported spending two or more hours per day online, most reporting that they used the Internet primarily for social communication through email and instant messaging (Jones, Johnson-Yale, Millermaier, & Pérez, 2009). Among 18-24 year-olds in another study, 88% of non-students, 86% of undergraduates, 82% of graduates, and 78% of community college students reported communicating through social networking sites such as Facebook and Badoo (Smith et al., 2011).

Much of college students’ computer and Internet use occurs under multitasking conditions such as the simultaneous use of multiple applications on the computer (e.g., a word processing document and online social communication applications), multiple tabs while browsing the Internet, and multiple media (e.g., computer, music, and television) (Burak, 2012; Junco & Cotten, 2012; Levine, Waite, & Bowman, 2007). One concern about such multitasking is its potentially disruptive effects when students switch between communicating with others and their academic work such as reading assigned material, writing a research paper, or even while in the classroom. In this paper, we examine the effect of engaging concurrently in online social communication while reading expository text, a common activity for college students.

Defining Multitasking

Multitasking has become a common practice, yet, there is no agreed-upon definition of it (Wallis, 2010). Consequently, we begin by defining multitasking and then clarify the particular kind of multitasking focused on in our study. Subrahmanyam and Greenfield (2011) identified three types of multitasking: (1) Screen-based multitasking is the use of more than one window or application such as a word processor and an Internet browser open at a given time. A variant of screen-based multitasking is online multitasking, with multiple open windows or multiple open tabs within a window, such as when a web-based instant messenger, email, and Facebook are open at the same time. (2) Media multitasking is the simultaneous use of different media such as the phone, computer, and television at the same time. (3) Multitasking with a digital medium while having an offline interaction, such as texting during a family dinner or having a face-to-face meeting with a laptop open.

Studies on multitasking have also used the term dual task to describe the simultaneous engagement in a primary and secondary task, such as using a laptop while listening to a lecture (Hembrooke & Gay, 2003; Watson & Strayer, 2010). Task switching is a related term meaning engagement in various tasks sequentially, such as when an employee switches from answering emails to answering phone calls to finishing a report (Czerwinski, Horvitz, & Wilhite, 2004). Although multitasking implies that multiple tasks are being carried out simultaneously, users are in fact rapidly shifting or switching between different tasks or media (Kirschner & Karpinski, 2010). Nonetheless, because the different applications are operating at the same time, the rapid switch between them is often seamless, and so we use the terms simultaneous and concurrent to characterize such rapid shifting between different computer applications. This paper examines such screen-based multitasking (Subrahmanyam & Greenfield, 2011), where a reader switches between online communication via email, the Facebook wall, and an instant messenger, while reading expository text.

Multitasking Among College Students

Although anecdotal reports attest to the popularity of multitasking among college students, there is little scientific research on the frequency with which they do so while engaged in schoolwork. In a recent study conducted in the Northeastern U.S., 80% of students reported that they sometimes (about 50% of the time) multitasked schoolwork with Facebook, email, or instant messaging (Junco & Cotten, 2012). In another study, 33% of students reported that they engaged in academic work while instant messaging and 73% reported that they engaged in another activity while instant messaging (Levine et al., 2007). Communication multitasking in the classroom is fairly common as well, and participants report that they text message and visit Facebook while in class (Burak, 2012; Junco, 2012). Ophir, Nass, and Wagner (2009) reported that when college students used a medium, on average they used about four other media concurrently.

Cognitive Implications of Multitasking

The foregoing research suggests that college students often multitask with communication media while doing schoolwork both in and out of the classroom. An important question is whether such constant switching of attention can disrupt their learning. To address this issue, we turn to cognitive load theory (Sweller, 1994). A fundamental assumption of the theory is that working memory (the part of the information processing system that temporarily holds the information being processed) has a limited capacity that constrains a learner’s ability to process new information. Cognitive load is defined as any demand on working memory storage and processing of information (Sweller, 1994). Cognitive loads impact learning (Schnotz & Kürschner, 2007) and there is evidence that performance degrades when the cognitive load is either excessively high (overload) or excessively low (underload) (Teigen, 1994).

The three types of cognitive load are intrinsic, extraneous, and germane. Intrinsic cognitive load is considered inherent to the nature of the learning task and is related to the number of information elements and their interactivity (Paas, Renkl, & Sweller, 2004). Intrinsic cognitive load was originally thought to be the same for all learners, but research suggests it can be reduced by learners’ prior knowledge and by decreasing element interactivity (Schnotz & Kürschner, 2007). In contrast, extraneous and germane cognitive loads involve the manner in which information is presented and the learning activities required. Extraneous load refers to the information and activities that are not related to schema construction and automation, whereas germane load refers to information and activities that are related to schema development and automation (Sweller, 1994). If the sum of the three types of load exceeds a learner’s working memory capacity, learning will be disrupted (Paas et al., 2004).

Cognitive load theory has been used extensively to assess the effectiveness of instructional materials, particularly with multimedia instruction (Paas et al., 2004). The theory is also relevant to environments wherein learners multitask with online communication while engaged in a learning task, such as listening to a lecture or reading assigned text. Although the communication activity is not part of the learning task and not related to task-related schema construction, it could nonetheless increase extraneous load because it is an activity that the learner is engaging in simultaneously (Wood et al., 2012). As such, this digital multitasking activity might tax the learner’s processing capacity and disrupt learning. Thus, an important question is whether college students’ online multitasking while engaged in out-of-class reading activities impacts their learning and consequently their academic performance. We address this issue in the following section.

Relation between College Students’ Multitasking and Academic Performance

The small body of work on the relation between multitasking and academic performance has produced conflicting findings. Correlational studies using self-report data have suggested a clear negative association: Multitasking with Facebook and schoolwork was negatively predictive of overall GPA (Junco & Cotten, 2012), multitasking with instant messaging and schoolwork was associated with higher distractibility (Levine et al., 2007), and laptop usage during class was negatively correlated with attention, clarity, and comprehension of the lecture (Fried, 2008).

On the other hand, controlled studies have yielded mixed results. For example, Hembrooke and Gay (2003) found that there were some costs to using a laptop while listening to a lecture. Half the participants who attended a lecture were allowed to keep their laptops open, whereas the other half attended the same lecture at a later time and were asked to keep their laptops closed. Compared to students in the closed-laptop condition, students in the open-laptop condition performed worse on a quiz that was given immediately after the lecture. Because students in the open-laptop condition were given the choice to access the Internet through a proxy server, the researchers were able to use the proxy log data to analyze the content that students accessed during the lectures. Students who spent at least 50% of their online browsing time on course-related pages were considered on-taskers and those who spent less than 50% of their time on course-related pages were considered off-taskers. The researchers found an inverse relationship between quiz performance and time spent on course-related pages, and a positive relationship between quiz performance and time spent on unrelated pages – meaning that on-taskers actually performed worse than off-taskers.

Kraushaar and Novak (2010) studied college students’ multitasking activities using software that monitored computer activity as well as self-report methods. Students’ activities during lecture were quantified and their multitasking behaviors were coded as either productive (course-related) or distractive (not course-related). The activity monitor logs indicated that during lecture, students were active on their laptops 42% of the time. Students who engaged in more distractive multitasking earned lower grades on a variety of course measures compared to those who engaged in less distractive multitasking. In a laboratory experiment, Fox, Rosen and Crawford (2009) examined the effects of instant messaging while reading an SAT or GRE passage. They found no significant effects of screen-based multitasking on reading comprehension, recognition memory, or free-recall memory for both passages.

There are some indications that prior experience with multitasking can moderate the effects of multitasking on academic performance. For instance, although Fox et al. (2009) did not find an experimental effect of multitasking, prior use of instant messaging did predict poorer performance on a reading comprehension test. In a similar vein, Ophir et al.’s (2009) study of trait media multitasking (heavy vs. light multitaskers) and task-switching ability revealed that heavy media multitaskers performed worse on measures of task-switching, filtering, and working memory management. Although these results seem to suggest that frequent multitaskers may be less able to filter irrelevant stimuli, given that the data are correlational, other variables such as distractibility, impulsivity, or impulse control may be responsible for the association. Regardless, it is important to measure participants’ preference for and experience with multitasking when studying the impact of media multitasking on academic performance.

The Present Experiments

To summarize, college students are heavy users of the computer and the Internet, and they often use multiple media while doing academic work. Research has not systematically studied the effect of such online media/screen-based multitasking on academic performance. Ophir et al. (2009) only looked at the relation between multitasking frequency and cognitive skills such as attention and working memory and did not assess a task related to academic performance. Although Fox et al. (2009) examined the effect of instant messaging while reading a passage, they did not include other kinds of screen-based multitasking such as Facebook and email. Perhaps the use of only one application was not sufficient to increase overall cognitive load and disrupt performance. Given that students often use multiple media at any given time (Ophir et al., 2009), it is important to examine whether the simultaneous use of multiple online communication applications while reading may have an additive and disruptive effect on students’ learning.

In Experiment 1, we examined whether using multiple online communication applications while reading expository text would increase participants’ cognitive load and disrupt their learning. Expository text seeks to describe, inform, and explain and is commonly found in textbooks, research papers, and other academic sources. Learning from expository text is a familiar academic task for college students and involves reading, comprehension, and recall of the material presented. College student participants completed a primary learning task (learning from expository text about content covered in introductory psychology courses) and a secondary communication task (responding to social messages), either sequentially or concurrently. We conceptualized that the primary learning task contributed to intrinsic load and the secondary communication task contributed to extraneous load. Self-report measures were used to assess participants’ beliefs about technology and prior experience with multitasking.

Experiments 2 and 3 further extended our examination of the effects of simultaneously using multiple online communication applications while reading expository text. We focused on general reading comprehension processes that are used when reading a variety of academic text and that may be impacted by multitasking. In order to test whether any disruptive effect of multitasking is moderated by the level of cognitive load, Experiment 2 manipulated the intrinsic load by creating an easy and difficult version of the primary learning task. To better control our multitasking manipulation, we used the stimulus presentation software SuperLab. Because any disruptive effect of multitasking on reading might be stronger over time, Experiment 2 included a follow-up test to assess students’ retention of passage content one week later. To test the alternative hypothesis that multitasking might have cognitive benefits by helping to create optimal learning conditions (Schnotz & Kürschner, 2007) or desirable difficulties during learning (Bjork & Bjork, 2009), Experiment 3 used a moderately difficult primary learning task. Experiment 3 also used a modified version of our multitasking manipulation, with an algebraic filler task to ensure that the sequential and concurrent conditions were fully equivalent in terms of the time between when participants finished reading the passage and when they answered the comprehension questions.

Experiment 1

We used a pre-test post-test design to examine the effect of using multiple online communication applications (Facebook, Gmail, and AOL Instant Messenger) on the learning of content knowledge presented in expository text. We used text that described the rights of human subjects and the review process for research with humans. We selected this topic as it is covered in a college-level introductory psychology course, from which our participants were drawn. After completing a pre-test to assess their knowledge of this topic, participants read PowerPoint slides with text on this topic and then immediately took a post-test assessing their learning of the material; half the participants concurrently communicated with the researcher while reading the text, whereas the other half did so sequentially. We also assessed whether participants frequently engaged in multitasking during reading as well as their beliefs regarding multitasking and technology.

Drawing from cognitive load theory (Sweller, 1994), we conceptualized that the primary learning task contributed to intrinsic load and the secondary communication task contributed to extraneous load; thus we predicted that engaging in the two tasks simultaneously would increase extraneous cognitive load and disrupt performance and tested the following hypothesis:

H1: Because simultaneously engaging in a primary learning task and a secondary communication task should increase the extraneous cognitive load, participants who engaged in online communication while reading will show smaller gains between their pre-test and post-test scores on the content knowledge tests.

Based on findings that prior multitasking experience was negatively associated with performing two tasks simultaneously (Fox et al., 2009; Ophir et al., 2009), we expected that endorsement of multitasking and frequency of multitasking would be negatively associated with performance on the primary learning task. Hence, we also tested the following hypothesis:

H2: Participants who reported that they felt more positively about multitasking and were more frequent multitaskers will show smaller gains between their pre-test and post-test scores on the content knowledge tests.

Method

Participants. We recruited 35 college students (10 male, 25 female, M_age = 22.34) from the psychology subject pool of a large public university in the Western U.S. All participants received 1.5 hours of research credit for participating in the study. The grade and ethnic distributions of our sample can be found in Table 1.

Table 1. Participants’ Grade and Ethnic Distribution for Experiments 1, 2, and 3.

Experimental Tasks. The experimental tasks consisted of a primary learning task and content knowledge tests as well as a secondary task and social messages quiz.

Primary learning task and content knowledge tests (pre-test and post-test). The primary learning task entailed reading text about the rights of human subjects and the institutional review board process for all research with human subjects. The text was presented via PowerPoint slides for ease of presentation as well as for greater control (both primary and secondary task information could be timed to precise moments in the presentation). The PowerPoint slides were saved as a single-file webpage and presented using auto-advancement. The pre-test and post-test questions on the content presented were multiple-choice and each question was directly linked to the information presented on a particular slide. The questions were the same for the pre-test and post-test, but a different order was used to present them.

Secondary task and social messages quiz. The secondary task simulated the social messages one might encounter when using online communication applications. We used three tabs to load the following applications: Facebook (www.facebook.com), email (www.gmail.com), and instant messenger (www.meebo.com); for all three applications, we used accounts specifically created for the experiment. For example, on Facebook, we used two accounts named CSULA Media Lab and CSULA Media Lab 2 – one for the researcher and one for the participant.

For the social messages, we created a simple list of statements and questions to mimic a casual conversation when two people meet for the first time. Each social message consisted of a statement providing some information about the experimenter (e.g., my favorite color is red) and a question asking for the same information from the study participant (what is your favorite color?). The messages were delivered via the three browser tabs in turn. The social messages quiz contained a series of multiple-choice questions on the content of the social messages and was administered after the post-test (e.g., what was the researcher’s favorite color?).

Measures. We used the following measures.

Intensity of Technology Use scale. We adapted the Facebook Intensity Scale developed by Ellison, Steinfield, and Lampe (2007) to include additional media (Internet, cell phones, etc.). The scale consisted of four items and assessed aspects of participants’ use of digital technologies, including the intensity of use and how the participant felt without access to them (e.g., “I feel out of touch when I haven’t gone online or texted someone on my cell phone for a while”). Responses to each item were on a 5-point Likert scale ranging from strongly disagree to strongly agree. Cronbach’s alpha (4 items) was .77, suggesting adequate internal consistency. Participants’ responses to the four items of the Intensity of Technology Use Scale were averaged to yield a technology intensity score.

Multitasking Profile. We developed a Multitasking Profile consisting of five statements about the degree to which participants endorsed multitasking and viewed themselves to be multitaskers (e.g., I consider myself as someone who does more than one thing at a time). Responses to each item were on a 5-point Likert scale, ranging from strongly disagree to strongly agree. Cronbach’s alpha (5 items) was .76, suggesting adequate internal consistency. Participants’ responses to the five items of the Multitasking Profile were averaged to yield a Multitasking Profile score.

Multitasking Frequency scale. Although our study only manipulates screen-based multitasking, we were interested in the descriptive multitasking environments of students while they read. Consequently, we adapted the Multitasking Frequency scale created by Ophir et al. (2009) to assess the extent to which participants typically read and used the computer while multitasking with other activities. Our scale consisted of one primary item about each of five primary media activities, and nine sub-items for each of those primary media activities. The five primary items assessed how much time on average participants used the following on a daily basis: Digital media for work, digital media for pleasure, print media for work, print media for pleasure, and computers. Responses were on a 5-point Likert-type scale (never, < 30 min, 1 hr, 2-3 hr, ≥ 4 hr). The nine sub-items corresponding to each primary item asked how frequently that primary activity was accompanied by the following secondary activities: Using the computer, watching television, listening to music, texting on the phone, talking on the phone, playing games, reading and writing email, visiting social networking sites, and instant messaging. These subscale items used a 5-point Likert-type scale (never, rarely, about half the time, often, almost always). Cronbach’s alpha for our Multitasking Frequency scale (50 items) was .94, indicating high reliability.

Following Ophir et al. (2009), we used participants’ responses to the Multitasking Frequency scale to create a Multitasking Frequency Index (MFI) to roughly estimate the typical number of secondary activities participants engaged in while reading or using the computer. Before calculating the MFI, we converted responses to the primary items on the Multitasking Frequency scale (never, < 30 min, 1 hr, 2-3 hr, ≥ 4 hr) to 0, 0.5, 1, 2.5, and 5 respectively, thus roughly quantifying the number of hours the subject engaged in each primary activity per day. Similarly, we converted responses on the Multitasking Frequency subscales (never, rarely, about half the time, often, almost always) to 0, 0.25, 0.5, 0.75, and 1 respectively, thus roughly quantifying the percentage of time during each primary activity that was spent multitasking with a given secondary activity. We then computed the MFI for each subject using the following equation (adapted from Ophir et al., 2009):

where m_i is the sum of the nine sub-item scores for primary activity i and h_i is the primary item score for primary activity i.

Modified NASA-RTLX scale. We modified the NASA Raw Task Load Index (NASA RTLX), which provides a measure of subjective workload (Hart & Staveland, 1988). The original scale contained six items to assess different aspects of task load (mental difficulty, physical demand, mental effort, pacing, perceived performance, and stress) and used a 21 point Likert-type rating scale. We modified the wording on the RTLX scale to be appropriate to the demands of our experimental task. Our modified version contained six sub-scales assessing respondents’ self-perceptions about their mental difficulty, physical demand, mental effort, pacing, performance, and subjective stress while taking part in the experiment. Additionally, we used a 5-point Likert-type rating scale ranging from very low to very high. Cronbach’s alpha (6 items) was .72, suggesting adequate internal consistency.

Demographic questions. Participants provided their age, gender, year in school, ethnic group, etc.

Design and procedure. We utilized a pre-test/post-test between-subjects design with condition (sequential, concurrent) as the between-subjects factor. The primary and secondary tasks were administered on a desktop computer running Windows XP and Internet Explorer 7 loaded with four browser tabs: One single-file webpage with the content for the primary task, and three online communication applications used to receive and respond to social messages for the secondary task. Participants first completed the pre-test to assess their knowledge of the content presented in the primary learning task. Next the experimenter described and demonstrated how to respond to the two experimental tasks: Reading the text presented via the PowerPoint slides (primary task) and responding to social messages (secondary task). For the secondary task, the experimenter sat in another room and sent the social messages from one of the two specially-created accounts for each of the three applications. Participants were told to check all three tabs as new messages could be sent to any of the three applications. For Facebook the researcher’s messages were sent as wall posts, and they were instructed to refresh the page to see additional wall posts.

After an instruction check, which consisted of sending simple math problems for the participant to respond to, the experimenter opened the fourth tab, which contained the PowerPoint slides for the primary task. Participants were randomly assigned to the concurrent or the sequential condition. In the concurrent condition, participants read the text while reading and responding to the social messages, which were sent once for each of the 14 slides. Each slide (with accompanying social message) was presented for a total of 90 s. In the sequential condition, participants completed the two tasks sequentially. After reading all slides without interruption, they read and responded to the 14 social messages. Slides with text were presented for 75 s each and those with the social messages were presented for 15 s each. Participants were told to respond when they wished and were also told that they could ignore any message they did not want to respond to.

When they completed the two experimental tasks, participants completed a multiple-choice post-test on the content presented in the primary task and a multiple-choice quiz on the content of the social messages in the secondary task. Participants then completed the following questionnaires online at Surveymonkey (www.surveymonkey.com): the Intensity of Technology Use Scale, the Multitasking Profile, the Multitasking Frequency Scale, the modified NASA-RTLX, and the Demographics questionnaire.

Results

Technology use and multitasking beliefs. Participants strongly agreed that technology was a part of their daily lives (M = 4.5, SD = 0.79), agreed that technology was a part of their daily routine (M = 4.18, SD = 0.97), felt out of touch without online access or texting on their cell phone (M = 3.66, SD = 1.39), and agreed that they could not imagine their lives without technology (M = 3.86, SD = 1.22). They also agreed that they were multitaskers (M = 4.0, SD = 0.80), enjoyed multitasking (M = 3.71, SD = 0.89), and believed they produced the same quality of work while multitasking as well as when completing tasks separately at different times (M = 3.65, SD = 0.94). In contrast, participants were neutral about whether multitasking was efficient (M = 3.46, SD = 0.95), and whether they were good at multitasking (M = 3.51, SD = 0.85). There were no significant gender differences in the technology intensity score and the multitasking profile score.

Frequency of multitasking. The mean MFI was 4.35 (range = 1.08 to 7.27, SD = 1.57) indicating that while reading digital media, reading print media, or using the computer, participants reported that they also engaged in four other activities at the same time. There was no significant difference in the MFI as a function of gender or condition. When reading print media for pleasure, participants reported that they multitasked with six activities (including using the computer, watching television, listening to music, texting on the phone, emailing, and social networking) about half the time, and only rarely multitasked with the remaining three activities (talking on the phone, playing games, and instant messaging). Participants said they multitasked less when reading print media for work and reported only three other activities (using the computer, listening to music, and texting on the phone).

Effect of multitasking on a primary learning task. To assess the effect of a secondary online communication task on content knowledge learned from a primary task, we conducted an analysis of covariance (ANCOVA) on participants’ post-test scores with condition as a between-subject factors and pre-test score as a covariate. Because there was a ceiling effect on the scores of the social messages quiz (M = 13.23, SD = 1.37) with 80% of the participants scoring higher than 92%, it was not included as a covariate. The ANCOVA yielded no main effect of condition, [F(1, 32) = 1.84, p = .184] and a significant effect of pre-test as a covariate, [F(1, 32) = 5.26, p = .029]. Although participants in the sequential condition scored higher on the post-test (M = 9.94, SD = 2.19) compared to participants in the concurrent condition (M = 8.67, SD = 2.87), this effect was not significant.

To assess the effects of our manipulation on participants’ perceptions of cognitive load, we conducted a MANOVA with scores on the task load sub-scales (mental difficulty, physical demand, mental effort, pacing, perceived performance, and subjective stress) as the dependent variables and condition as the between-subjects factor. There were no significant multivariate or univariate effects (all ps > .05) as a function of condition.

Relation between, technology use, multitasking beliefs, and content learning on the primary learning task. We assessed learning by computing the learn difference score by subtracting the pre-test score from the post-test score. A correlational analysis of the score on the Intensity of Technology Use scale, the Multitasking Profile score, pre-test, post-test, and learn difference scores yielded no significant correlations. Interestingly, the technology intensity score was correlated significantly with the overall frequency of self-reported multitasking as measured by the MFI [r(33) = .42, p = .015].

Relation between prior multitasking experience and content learning on the primary learning task. Next, we conducted an exploratory correlational analysis on the MFI and content knowledge measures (pre-test, post-test, and learn difference). Although the MFI was not correlated with the pre-test and post-test scores individually, it was significantly correlated with the learn difference score [r(33) = -.36, p = .033]. Further analysis suggested that the correlation was significant for females [r(23) = -.52, p = .007], but not for males [r(8) = .14, p = .697].

Discussion

To recap, survey responses confirmed that as expected, the college students in our sample reported that technology was important to them and they both endorsed multitasking and reported that they frequently engaged in multitasking while reading print and digital media. In fact, our participants reported that on average they multitasked with four other activities while reading. Our first hypothesis, that simultaneously engaging in a secondary online communication task would impair content learning in a primary academic-learning task, was not supported; contrary to our prediction, participants who simultaneously engaged in a primary learning task and a secondary communication task did not perform worse than participants who completed the two tasks sequentially. Our second hypothesis that participants who endorsed multitasking and reported being more frequent multitaskers would perform worse on a learning task, was also only partially supported. We found no association between gains on the primary learning task and endorsement of multitasking. However, there was a negative association between learning gains and prior multitasking frequency, but only for female participants.

Our result that multitasking did not disrupt performance in the primary learning task is similar to that of Fox et al. (2009), who also found no effects of instant messaging on reading comprehension. This finding should be considered preliminary given the small sample size (n = 35). Moreover, our primary learning task used one expository text and our measure of learning assessed improvement in content knowledge about the topic covered in the text; the topic of human subjects research might have been familiar for some participants and too easy for others. We also tested content gains immediately after the learning task and it is possible that any disruptive effect of multitasking is more likely to be found with the passing of time.

Additionally, for our multitasking manipulation, participants were told they could respond whenever they wanted and could even ignore the messages if they wished to do so. Furthermore, we limited the messages to one per slide for the concurrent condition or sent them in regular, back-to-back succession for the sequential condition, whereas the frequency and pattern of real-world interactions are likely to vary with an individual’s engagement and with the conversation topic. Finally, the content of the social messages was somewhat superficial and most answers were short and the resulting conversation was limited. Thus, participants’ mental investment and motivation for the social communication may have been too minimal to interfere with the primary learning task; support for this comes from the lack of significant differences between the two conditions on perceptions of task load. In Experiment 2, we address these limitations with a larger sample, modified learning task and multitasking manipulation, and an immediate as well as one-week follow up to assess the effects of multitasking.

Experiment 2

Experiment 2 examined the immediate effect of online communication when reading expository text on reading comprehension and the recall of content a week later. The primary learning task was modified as follows: to ensure that the text was novel, challenging, and not tied to course knowledge, we used passages taken from the Critical Reading section of the Scholastic Assessment Test (SAT), a measure of students’ readiness for college. Although cognitive load was defined the same way as in Experiment 1, we manipulated it by creating an easy and difficult version of the primary learning task. We assessed reading comprehension immediately afterward and recall of passage content one week later. To control for individual differences in reading ability, comprehension, working memory, and other variables that might moderate the effect of multitasking, we used a within-subjects design such that every participant completed the easy and difficult learning task, in both the sequential and the concurrent condition. To standardize the multitasking manipulation and overall testing process, we utilized automated experimental software (Superlab 4.2) to administer the experiment. The same measures used in Experiment 1 were used to assess multitasking beliefs, frequency, and intensity of technology use. As a manipulation check of our multitasking manipulation, we assessed participants’ prior experience with the communication applications (e.g., Facebook, Gmail, etc) that were used and also assessed the degree to which participants felt that the simulation was similar to their actual experience with those particular social media applications. Similar to Experiment 1, we tested the following hypotheses:

H1: Participants would score lower in the reading blocks in which they read the text and engaged in online communication concurrently compared to the reading blocks in which they read the text and engaged in online communication sequentially, with greater effects for the difficult learning task.

H2: Participants who reported that they felt more positively about multitasking and were more frequent multitaskers would score lower overall on reading comprehension and content recall.

Method

Participants. A total of 90 college students (21 male, 69 female, M_age = 20) were recruited from the same subject pool as Experiment 1. Grade and ethnic distributions can be found in Table 1.

Experimental tasks. The experimental tasks consisted of a primary learning task, reading comprehension and content recall tests, as well as a secondary task.

Primary learning task, reading comprehension, and content recall tests. The expository text for the primary learning task consisted of two passages selected from the Critical Reading section of SAT released tests. They were selected using data provided by the test developers, which detailed the percentage of test-takers who correctly answered each reading comprehension question for a given passage. A passage containing three easy and four moderate questions was assigned to the easy block, whereas a passage containing three difficult and four moderate questions was assigned to the difficult block. Passages were then equated for approximate line count. For the reading comprehension questions, we used the multiple choice questions developed by the test developer. For the content recall test, the first and second authors generated multiple choice questions about the details and themes presented in the passage.

Secondary task. The social messages from Experiment 1 were utilized in addition to a set of newly developed messages that were created to generate deeper conversational discourse. The new messages were designed to stimulate a more realistic and engaging conversation by focusing on topics related to deep/controversial issues and to participants’ past memories. They were pilot tested for mental effort to respond, strength of emotional response, and emotional tone; social questions that received the highest ratings on the various dimensions were selected.

Measures. The following measures remained unchanged from Experiment 1: Intensity of Technology Use scale (α = .76), Multitasking Profile (α = .79), Multitasking Frequency scale, Modified NASA-RTLX scale, and the Demographics Questionnaire. In addition, we assessed participants’ prior experience with the social media applications (e.g., AOL Instant messenger, Facebook, etc.) used in the multitasking manipulation; for each application, a yes or no format was used to ask whether a participant had used the application prior to the experiment. We also assessed participants’ perceived expertise with computers, outlook, email programs, AOL instant messenger, any other instant messenger program, Facebook, and any other social networking site on a 5-pt Likert-type scale (rarely, beginner, average, advanced, expert). Finally, the validity of our multitasking simulation was assessed by asking participants to rate on a 5-pt Likert-type scale (very different to very similar) how similar our simulation was to their prior experience on AIM, Facebook, outlook, and social communication applications.

Design and procedure. We used a within-subjects design with difficulty (easy/difficult) and condition (sequential/concurrent) as within-subjects factors. All participants completed both the easy and difficult learning tasks in both the sequential and concurrent conditions. Participants completed a total of five blocks that consisted of one practice block, and four blocks created by crossing the two within-subject factors: easy-sequential, easy-concurrent, difficult-sequential, and difficult-concurrent. To control for any confounding effect of order, we utilized a balanced Latin square with eight conditions. In addition, we controlled for the effect of passage assignment on task by counterbalancing whether a passage was sequentially or concurrently presented with online communication. The experimenter loaded the Superlab 4.5 software and selected 1 out of eight ordering conditions; the next participant would proceed with the next order (e.g. 1^st→ A, 2^nd→ B, etc.). The experimenter proceeded to demonstrate the learning task and also showed the participant how to respond to the social messages. The participant was instructed to read the directions provided on the screen and to inform the experimenter should there be any problems or inquiries. Details of the experimental accounts and the procedure for sending and receiving the social messages were similar to that of Experiment 1 with one exception; the social messages took over the screen and there was no need for participants to periodically check the different applications for new messages.

Participants completed the reading comprehension blocks at their own pace and the time ranged from 0:45 to 1:30 hrs. The first block was always the practice block. The remaining four blocks (easy/sequential, etc) varied according to the ordering condition. For the two sequential task blocks, participants read the passages completely, and then clicked the advance button to proceed to the series of social messages (e.g. When and how did you meet your best friend?). For the two concurrent task blocks, participants received an interruption message every minute for a total of five messages; the messages varied in the medium (IM, email, Facebook) with which it was delivered. After each block, participants completed the reading comprehension questions on the passage they read, followed by a 30sec break. After completing all five blocks, participants completed the questionnaire packet (e.g. task load, multitasking frequency, etc.) online at Surveymonkey (www.surveymonkey.com). This concluded the first session of the experiment. For the follow-up session that assessed participants’ recall of passage content one week later, participants completed the content recall quiz on Qualtrics (www.qualtrics.com).

Results

Technology use and multitasking beliefs. Consistent with findings in Experiment 1, participants in Experiment 2 also reported that technology was important to them and that they both endorsed multitasking and frequently multitasked. Participants agreed that technology was a daily activity (M = 4.49, SD = 0.67), technology was a daily routine (M = 4.33, SD = 0.82), felt out of touch without access to the Web or texting on their cell phone (M = 3.56, SD = 1.31), and could not imagine their lives without technology (M = 3.68, SD = 1.32). They also agreed that they were multitaskers (M = 4.06, SD =0.80), enjoyed multitasking (M = 3.88, SD = 0.80), and believed they could multitask as well as do a single task at a time (M = 3.79, SD = 0.85). In contrast, participants were neutral about whether multitasking was the most efficient method of doing things (M = 2.96, SD =1.00) and about being good at multitasking (M = 3.42, SD = 0.83). There were no significant gender differences in the technology intensity score and the multitasking profile score.

Frequency of multitasking. The mean MFI score was 4.00 (range = 0.33 - 8.33, SD = 1.60) indicating that participants on average used four other media while reading. There was a significant difference in the MFI score as a function of gender [t(88) = -2.33, p = .022], with females reporting that on average they multitasked with more devices (M = 4.17) than males (M = 3.25).

The effects of online communication while reading on comprehension. To test for effects of condition order, a repeated measures ANOVA was conducted on the reading comprehension scores with difficulty and condition as within-subjects factors and condition order as the between-subjects factor. We found no consistent main effect for order or interactions of any factor with order (all ps > .05) and collapsed the data across orders for all subsequent analyses.

To assess the effect of online multitasking while reading on comprehension, we conducted a repeated measure ANOVA on comprehension scores with difficulty (easy and difficult) and condition (sequential and concurrent) as within-subjects factors. There were significant main effects of difficulty, [F(1, 87) = 142.25, p <.001, η² = .62], condition [F(1, 87) = 8.57, p = 005, η² = .09], and a significant interaction of difficulty and condition, [F(1, 87) = 12.44, p = .001, η² = .13]. To explore the interaction between difficulty and condition, paired-samples t-tests were conducted within each level of the two factors. Family-wise error rate was controlled using the Bonferroni threshold (p = .0125). On the easy task, participants scored lower in the sequential compared to the concurrent [M = 4.43(1.93) vs. 5.28(1.58)] condition [t (89) = -3.99, p < .001]. In contrast, for the difficult task, there was no significant difference [t (89) = 0.11, p = .912] between the sequential (M = 3.06, SD =1.85) and concurrent condition (M = 3.03, SD = 1.74). These results suggest a partially counter-intuitive beneficial effect of online communication while reading, but only for the easy task. Note that participants scored higher on the easy learning tasks compared to the difficult learning tasks in both the sequential and [t(89) = 5.84, p < .001] concurrent conditions [t(89) = 9.42, p < .001], confirming that the difficult task was indeed more difficult for them.

The effect of online communication while reading on recall of content (one week-later). A repeated measures ANOVA on the content recall scores with difficulty and condition as within-subjects factors and condition order as the between-subjects factor yielded no consistent main effect for order or interactions of any factor with order (all ps > .05). Thus the data were collapsed across order for further analysis. To assess whether longer-term recall may be affected by online social multitasking we conducted a repeated measures ANOVA on content recall scores with difficulty (easy and difficult) and condition (sequential and concurrent) as within-subjects factors. We found no significant main effect of difficulty [F(1, 87) = .46, p = .498], task [F(1, 87) = .00, p = .967], nor a significant interaction between the two factors, [F(1, 87) = 1.13, p = .292]. Next we conducted a second repeated measures ANOVA with overall prior experience with online communication applications as a covariate, since that factor was significantly correlated with five of our eight dependent measures. We found a significant effect of the interaction between difficulty and condition, F(1, 87) = 4.78 , p = .032, η² = .05. Paired-samples t-test for each level of difficulty and condition yielded no significant effects (all ps > .05).

Perception of experiment task load. Overall, participants reported that they felt the experiment had medium mental difficulty (M = 2.69, SD = 0.82), medium pace (M = 2.64, SD = 0.74), required medium mental effort (M = 3.35, SD = 0.82) and that they performed “okay” (M = 3.22, SD = 0.75). Participants reported they did not perceive the experiment to be physically demanding (M = 2.03, SD = 0.99) nor stressful (M = 2.39, SD = 0.97). A correlational analyses of task load with comprehension and content recalls cores yielded a significant finding only for pacing with comprehension scores in the easy concurrent [r(88) = -.31, p = .004], easy sequential [r(88) = -.26, p = .016], and difficult concurrent [r(88) = -.35, p = .001] blocks and content recall for the difficult concurrent [r(88) = -.23, p = .030] block.

Participants’ prior experience with communication applications and simulation validity check. Table 2 shows the mean ratings of participants’ expertise with the applications used in the multitasking simulation and the similarity between their prior experience with these applications and our simulation. Examination of the means suggests that participants felt they had above average expertise for most of the applications except for MS Outlook Email, and instant messaging applications. Similarly, they felt that the Facebook simulation and concurrent reading while socializing were similar to their typical experience. In contrast, participants felt neutral regarding the similarity of the AIM and MS Outlook to their typical experience with these applications.

Table 2. Participants’ Mean (SD) Ratings of their Expertise with the Online Applications Used in the Multitasking Simulation and the Similarity between their Prior Experience and our Simulation.

Relationship between technology use, multitasking beliefs, prior multitasking experience, prior software experience, and reading comprehension and content recall. A correlational analysis was conducted between participants’ technology use, multitasking profile score, prior multitasking and software experience as well as reading comprehension and content recall scores. For brevity, only the significant results are reported. The MFI was significantly and negatively correlated with reading comprehension performance in the easy-concurrent [r(88) = -.37, p <.001] block. Similarly, the MFI was significantly and negatively correlated with content recall in the difficult-concurrent [r(88) = -.36, p <.005] block.

Next, participants’ level of experience with MS outlook, AIM, other instant messenger, Facebook, and other social networking sites were collapsed to compute an overall measure of prior experience with online communication applications. A correlational analysis of the overall online communication experience with the various reading comprehension and content recall scores revealed that prior experience was significantly correlated with reading comprehension in the easy-sequential [r(88) = -.37, p < .001)] easy-concurrent [r(88) = -.35, p = .001], and difficult-sequential [r(88) = -.29, p = .006] blocks; prior experience was also significantly correlated with the content recall score in the easy-sequential [r(88) = -.32, p = .003] and difficult-concurrent [r(88) = -.37, p <.001] blocks.

Discussion

As expected, the college students in our sample reported that on average they multitasked with four other activities while reading. Similar to the results of Experiment 1, the results of Experiment 2 provided partial support for our hypotheses. Our first hypothesis, that increasing cognitive load by manipulating learning task difficulty and multitasking would disrupt reading comprehension and content recall was not supported. Contrary to our prediction, we found a counter-intuitive benefit for multitasking such that participants who simultaneously engaged in online communication while engaged in the easy learning task scored higher on the test of reading comprehension. Our second hypothesis that participants who held more positive beliefs about multitasking and were more frequent multitaskers would demonstrate lower reading comprehension and content recall scores was only partially supported. Although we did not find a consistently negative association between the multitasking belief, frequency variables, reading comprehension, and content recall, the significant associations that we did find were in the predicted direction.

One possible reason to account for the counter-intuitive beneficial finding of the concurrent condition for the easy task draws from the notion of optimal learning conditions from cognitive load theory (Schnotz & Kürschner, 2007). On this view, a match between learner expertise and task difficulty produces the best performance. It is possible that the easy-sequential block was not sufficiently challenging and engaging. Thus, it might not have produced an optimal learning experience (Schnotz & Kürschner, 2007), leading to lower performance relative to the easy-concurrent block. Similarly, Speier, Vessey, and Valacich (2003) found that interruptions facilitated faster and more accurate performance on simpler tasks by focusing the decision maker’s attention on important cues. In addition, Adler and Benbunan-Fich (2012) also found that performance efficiency showed an inverted-U pattern with multitasking. This inverted-U pattern could explain why the beneficial effect of multitasking was not observed in the difficult condition (Adler & Benbunan-Fich, 2012). At the same time, the overall difficulty of the difficult task may have washed out the variance needed to determine if a beneficial effect of multitasking could have occurred in the difficult conditions. Cognitive load theory accounts for this loss of performance discrimination in the difficult conditions as a result of over taxing the individual; any additional extraneous load caused by multitasking would be unnecessary and past the participant’s working memory threshold. To test this possibility, Experiment 3 used a moderately difficult task using passages that had a moderate number of difficult questions.

Experiment 3 also added a filler task for participants in the concurrent condition to make it more equivalent to the simultaneous condition. Recall that participants in the sequential condition read the passage, responded to the social messages, and then completed the comprehension task; participants in the concurrent condition however, completed the comprehension test immediately after simultaneously reading the passage and responding to the social messages. Perhaps the short delay contributed to the lower score that we found for participants who had read the easy passage in the sequential condition. To control for any possible effects of such a delay on learning performance, Experiment 3 used a filler task that participants in the concurrent condition completed after they finished the reading/online communication task and before they took the comprehension task.

To recap, Experiment 3 used a similar design as Experiment 2. Participants completed a moderately difficult primary learning task and a secondary task either sequentially or concurrently. Subsequently, they completed the reading comprehension test that assessed their comprehension of the material in the primary learning task. Participants in the concurrent condition also completed a filler task consisting of simple math problems prior to completing the comprehension task. In contrast to Experiment 2, task load was assessed twice to better compare task demands between the concurrent and sequential blocks. The following two hypotheses were tested:

H1: Participants will score higher in the concurrent reading block compared to the sequential reading block due to a beneficial effect of multitasking.

H2: Participants who reported that they felt more positively about multitasking and were more frequent multitaskers would score lower overall on the reading comprehension task.

Experiment 3

Method

Participants. A total of 40 college students (20 male, 20 female, M_age = 20, age range: 18-25) were recruited from the same pool. Grade and ethnic distributions can be found in Table 1.

Experimental tasks. The experimental tasks consisted of a primary learning task, reading comprehension test, secondary task and filler task.

Primary learning task and reading comprehension test. Reading passages were selected using a procedure similar to that described in Experiment 2. The following criteria were used to select the passage for the moderately difficult block: six moderately difficult questions and one easy question. The questions were developed by the test developer and consisted of multiple choice questions.

Secondary task. The secondary task remained unchanged from Experiment 2.

Filler task. The filler task consisted of simple algebraic problems such as: x + 22 = -9, what is x?

Measures. The following scales remained unchanged from Experiment 2: Intensity of Technology Use Scale (α =.81), Multitasking Profile (α = .89), Multitasking Frequency scale, Modified NASA-RTLX scale, Demographics Questionnaire and the Prior Software Experience and Simulation Validity Scale.

Design and procedure. As before, a within-subjects design was used with condition (sequential, concurrent) as the within-subjects factor. Task order was counterbalanced so that half the participants completed the sequential condition first and half completed the concurrent condition first. The procedure was similar to Experiment 2 with three exceptions: (1) participants only completed 3 blocks (1 practice, 2 moderately difficult) (2) participants in the concurrent task block completed the problems in the filler task for two minutes after reading the passage (3) participants did not return for a follow-up test.

Results

Technology use and multitasking beliefs. As before, participants reported that technology was important to them and they both endorsed multitasking and frequently multitasked. Participants agreed that technology was a daily activity (M = 4.45, SD = 0.85), agreed that technology was a daily routine (M = 4.26, SD = 0.99), felt out of touch without access to the Web or texting on their cell phone (M = 3.45, SD = 1.15), and agreed that they could not imagine their lives without technology (M = 3.75, SD = 1.17). They also reported that they were multitaskers (M = 3.88, SD = 0.65), enjoyed multitasking (M = 3.55, SD = 0.78); they were neutral about whether they could multitask as well as do a single task at a time (M = 3.38, SD = 0.95), whether multitasking was the most efficient method of doing things (M = 2.83, SD = 1.13) and whether they were good at multitasking (M = 3.15, SD = 1.00). There were no significant gender differences in the technology intensity score and the multitasking profile score.

Frequency of multitasking. In contrast to Experiments 1 and 2, the mean MFI was a bit lower at 3.28 (range 1.07 to 5.83, SD = .94). There was no significant difference in the MFI as a function of gender, F (1, 39) = 2.82, ns.

The effect of online communication while reading on comprehension. To test for effects of condition order, a repeated measures ANOVA was conducted on the reading comprehension scores with order as the between-subjects factor. A significant effect of order was obtained [F(1, 38) = 6.05, p = .019]. Given the lack of significant order effects in the prior analyses in Experiment 2, it is likely that the significant effect was caused by chance. Nonetheless, as a precaution we conducted and report the analyses separately for the two orders.

To assess the effects of online multitasking while reading on comprehension, paired sample t-tests were conducted on the comprehension scores separately for the two orders. For participants who completed the sequential block first, there was no significant difference between the scores on the sequential (M = 3.20, SD = 1.47) and concurrent (M = 3.45, SD = 1.43) blocks, t(19) -0.653, p = .522); we similarly found no significant difference between the scores on the sequential (M = 2.00, SD = 1.72) and concurrent (M = 2.65, SD = 1.50) blocks for participants who completed the concurrent block first, t(19) = -1.78, p = .091. Across both orders, the mean scores revealed a similar non-significant trend in the predicted direction with participants scoring higher in the concurrent condition compared to the sequential condition.

Perception of experiment task load. Having participants complete the TLX after each block allowed us to compare differences in their perceptions of task load about the sequential and concurrent conditions. A set of paired-samples t-tests comparing the task loads between the concurrent and the sequential condition yielded a significant difference only for mental effort. Participants felt that the sequential task required less mental effort than the concurrent task [2.95 vs. 3.30, t(39) = -2.270, p = .029]. Participants’ perception about the pacing of the task was negatively correlated with performance in the concurrent condition, although it was only marginally significant [r(38) = -.31, p = .052]; perception about pacing was not correlated with performance in the sequential condition [r(38) = -.12, p = .454].

Participants’ prior experience with communication applications and simulation validity check. Table 2 shows the mean ratings of participants’ expertise with the applications used in the multitasking simulation and the similarity between their prior experience with these applications and our simulation. Examination of the means suggests that participants had average or above average expertise with the various online applications used in the Experiment. Their responses also validated our simulation and were similar to the result of Experiment 2. Except for prior experience with AOL Instant Messenger and moderate-concurrent [r(38) = -.40, p = .011] and experience with other instant messengers and moderate-concurrent [r(38) = -.38, p = .016], no other individual factor (e.g., prior multitasking experience, etc) was correlated with reading comprehension performance.

Discussion

Our finding of a non-significant trend in the predicted direction is consistent with our hypothesis that performance in the concurrent block would be higher than the sequential block for the moderately difficult learning task. It raises the possibility that multitasking might not have any disruptive effects at low to moderate levels of cognitive load. We had specifically included a filler task to ensure that the concurrent block was fully equivalent to the sequential block in terms of the time gap between finishing reading the passage and answering the questions. In spite of this precaution, we still found a trend toward higher scores in the concurrent block suggesting that under certain conditions multitasking might not be disruptive to reading and learning. In retrospect, adding a sequential filler task may have increased the memory load for the concurrent block and thus decreased performance, leading to no significant effect. We also only found little support for our hypothesis that beliefs about and prior experience with multitasking would be negatively associated with reading comprehension performance.

General Discussion

As computers, the Internet, and other digital tools have become central in young people’s lives, so has multitasking with these new media forms. We undertook this study to document the extent of college students’ multitasking while engaging in reading tasks, to examine the effect of screen-based multitasking on their academic performance, and to investigate the relation between prior multitasking, beliefs, experience and their reading comprehension and learning while multitasking.

Our descriptive results are consistent with survey reports and show that the college students in our study frequently engage in multitasking while reading. Across the three experiments, our participants reported that on averaged they multitasked with 3-4 other activities while reading. Our hypothesis that simultaneously engaging in a secondary online communication task would impair performance in a primary academic-learning-type task was not supported. In Experiment 1, we found no significant disruptive effect of multitasking. In Experiment 2, we again found no disruptive effect of multitasking; on the contrary, we found a counter-intuitive beneficial effect of multitasking for the easy learning task. Experiment 3 also found no disruptive effect of multitasking, and a trend suggesting a beneficial effect of multitasking for a moderately difficult learning task. We also found mixed support for our hypothesis, that participants who endorsed multitasking and reported being more frequent multitaskers, would perform worse on our learning tasks. Across the three experiments we found varying degrees of association between multitasking beliefs and experience and content learning, reading comprehension, and recall.

College Students, Technology, and Multitasking

Our results confirmed that technology played an important role in the lives of our college student participants. Participants’ beliefs regarding technology and multitasking were relatively consistent across the three studies. Participants felt that technology was a daily and routine activity, felt out of touch without it, and could not imagine life without technology. They also considered themselves to be multitaskers and reported that they enjoyed multitasking, and could multitask as well as they could single task. Interestingly, they felt neutral as to whether multitasking was more efficient and whether they were good at multitasking.

College students’ beliefs about multitasking were also consistent with their self-reported multitasking practices. Across three experiments, our participants’ Multitasking Frequency Index confirmed that they were were avid multitaskers. Their Multitasking Frequency Index ranged from 4.35 (Experiment 1) to 3.28 (Experiment 3), suggesting that on average they were using between three and four other media forms while reading. Our results are consistent with those of Ophir et al. (2009), who also found that college students multitasked with four other media when using a given media form. It should be kept in mind that this is only an average and the number of media devices that participants reported simultaneously using actually ranged from 1 to 8. Altogether, our results provide further evidence that media multitasking is here to stay especially as tablets, smart phones, iPods, and other portable devices become smaller and more affordable.

There was also a distinct set of activities that participants preferred to multitask with while reading. For instance in Experiment 1, the five other activities they were most likely to engage in were using the computer, listening to music, texting on the phone, using social networking sites, and watching television. The three activities least likely to be multitasked with were instant messaging, emailing, and playing games. Our finding that text messaging was one of the more commonly multitasked activities is consistent with national trends in the U.S (Rideout, Foehr, & Roberts, 2010) and highlights the need for more study of youth cell phone use especially in learning contexts. The infrequency of multitasking with instant messaging and emailing is likely related to the influx of smart phones and young people’s preference for communicating via texting in place of instant messaging and email. According to the Nielsen report, 83% of 13-17 year-olds in the U.S. used text messaging, whereas only 40% used instant messaging on their mobile phones (Nielsen Company, 2009). Furthermore, 8-18 year-olds in the U.S. spent 1.35 hours per day texting and only 11 minutes instant messaging (Rideout et al., 2010). Although, laptop use studies suggest that students under report their instant message usage (Kraushaar & Novak, 2010), it is likely that younger students are not using instant messaging as much as older populations.

Also worth noting was our finding that there were no gender differences in beliefs about multitasking and the importance of technology. With regard to the number of devices used simultaneously, we found no gender difference in Experiments 1 and 3; in Experiment 2, we did find that female participants reported multitasking with one more device than males. The lack of consistent gender differences with regard to multitasking and technology in general is in contrast to the early years of computers and games, when males consistently used computers and played electronic games more frequently and more intensely than females (Subrahmanyam & Greenfield, 2011). The narrowing of the gender gap has been evident since the emergence of online communication tools (Subrahmanyam & Smahel, 2011) and our findings provide further support that males and females are now equally comfortable with technology and multitasking both in terms of beliefs and use. Another trend worth noting is the decrease in the MFI (4.35 to 4.00 to 3.27) from Experiment 1 to 3. This could either be an artifact of the smaller sample size in Experiments 1 and 3 compared to Experiment 2 or it could reflect a real decrease as smart devices (e.g., iPhones, Android phones) with increased capabilities are becoming more affordable and thus more widely owned.

Impact of Online Communication While Reading Expository Text

Drawing on cognitive load theory (Sweller, 1994), which posits that resources for cognitive processing are limited and learning may be disrupted when they are overloaded, we hypothesized that asking subjects to simultaneously communicate and read expository text would disrupt their comprehension and recall of the text content. We specifically varied the difficulty of the primary learning task in order to assess whether the intrinsic load of the learning task might moderate the effect of multitasking on learning. Across three experiments, this hypothesis was not supported and we found no disruptive effect of communicating online while reading. On the contrary, Experiment 2 found a counterintuitive beneficial effect of multitasking for the easy learning task. Our lack of a significant disruptive effect for multitasking is similar to that of Fox et al. (2009), who also found no effects of instant messaging on reading comprehension. They had speculated that conversing with one person on AOL instant messenger might not have been demanding enough to disrupt their participants’ learning. To ensure that the extraneous load in the concurrent condition would be high enough to disrupt learning, we made our online communication manipulation more complex by asking subjects to interact via three different applications. For Experiments 2 and 3, we used social messages that would encourage deeper conversation and thus increase the cognitive resources that participants would have to devote to the secondary task. Additionally in Experiments 2 and 3, we chose text that was likely to be challenging to ensure adequate intrinsic load for our learning task. Despite these extra steps, it appears that the learning situation did not overload our participants’ processing capacities. There are several possible reasons why this might have been so and we consider them in turn.

Firstly, as members of the millennial generation, our participants grew up with technology and were comfortable with multitasking. Their response on the Multitasking Frequency Scale revealed that on average they multitasked with four other media forms when reading. For these practiced multitaskers, reading while communicating on three other applications might simply not have been very taxing. Secondly, for our college student participants, the reading comprehension task was a familiar one and they have perhaps developed strategies to quickly read text and extract responses while doing other things. Corroborating this was our finding of no significant differences between the sequential and concurrent condition on the social messages quiz scores (Experiment 1). Additionally responses to multiple measures of task load perceptions revealed that participants in Experiments 1 and 2 found the experimental tasks to be of average demand; because the task load index was only administered once, we were not able to compare perceptions between the conditions. To address this gap, Experiment 3 administered the task load measures immediately after the sequential and concurrent condition. Interestingly, participants rated the mental effort of the concurrent condition to be significantly greater than the sequential condition. There was still no disruptive effect of concurrent tasking; instead, participants in the concurrent condition scored higher than those in the sequential task, but not significantly so. It is impossible to determine whether our lack of a disruptive effect was because of participants’ prior experience with multitasking, their positive perceptions of their multitasking abilities, or because our concurrent task was not taxing enough to them. Either way, our multitasking manipulation did not increase the extraneous cognitive load enough to disrupt our participants’ reading of expository text.

Not only did our multitasking manipulation not disrupt learning, it lead to significantly better performance for the easy passage (Experiment 2) and a trend toward better performance for the moderately difficult passage (Experiment 3). Recall that in Experiment 3, participants found the concurrent condition to be more demanding than the sequential condition. Thus, an alternate possibility is that when multitasking makes the learning task sufficiently challenging, it might lead to better performance. This is consistent with the proposal that the lowest level of cognitive processing load does not necessarily lead to optimal performance (Schnotz & Kürschner, 2007). Along these lines, Bjork and Bjork (2009) have proposed that desirable level of difficulties actually maximize learning partly because they help with encoding and retrieval activities that help with learning. It is possible that the interruptions and task switching of the concurrent condition made the easy passage a bit more challenging. With each interruption and switch, participants might have had to re-read portions of the passage to find the part of the text they were reading at the time of interruption. Repeatedly doing this might have led them to better encode the text leading to better performance on the comprehension task.

Another reason multitasking might have been beneficial for some of the participants is the concept of optimal flow in web users’ experience. Studies show that the Internet facilitates flow (Agarwal & Karahanna, 2000; Chen, Wigand, & Nilan, 2000), which is an optimal, extremely enjoyable experience (Csikszentmihalyi, 1990). Engaging in online communication while reading might have indirectly facilitated a state of flow by activating previous online multitasking experience, wherein the participants achieved a flow state. Recall that in both Experiments 2 and 3, a majority of participants reported that they had prior experience with the applications (Outlook, Facebook, and AIM) used in our study and considered themselves to be well-versed with regards to computers, Facebook, and other email clients. For these habitually multitasking participants, the uninterrupted reading experience in the sequential condition might have caused discomfort, thus impeding flow, and lowering performance. However, because we did not assess flow, we cannot determine whether it was indeed occurring for participants in the concurrent condition, who read the easy and moderately difficult passage. Future studies on multitasking must assess the occurrence of flow by including a measure of enjoyment/entertainment to determine the extent to which participants enjoyed the experimental tasks.

Regardless of the particular reason why multitasking did not disrupt performance, our results suggest that college students such as our participants have become adept at multitasking with online communication while reading. Having grown up enmeshed in technology with multitasking as the norm rather than the exception, they seem to have adapted to rapidly switching between familiar, every day activities such as the kinds we used in our experiments. For them multitasking might not automatically increase cognitive load and thus disrupt performance. This does beg the question of whether and when multitasking might get overwhelming for learners. Thus future work must examine the degree and kind of multitasking that may be manageable. For instance, at lower levels of cognitive load such as with our easy and moderately difficult passage, multitasking might induce flow enhancing performance. At more moderate levels of cognitive load such as our difficult text, it might have no effect (positive or negative) on performance; it may be that only at significantly higher levels of cognitive load such as when engaged in unfamiliar or complex learning tasks, that it disrupts performance. The nature of the communication task might also play a role in the effect of online communication on reading. Although we modified the social communication task in Experiments 2 and 3 so the conversation was intellectually more demanding, it was neutral in emotional tone. Anecdotal and survey research suggests that online communication can be emotional in tone and contain misunderstanding and conflict (Subrahmanyam, Garcia, Harsono, Li, & Lipana, 2009). Negative or conflict-tinged interactions might disrupt flow and consequently lower performance. Finally, developmental factors might moderate the effect of multitasking. For younger learners or learners who grew up with technology, multitasking might be beneficial at lower loads and disruptive only at very high levels of task difficulty. In contrast, for older learners or learners who did not multitask when young, multitasking might not be beneficial or may be disruptive at much lower levels of task difficulty. Future research should examine variables that might influence the effect of multitasking on learning, such as participant age, the number of online communication applications, the types of social interaction (positive, neutral, or negative), and learning task difficulty. Importantly, researchers should consider the possibility that multitasking can have both positive and negative effects.

Relation between Individual Factors, Learning, and Comprehension

As we noted earlier, our participants’ multitasking profiles indicated they were intense users of technology and felt positively about multitasking. Across three experiments, we found limited support for our hypothesis that prior multitasking and positive views about multitasking would be associated with lower performance on the experimental tasks assessing learning and comprehension. Multitasking frequency was negatively associated with experimental performance, but only in Experiment 2, whereas this relationship was not significant in Experiments 1 and 3. Similarly, although pacing and performance were not significantly related in Experiment 1, they were significantly and negatively associated in Experiments 2 and 3. This is logical given that we found no significant effects of our manipulation in Experiment 1. However, when our manipulation had an effect on learning, such as in Experiment 2, participants’ rating of pacing was negatively associated with academic performance. Lastly, participants’ prior experience with AIM and other instant messengers (measured only in Experiments 2 and 3) were negatively associated with performance in both Experiments 1 and 2.

Although we found only limited support for a relation between multitasking experience and learning, prior studies have found a negative correlation between multitasking frequency and lower cognitive performance (Fox et al., 2009; Ophir et al., 2009). Ophir et al., (2009) suggest that participants who engage in more multitasking may be more prone to distractibility. It is also possible that some of our participants who typically engaged in more multitasking were bored and less mentally engaged in our reading task. Regardless, given our mixed findings, future research on the effect of multitasking must examine the moderating role of pre-existing differences in reading ability, attention, academic engagement, and distractibility.

Limitations and Future Directions

Our studies were one of the first to study the effect of primary learning task difficulty and multiple modes of online communication while reading. Nonetheless they were not without limitations. The main limitation stems from the difficulties inherent to investigating multitasking in a laboratory. In pilot work we have found that many participants will not multitask on their own volition when given a reading task. This is likely because of the demand characteristics of the experimental setting and because they anticipate that they will have to take a test which assesses their comprehension and recall. Similarly, Adler and Benbunan-Fich (2012) gave their participants the freedom to multitask and observed that some participants still chose to do the tasks sequentially.

As a result we simulated a hypothetical multitasking experience for participants modeled after their typical experience. Simulating online communication in the laboratory ensured that participants’ reading was interrupted and also gave us greater control of the multitasking via social interaction. In particular it allowed us to control the frequency of messages, the conversation partner, the message topic, the reading topic, and the time participants spent reading. However, this also led to a limitation of our study in that the simulated online communication was somewhat artificial and lacked the natural flow of a conversation. Additionally, participants were told they could respond when they wanted and even ignore the messages if they wanted. Most responded after reading the message and none chose to not respond. Even with the modified social messages in Experiments 2 and 3, the responses were short with little reciprocity so the simulated conversation was one-sided.

Furthermore, we limited the messages to one per slide for the concurrent condition or sent them in regular, back-to-back succession for the sequential condition, whereas the frequency and pattern of real-world interactions are likely to vary with an individual’s engagement and with the conversation topic. It is worth noting that Fox et al. (2009) did not use a preset list of questions, yet also found no significant difference between multitasking and sequential conditions. Another limitation for Experiment 2 was that we used the statistics provided by the College Board to assess difficulty; future studies should use the measures of passage difficulty as well as include three levels of difficulty (easy, moderate, and difficult) in one experiment.

The particular platform used for multitasking is an important consideration that we did not address. Our participants reported that they often texted while reading, so such multitasking should be examined. Additionally, tablets (e.g., iPad, Kindle Fire, etc.) have become a popular platform for reading text and should be included in future studies. The diversity of concurrent computer use was also not addressed in our study, and it may be that some types of multitasking are detrimental to learning whereas other types are harmless or even beneficial. College students may multitask with productive behaviors such as note-taking or accessing a dictionary while reading, much like the course-related multitasking reported in Kraushaar & Novak’s (2010) study. As multitasking continues to grow in frequency and complexity, research must examine the specific effects of various multitasked activities and also how they may interact to impact learning and academic performance. Future research should explore which activities may be multitasked together with relative ease and which activities are in greater cognitive conflict, to help students multitask strategically and in ways that do not disrupt their learning.

In sum, over three experiments, we found little evidence of a disruptive effect of concurrently engaging in online communication while reading expository text. On the contrary, because we varied the difficulty of the primary learning task, we found preliminary evidence that multitasking might even enhance performance at low levels of intrinsic load. It is possible that college students multitask frequently multitask and so are able to both read and engage in social communication without disrupting performance in familiar and routine tasks. They are seemingly similar to the “supertaskers” of one study, who were effectively able to multitask with a cell phone while driving (Watson & Strayer, 2010). Having grown up with digital media, these “digital natives” may be able to use multiple media more efficiently. But even for them, cognitive load theory suggests that there may be certain activities and a maximum number of devices that can be used simultaneously without disruptions in performance. The challenge for researchers is to identify the conditions when multitasking might be beneficial and when it might be disruptive for performance in learning settings.

Acknowledgement

The authors are deeply grateful to Andrew Frane for his detailed edits on an earlier version of the manuscript and for suggesting the use of a formula to communicate the computation of the MFI.