“Cognitive control in media multitaskers” ten years on: A meta-analysis

Introduction

Frequent interaction with digital communications media has become a core component of life today (Vorderer et al., 2016), with increases in accessibility and connectivity facilitating opportunities for constant communication, information retrieval, and entertainment. Media use, therefore, regularly occurs in conjunction with other media or non-media activities —media multitasking (le Roux & Parry, 2019). While some researchers define media multitasking as the concurrent use of two or more forms of media (e.g., Voorveld, 2011), other researchers define it as the use of media alongside other non-media tasks (e.g., Ophir et al., 2009), and some consider it to refer to the use of media alongside other media or non-media tasks (e.g., Lang & Chrzan, 2015). Like multitasking in general, media multitasking, therefore, involves either the concurrent completion of at least two tasks (i.e., dual tasking) or the rapid switching between tasks. Irrespective of media involvement, at a cognitive level, both occur through rapid switching (Salvucci & Taatgen, 2008). The key distinction between multitasking in general and media multitasking, therefore, concerns the involvement of a digital medium in some form or another.

Given the widespread use of media in daily life (Vorderer et al., 2016), the prevalence of media multitasking across generations and socioeconomic spectra (Baumgartner & Sumter, 2017; le Roux & Parry, 2017; Voorveld & van der Goot, 2013; Rideout et al., 2010), and the possibility that frequent behaviours may impact cognitive performance in various domains (Dehaene & L. Cohen, 2007; Dux et al., 2006; Lin, 2009), it is important to understand the implications that this new media phenomenon—pervasive multitasking facilitated by accessible media—may hold for an individual’s cognitive functioning. Researchers have, as a result, investigated possible relationships between media multitasking and outcomes relating to cognitive control (see Uncapher & Wagner, 2018; van der Schuur et al., 2015 for reviews).

Background

Ophir et al. (2009) first investigated associations between media multitasking and cognitive control, producing the media multitasking index (MMI) — “an indication of the level of media multitasking the participant is engaged in during a typical media-consumption hour” (p. 15586). In a series of studies Ophir et al. (2009) classified participants as either heavy or light media multitaskers based on their MMI scores and compared their performance across a number of assays of cognitive control. For filtering environmental distractions, Ophir et al. (2009) show that, while light media multitaskers’ (LMMs; n = 22) performance was not impacted by distractors, heavy media multitaskers (HMMs; n = 19) were negatively impacted by the presence of distractors in both the change-detection and AX-continuous performance tasks. For filtering of irrelevant stimuli in working memory, Ophir et al. (2009) show that HMMs (n = 15) were less effective at filtering than LMMs (n = 15). For task switching, HMMs (n = 15) exhibited larger switch costs and slower response times than LMMs (n = 15). Ophir et al. (2009) interpret these results to suggest that HMMs have a greater tendency for exogenous attentional control, are less adept at filtering irrelevant internal or external stimuli and are less effective at switching between tasks.

Building on Ophir et al. (2009), over the previous decade, numerous studies have investigated associations between media multitasking and cognitive control using both self-report and performance-based assays of cognitive control. In the first review of studies in this domain (k = 9) van der Schuur et al. (2015, p. 208) concluded that, “despite common concerns regarding the negative effects of media multitasking on cognitive control, the existing studies only partly support this concern.” Specifically, the review found that, at the time, studies using self-reports produced evidence supporting a negative relationship between media multitasking and cognitive control in everyday life. In contrast, studies adopting a performance-based paradigm were found to be more divergent. More recently, two reviews have focused solely on studies using performance-based assays, while one has only considered self-report indications of everyday executive functioning. Uncapher and Wagner (2018) reviewed 21 reports and concluded that, while “the literature is still sparse, and is marked by both convergent and divergent findings, the balance of evidence suggests that heavier media multitaskers exhibit poorer performance in a number of cognitive domains, relative to lighter media multitaskers” (p. 9889). Prior to this review, Wiradhany and Nieuwenstein (2017) conducted a meta-analysis of 39 performance-based assays across 12 studies assessing the same cognitive functions as Ophir et al. (2009), and found a small association between media multitasking and ‘distractibility’ (d = 0.17). Subsequently, to consider associations between media multitasking and self-reported indications of everyday executive functioning, Wiradhany and Koerts (2021) conducted a series of meta-analyses for four distinct functions (attention regulation, behaviour regulation, impulsiveness/inhibition, and memory) involving 50 assessments from 16 studies. Across these analyses, small associations between media multitasking and problems in all four cognitive processes were found (z = .16 to z = .22).

Rationale for Associations Between Media Multitasking and Cognitive Control

Following Ophir et al. (2009), cognitive control has become one of the primary outcomes of interest for studies investigating media multitasking and its potential effects. While various conceptions exist (see Burgess, 2004; Diamond, 2013; Miyake et al., 2000, for instance), cognitive control can broadly be understood to refer to the mechanisms underlying the execution of goal-directed behaviour (Gilbert & Burgess, 2008) and, while some operational unity exists, this operation occurs through the involvement of various cognitive or executive functions (Miyake et al., 2000). Importantly, the exact nature of these executive systems is difficult to delineate. This is largely due to the functional unity of executive operations and the goal-related nature of cognitive control. Burgess (2004) explains that the term ‘executive functions’ is not an operational definition. Rather, in his view, it is a theoretical definition, with many distinct (and often overlapping) models describing the operation of cognitive control. Miyake et al. (2000), for instance, describe three key executive functions. The first, shifting, concerns switching between multiple tasks or mental sets. The second executive function relates to the updating and monitoring of representations in working memory, and the third describes the inhibition of prepotent responses.

Specifically focusing on prior media multitasking research, Uncapher and Wagner (2018) categorised the outcomes applicable for various executive functions into five broad categories: working memory, interference management (internal and external filtering), sustained attention, task goal management (task-switching, dual-tasking), and inhibitory control (prepotent impulsive response inhibition). In this context, functions categorised as pertinent for working memory support the encoding, short-term maintenance, retrieval, and possible updating of task-relevant information. Interference management describes executive functions relevant to the filtering of task-irrelevant or distracting information. Sustained attention, similarly, concerns the ability to maintain attentional allocation to a stimulus, typically in a goal-directed manner. Task goal management, traditionally known as cognitive flexibility, concerns task-switching and dual-tasking functions, while the final category of executive functions highlighted, inhibitory control, concerns the ability to withhold prepotent or impulsive responses. Notably, these broad, high level functional categories—working memory, interference management, sustained attention, task goal management, and inhibitory control—embody the notion that the executive functions are highly correlated yet distinct, with each drawing on aspects of more fundamental executive functions while still referring to distinct cognitive functions and processes. It is believed that Uncapher and Wagner’s (2018) categorisation of extant performance-based media multitasking research along the lines of these five functions is useful, and that there is merit in extending this line of reasoning to include self-reported data and a meta-analytic synthesis of relevant findings.

To understand the possible nature of relationships between media multitasking and various cognitive control functions, at both the level of individual functions and higher-level functional categories various theories have been proposed. Despite this, studies in this domain have generally used cross-sectional research designs. Therefore, our understanding about the causal direction (and mechanisms) for possible relationships between media multitasking and cognitive control are limited.

Importantly, while various theories and models, extending from both cognitive psychology and mediated communication domains, have been applied to the study of media multitasking (e.g., cognitive load theory, threaded cognition, or the limited capacity model of attention; Kahneman, 1973; Paas & van Merrienboer, 1994; Salvucci & Taatgen, 2008), such theories primarily concern performance effects associated with concurrent multitasking with media (e.g., momentary attentional performance or academic performance; van der Schuur et al., 2015), and not possible longer term effects on cognition. Similarly, a number of studies consider the cognitive demands associated with media multitasking behaviour (Lang & Chrzan, 2015; Wang et al., 2015). While such research provides an important conceptual foundation that may contribute to understanding the phenomenon and its effects, little concrete theoretical and empirical work has been conducted to investigate the mechanisms that may underlie associations between media multitasking tendencies and cognitive control.

To this end, while Ophir et al. (2009) posited that HMMs have a greater tendency for bottom-up attentional control (the breadth-biased hypothesis), no causal mechanism has emerged. Media multitasking can be associated with a breadth-biased approach to attention either because it causes HMMs to function in such a manner, or this bias may exist due to various individual differences. These alternatives describe two distinct directions of causality between media multitasking and diminished cognitive control. The deficit-producing hypothesis proposes that, through repeated outsourcing of control to exogenous distractions, endogenous control of attention is attenuated (Ralph et al., 2014). Alternatively, through processes of attentional and instrumental learning, individuals become habituated to adopting broader distributions of attention. Other researchers have proposed that individual user characteristics may give rise to media multitasking. These differences extend past media use and, therefore, may impact cognitive outcomes (Jeong & Hwang, 2016). The strategic hypothesis posits that individual differences in media multitasking are indicative of general strategies for behaviour and attention (Ralph et al., 2015). HMMs self-report increased attentional failures in everyday life, not because of deficits in executive functioning but, rather, they adopt an attentional strategy allowing themselves to become distracted (le Roux & Parry, 2019; Ralph et al., 2018). While this strategy may be reflected in self-reports of attentional failures, it may not manifest in more artificial laboratory settings. Ralph and Smilek (2017) argue that individual differences in thresholds for engagement may explain this strategic choice. Because attentional capacities are limited (Kahneman, 1973), rapid attentional switches, which have been shown to increase arousal levels (Yeykelis et al., 2014), are necessary. Baumgartner et al. (2017) suggest that those who frequently engage in media multitasking may become habituated to such elevated arousal levels and, accordingly, adopt strategies to achieve commensurate arousal when not engaged. Alternatively, in the self-selection hypothesis, it is proposed that individuals with deficits in cognitive control may multitask more than others (Ralph et al., 2014).

These propositions all presume the existence of a practically meaningful association between media multitasking and cognitive control. Additionally, these propositions are not necessarily mutually exclusive. The cognitive control of those who are biased to, or choose to adopt particular attentional strategies, may be further impacted by their media multitasking. Baumgartner et al. (2017), for instance, show that, for some adolescents, media multitasking moderates pre-existing individual differences. Despite a decade of research, given the divergent findings produced, the true nature of associations between cognitive control and media multitasking remains uncertain.

An important element present in findings discussed in previous reviews (van der Schuur et al., 2015) and in some of the suggested mechanisms (le Roux & Parry, 2019; Ralph et al., 2018; Ralph & Smilek, 2017), is the difference between self-reported and task-based measures of executive functioning. As noted, van der Schuur et al. (2015) found that results differed between studies dependent on how executive functioning was assessed, with negative outcomes consistently shown with self-reports, but more divergent outcomes with task-based assays. While both measurement approaches assess executive functioning, as Toplak et al. (2013) note, there tends to be only small associations between the approaches. This is similar for other domains of executive functioning (e.g., self-regulation), where there is only a small association between self-report and task-based measures (Enkavi et al., 2019; Sharma et al., 2014). These differences have led some researchers to suggest that differences may result from low validity in either self-reports or tasks (Barkley & Fischer, 2011). Others, in contrast, suggest that they are both valid measures of executive functioning and, importantly, capture different aspects of the construct (Toplak et al., 2013; Friedman & Banich, 2019). Performance-based assays assess the underlying processing efficiency of an executive function in an artificial setting. Self-reports, by contrast, relate to reflections on action in context. Nonetheless, the absence of strong convergent validity between self-reported and task-based measures of executive functioning, and prior evidence that the association between cognitive control and media multitasking differs depending on how cognitive control is assessed, suggest, firstly, that a systematic investigation of the available evidence will show continued support for this difference and, secondly, that other confounding factors that account for the difference between self-reported and task-based measures of executive functioning (e.g., goals, motivations, and other contextual factors; Barkley & Fischer, 2011; Lin, 2009) may also account for differences involving media multitasking.

The Present Study

Given the mixed results in this domain, after a decade of research, there is a need to synthesize extant evidence across cognitive control functions and measurement approaches. Although recent reviews have provided useful summaries, they are either out-of-date, limited in scope, do not include a comprehensive sample of outcomes, or do not directly compare outcomes for different cognitive control functions and measurement approaches. Given the prevalence of media multitasking, and the importance of attentional control and sustained attention across a variety of academic, professional, and social contexts (Parry et al., 2020), a more definitive understanding of associations between media multitasking and cognitive control is required. Although some researchers have claimed that the “weight of current evidence shows that in some contexts heavier media multitaskers underperform relative to lighter media multitaskers in a number of cognitive domains” (Uncapher & Wagner, 2018, p. 9890), concrete statistical synthesis of the full range of evidence has yet to be conducted. Therefore, the primary objective of this study is to synthesize existing research into associations between trait¹ media multitasking and cognitive control, across measurement approaches, for different cognitive control functions.

In providing a synthesis of existing findings, the review aims to uncover potential sources of bias, identify the strength of associations across studies, populations, and cognitive functions and, potentially, help to explain seemingly contradictory findings. As we seek to further understand the phenomenon, uncovering sources of variation across studies and quantitatively assessing the magnitude of the supposed association across all relevant primary studies will provide researchers with a robust foundation upon which future investigations can build. To this end, building on previous research in this regard, a number of hypotheses are specified. Despite mixed outcomes, acknowledging previous research, it is hypothesised that:

H1: Those classified as having higher levels of trait media multitasking will indicate greater problems with cognitive control (i.e., more distractibility) relative to those classified as having lower levels of trait media multitasking.

Additionally, given differences observed across measurement approaches (self-report vs. performance-based), it is hypothesised that:

H2: Associations between media multitasking and cognitive control will differ based on the measurement approach adopted in primary studies (a), with stronger associations with distractibility being evident in self-reported data than in performance-based data (b).

Acknowledging divergent outcomes observed between different functions of cognitive control, we ask:

RQ1: To what extent do the associations between media multitasking and cognitive control differ based on the function of cognitive control assessed?

To address these objectives a meta-analysis of studies published since Ophir et al. (2009) (inclusive) was conducted. Gurevitch et al. (2018, p. 177) note that, in cases such as this, it is common to include results from a population of studies involving heterogenous samples, methodologies, and outcomes to evaluate the evidence for an effect under question. Despite this, emphasis is placed on identifying key moderators, conducting subgroup analyses, and assessing the heterogeneity of outcomes. The results of this investigation will provide perspective on the current body of knowledge on media multitasking effects and, on this basis, inform future research on media multitasking.

Method

A research protocol was developed in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) (Liberati et al., 2009). The protocol was pre-registered and is available through the Open Science Framework at: https://osf.io/yt5n3

Eligibility Criteria

We used the following criteria for inclusion. First, as with Uncapher and Wagner (2018), we restricted the review to studies that used quantifiable indices of media multitasking behaviour (i.e., the MMI or similar instruments for trait media multitasking – adjusted or short versions of this measure). This implies that assessments conducted under experimental conditions were ineligible (only pre-manipulation assessments were eligible). Second, studies must have related media multitasking to indications of cognitive control through either performance-based or self-reported measures. Specifically, studies must have considered associations between media multitasking and one of the five broad, high-level functions of cognitive control described by Uncapher and Wagner (2018): working memory, interference management, sustained attention, task goal management, or inhibitory control. This will ensure that comparisons presented will be broadly applicable to established cognitive domains and cover the range of work previously synthesized, while also enabling their combination and synthesis across measurement approaches and designs, without being too narrow or broad in their characterisation. For those eligible studies that used self-report scales, measures of everyday executive functioning must have been collected for at least one of these five functions while, for eligible studies adopting a performance-based approach, tasks assessing performance for any of the cognitive domains were eligible.

In addition to these criteria, further restrictions were specified. As Ophir et al.’s (2009) study was published in 2009, eligible studies conducted subsequent to this date (inclusive), and prior to the period of data collection (the second half of 2019), were included in the analysis. As with Uncapher and Wagner (2018), assessments were excluded if they involved associations between media multitasking and real-world indications of task performance (e.g., academic performance, job performance). To calculate effect sizes sufficient quantitative information is required. Therefore, studies solely relying on qualitative methodologies to assess media multitasking tendencies or cognitive control outcomes were excluded. No restrictions were placed on eligible participant populations, nor were restrictions placed on the reporting language. Additionally, both published literature and, in the case of grey literature (e.g., theses or pre-prints), unpublished studies were eligible. For theses or pre-prints subsequently peer-reviewed, only the published record was considered.

Search Strategy

A search strategy consisting of three phases was implemented. The first phase involved searching four comprehensive bibliographic databases that provide good coverage of general and psychological academic literature within both conference and journal publications (Web of Science, Pubmed, Scopus, and PsycARTICLES). Together, while some overlap exists, these databases cover over 200 million records across academic fields. A generic search string was used and, for each database, adjusted as required. The first clause included terms relating to media and related technologies; the second related to multitasking behaviour; and the third included terms relating to cognitive control. The full search strings are provided in Appendix. The second phase involved manually searching three relevant journals (Attention, Perception, & Psychophysics, Computers in Human Behavior, and Psychonomic Bulletin and Review). Finally, after assessing eligibility, additional ‘backward’ and ‘forward’ searches were conducted.²

Data Extraction

While the search procedure was designed to identify eligible reports, the unit of analysis in this study is the individual assessment of an association between media multitasking and a relevant aspect of cognitive control. For self-reports an assessment was defined as a comparison between trait media multitasking levels and the outcomes of a relevant scale or (if applicable) sub-scale. The full scale was considered if it assessed everyday executive functioning relating to one of the five functions. However, if a scale assessing a broad range of cognitive functions was used (e.g., the Behaviour Rating Inventory of Executive Function–BRIEF), relevant sub-scales were considered as separate outcomes (e.g., the working memory, inhibition, and shifting sub-scales in this case) if available. However, if a full scale could not be directly matched to a single cognitive function (e.g., the Cognitive Failures Questionnaire–CFQ) then it was not included in the analysis.

For performance-based assays, an assessment was defined as the outcome for each relevant metric assessing an outcome relevant for the categories described by Uncapher and Wagner (2018). To this end, as with Wiradhany and Nieuwenstein (2017), a single outcome per task was selected for inclusion in the analysis. To avoid possible selection biases, we opted to select outcomes most closely approximating those used by Wiradhany and Nieuwenstein (2017), who based their selection on similarity to Ophir et al. (2009) and the degree to which a task outcome-metric assessed performance under conditions of distraction. In instances of tasks not included in this earlier study, we considered the metrics available, their relation to those reported in Wiradhany and Nieuwenstein (2017) and, further, their description and relation to the tasks, task-metrics, and outcomes described by Uncapher and Wagner (2018). It is believed that, in this way, not only was comparability with previous studies supported but, additionally, the metrics most appropriate were selected for inclusion. Importantly, given the need to synthesise outcomes on a common scale, and the differences in research and analysis designs adopted, interaction effects were not considered eligible outcome metrics. In longitudinal designs the effect size for the first-time point was extracted. Similarly, for experimental designs effect sizes were only extracted from any pre-manipulation analyses conducted. Any post-manipulation comparisons, or comparisons under experimental conditions, were excluded.

 If studies reported multiple measures for media multitasking, only a single measure was considered. Selection was based on similarity to Ophir et al.’s (2009) MMI, with comparisons based on the measure presenting the closest approximation of the MMI chosen. If a study reported both correlational and between-groups outcomes for the same assessment, given the well-documented problems with artificial dichotomisation (J. Cohen, 1983), outcomes for the correlational comparison were selected. If sufficient data to compute the effect sizes were not available in the reports (or the available supplementary materials), we contacted the corresponding authors to ask for further data analysis, missing data points, or raw data. Alternatively, we used a web plot digitiser to extract data from reported figures.

Data Analysis Procedures

Effect sizes were calculated as correlation coefficients. Ranging from -1 to 1, positive values indicate a positive relation between trait media multitasking and distractibility and negative values the opposite, with higher absolute values representing a stronger association. A positive value corresponds to an effect in the direction hypothesised in H1. For studies adopting correlational designs the Pearson product-moment correlation coefficient r was adopted as an effect size estimate. To normalise the sampling distribution of effect sizes (and stabilise the variance) Fisher’s r-to-z transformation was applied. Noting that there exists conflicting recommendations on effect size conversion and variance estimation (Borenstein et al., 2011; Hunter & Schmidt, 2004; Jacobs & Viechtbauer, 2017; Rosenthal, 1991), on the basis of current recommendations (Jacobs & Viechtbauer, 2017; Viechtbauer, 2010), we transformed statistics reported in primary studies adopting either dichotomised or extreme-groups designs into appropriate correlation coefficients suitable for synthesis. For studies adopting a between-groups design, effect sizes were computed as the biserial correlation coefficient r_b (Jacobs & Viechtbauer, 2017) from the reported statistical tests or descriptive statistics. For studies adopting an extreme-groups design the point biserial correlation coefficient r_pb was first calculated and then transformed into the r_b.

Adopting a significance level of α = 0.05, to describe the association between media multitasking and cognitive control, a random-effects model was produced. Subsequently, to provide greater insight into the outcomes for each cognitive control function, separate random-effects models were calculated. Additionally, separate models were produced for each measurement approach used to assess a given cognitive control function. To account for variance inflation resulting from correlated observations for different measures for the same participants we calculated the robust variance estimation (RVE). For the overall model, to consider the possible moderating effect of measurement approach and cognitive control function, we conducted two moderator analyses using a meta-regression approach (Borenstein et al., 2011). In these mixed-effects models the moderators were added as categorical variables. Heterogeneity in effect sizes was assessed with Cochran’s Q and I².

To evaluate the impact of small-study effects, as Egger et al. (1997) suggest, we used a funnel plot to compare the observed effect sizes to measures of their precision. Additionally, in highlighting the area of statistical significance the presence of studies producing null effects can be observed. To quantify the asymmetry, we performed Egger’s regression test (Egger et al., 1997). In the event of asymmetry, we adopted Duval and Tweedie’s (2000) procedure to impute the ostensibly missing effect sizes and account, to some extent, for this bias.

Results

Included Studies

The initial search yielded 6211 results. After duplicates (n = 1,354) were removed the titles and abstracts of the remaining papers (n = 4,857) were assessed for eligibility. Ineligible records (n = 4,794) were excluded before the full texts of those remaining (n = 63) were assessed. This process was conducted by the first author, with separate assessment by the second. Disagreements on inclusion were resolved through discussion after interrogating the relevant study components. Ineligible results (n = 36) were removed. This resulted in a sample (n = 27) with which forward and backward searches were performed. The final sample supplemented by the results of these searches was then established (n = 32). A summary of this search and inclusion process is provided in Figure 1. As suggested by a reviewer, an additional search was conducted in October 2020 including ‘television’ as an additional search term (see Appendix). After removing results produced during the original search, this search yielded 338 new results. Of these, 315 were eliminated based on their title or abstract with 23 full texts assessed. Two of these were already included in the review and 17 were eliminated, leading to the inclusion of four additional papers during the review process. From the 36 records identified, 46 studies and 118 assessments were included in the review.³

Table 1 provides an overview of the characteristics of the included assessments. Across studies a mean sample size of 183.11 (SD = 227.19, Mdn = 105) was used, with the smallest sample (n = 20) seen in Wiradhany and Nieuwenstein (2017) and the largest, 1367, in Ernst (2014). Studies adopting performance-based assays included a mean sample of 104.34 (SD = 104.95, Min = 20, Max = 523, Mdn = 72), while those adopting self-reported measures involved a mean of 310.89 (SD = 303.71, Min = 50, Max = 1367, Mdn = 197) individuals. A majority of studies targeted a student population (72.88%), with only 20 involving adolescents (16.95%), and 12 either not indicating a specific population or involving a mixed population of adolescents, students and adults (10.17%). Of the 118 assessments included, 73 (61.86%) were performance-based and 45 (38.14%) were self-reported and, on average a study included a mean of 2.56 (SD = 1.97, Min = 1, Max = 10, Mdn = 2) assessments. As indicated in the second column of Table 1, a majority of included assessments (n = 75, 63.56%) involved the original MMI proposed by Ophir et al. (2009) as a measure of media multitasking, while 43 (36.44%) used modified versions of this measure. For those that modified the measure, 31 presented shortened versions with less media categories, 19 retained 12 categories but presented different categories, two presented the items in a different format, and two used different time frames (notably, some studies included more than a single modification to the measure).

Figure 1. A Flowchart for Study Inclusion.

Table 1. Overview of Studies, Measures, and Metrics Included in the Meta-Analysis.

Overall Meta-Analytic Result

Overall, across all 118 assessments, the effect size for the association between media multitasking and cognitive control is small (z = .138, 95% CI [.107, .170], p < .001; with RVE: 95% CI [.102, .174], p < .001) and, as would be expected, highly heterogeneous (I² = 79.76%, Q(117) = 531.795, p < .001). This effect size, while small, supports H1. No relationship between effect size and study precision was found (z = .575, p = .565).

To address H2 and RQ1, two moderator analyses were conducted. In the first, measurement approach was shown to moderate the results (Q_M(1) = 14.544, p < .001) and similarly, in the second, cognitive control function also served as a moderator (Q_M(4) = 18.551, p = .001). To further understand these effects, two random effects models were produced for the measurement approach. For performance-based assays the effect size is negligible but statistically significant (z = .091, 95% CI [.044, .139], p = .001), with a substantial degree of heterogeneity (I² = 73.75%, Q(72) = 231.018, p < .001). For self-report measures the pooled effect size is small, statistically significant (z = .200, 95% CI [.165, .231], p < .001), and heterogeneous (I² = 73.40%, Q(44) = 177.87, p < .001). For self-reports, Egger’s test was not statistically significant (z = -.803, p = .422) while, for performance-based assays, the test indicated an association (z = 3.129, p = .002). To account for this, the trim-and-fill method (Duval & Tweedie, 2000) was used to impute the ostensibly missing effects. In this sensitivity analysis, the pooled effect for performance-based assays was no longer statistically significant (z = .032, 95% CI [-.024, .088], p = .260).

Two further post hoc exploratory moderator analyses were conducted for the overall meta-analytic result to determine if the population targeted or the form of media multitasking measure accounted for the heterogeneity in outcomes. The moderator analysis for media multitasking measure was not statistically significant (Q_b(1) = .004, p = .948), suggesting that findings likely do not differ between studies using either the original (z = .137, p < .001) or a modified MMI (z = .139, p < .001). In contrast, after excluding three comparisons that did not specify the specific population involved, population was found to account for part of the heterogeneity in the overall result (Q_b(2) = 16.983, p < .001), with statistically significant associations found for adolescents (z = .170, p < .001) and students (z = .150, p < .001), but not adults (z = -.082, p = .142). Notably, only nine comparisons included samples of adults not recruited from a student pool, limiting interpretation of this finding.

Sustained Attention

Associations between media multitasking and sustained attention have been assessed in 15 distinct studies, with a majority of assessments (n = 26) using self-report scales (n = 22) rather than performance-based assays (n = 4). Overall, across measurement approaches, the pooled effect size is small (z = .192, 95% CI [.139, .245], p < .001; with RVE: 95% CI [.132, .252], p < .001) and heterogeneous (I² = 83.88%, Q(25) = 150.00, p < .001).

Self-Report Findings

Figure 2 depicts a forest plot of the effect sizes used in the meta-analysis for associations produced on the basis of self-reported assessments. Across assessments, the pooled effect size is small (z = .200, 95% CI [.142, .257], p < .001; with RVE: 95% CI [.131, .269], p < .001), with a high degree of heterogeneity (I²= 86.08%, Q(21) = 142.134, p < .001). For this subgroup, no relationship between effect size and study precision was indicated by Egger’s test (z = -1.656, p = .097).

Figure 2. Forest Plot of the Effect Sizes (Fisher’s z) for Studies Assessing the Association Between Media Multitasking and Sustained Attention With Self-Report Instruments.

Performance-Based Findings

Four performance-based assessments have been conducted considering associations between media multitasking and sustained attention. Figure 3 depicts a forest plot of the effect sizes for these assessments. Across the four outcomes, the pooled effect size is small and not statistically significant (z = .133, 95% CI [-.092, .357], p = .156; with RVE: 95% CI [-.092, .357], p = .156). Additionally, indicating that this outcome is relatively consistent, heterogeneity is low (I² = 44.09%, Q(3) = 5.343, p = .148), and no indication of an association between effect size and study precision (z = -.798, p = .425) was found.

Figure 3. Forest Plot of the Effect Sizes (Fisher’s z) for Studies Assessing the Association Between Media Multitasking and Sustained Attention With Performance-Based Instruments.

Working Memory

Twenty-nine assessments of relationships between media multitasking and measures of working memory were found. Of these, 26 are based on performance-based assays, while only three use self-report measures. Overall, the effect size is small (z = .181, 95% CI [.107, .256], p < .001; with RVE: 95% CI [.108, .255], p < .001), and heterogeneous (I² = 76.35%, Q(28) = 83.776, p < .001).

Self-Report Findings

For associations with working memory only three studies have used self-report measures (Baumgartner et al., 2014; Magen, 2017; Rogobete et al., 2020). In all three, the WM subscale of the BRIEF was used. In this circumstance, Borenstein et al. (2011, p. 364) recommend reporting the pooled effect sizes but, importantly, urge caution in their interpretation. Therefore, while a statistically significant effect size exists (z = .203, 95% CI [.144, .262], p < .001), the between-studies variance may be substantially in error which, accordingly, undermines the accuracy of the confidence interval. Additionally, as there are only three assessments, any understanding of the degree of dispersion and heterogeneity of outcomes is limited.

Performance-Based Findings

As would be expected given the proportion of WM assessments using performance-based assays, and the overall model for this association, the effect size for studies adopting this measurement approach was small (z = .182, 95% CI [.094, .271], p < .001; with RVE: 95% CI [.095, .270], p < .001), and heterogeneous (I² = 76.41%, Q(25) = 78.211, p < .001). Figure 4 depicts a forest plot of the relevant effect sizes. Egger’s test indicated no relationship between effect size and study precision (z = 1.582, p = .114).

Figure 4. Forest Plot of the Effect Sizes (Fisher’s z) for Studies Assessing the Association Between Media Multitasking and Working Memory With Performance-Based Instruments.

Inhibitory Control

Associations between media multitasking and indications of inhibitory control have been assessed in 17 studies, with a majority of assessments (n = 27) using self-report scales (n = 17) rather than performance-based assays (n = 10). Overall, the effect size is small (z = .163, 95% CI [.114, .211], p < .001; with RVE: 95% CI [.109, .216], p < .001), and heterogeneous (I² = 61.18%, Q(26) = 63.521, p < .001).

Self-Report Findings

Across self-report analyses, the effect size is small (z = .210, 95% CI [.173, .247], p < .001; with RVE: 95% CI [.171, .249], p < .001), with a low level of heterogeneity (I² = 21.16%, Q(16) = 19.293, p = .254). Egger’s test indicated no relationship between effect size and study precision (z = .575, p = .566). Figure 5 depicts a forest plot of the relevant effect sizes for this analysis.

Figure 5. Forest Plot of the Effect Sizes (Fisher’s z) for Studies Assessing the Association Between Media Multitasking and Inhibitory Control With Self-Report Instruments.

Performance-Based Findings

For associations with media multitasking the effect size is small and not statistically significant (z = .051, 95% CI
[-.048, .149], p = .313; with RVE: 95% CI [-.054, .155], p = .293), with a moderate level of heterogeneity (I² = 57.13%, Q(9) = 21.238, p = .012). No relationship between effect size and study precision was indicated by Egger’s test (z = .606, p = .545). Figure 6 depicts a forest plot of the effect sizes included in the meta-analysis.

Figure 6. Forest Plot of the Effect Sizes (Fisher’s z) for Studies Assessing the Association Between Media Multitasking and Inhibitory Control With Performance-Based Instruments.

Interference Management

All fifteen assessments considering associations between media multitasking and indications of interference management have adopted a performance-based paradigm. Overall, the effect size is small and not statistically significant (z = .057, 95% CI [-.051, .166], p = .302; with RVE: 95% CI [-.061, .175], p = .315). Additionally, while only a single measurement approach has been used, there remains a considerable degree of heterogeneity in outcomes (I² = 76.41%, Q(14) = 50.579, p < .001). Figure 7 depicts a forest plot of the relevant effect sizes.

Figure 7. Forest Plot of the Effect Sizes (Fisher’s z) for Studies Assessing the Association Between Media Multitasking and Interference Management With Performance-Based Instruments.

Egger’s test indicated there to be a relationship between effect size and study precision (z = 2.835, p = .005). Therefore, in accordance with Duval and Tweedie (2000), we produced a funnel plot in which effect sizes were plotted against their precision, highlighting the conventional areas of statistical significance. As is evident in the first plot in Figure 8 the distribution is asymmetrical, with a number of small sample studies showing large, positive effects in contrast to an absence of similarly small studies showing effects on the other side of the distribution. Using the trim-and-fill procedure (see the second plot in Figure 8), the possible presence of small-study effects is highlighted. Moreover, in imputing the ostensibly missing effects, as a sensitivity analysis, the meta-analytic results were further supported (z = -.028, 95% CI [-.156, .101], p = .674). While not conclusive, this asymmetry is typically either interpreted as evidence of biases in reporting or the existence of other confounding factors not assessed in the analysis (i.e., differences in study populations or tasks).

Figure 8. Funnel Plots Showing the Relationship Between the Effect Sizes and Standard Errors of Studies Adopting Performance-Based Assays to Assess Associations Between Interference Management and Media Multitasking.

Task Management

Twenty-one assessments for associations between media multitasking and task management have been conducted, with a majority (n = 18) using performance-based assays. Overall, the effect size is small, not statistically significant (z = .031, 95% CI [-.039, .101], p = .387; with RVE: 95% CI [-.058, .120], p = .467), and heterogeneous (I² = 64.81%, Q(20) = 49.940, p < .001).

Self-Report Findings

As with assessments for self-reported WM, only three studies (Baumgartner et al., 2014; Magen, 2017; Rogobete et al., 2020) have used self-report measures to consider associations between media multitasking and task management. In all three cases the shifting subscale of the BRIEF was used. Again, while a statistically significant effect was found (z = .105, 95% CI [.044, .166], p < .001), the same cautions apply to interpretation.

Performance-Based Findings

The effect size for studies adopting this measurement approach was small, not significant (z = .015, 95%
CI [-.074, .104], p = .742; with RVE: 95% CI [-.101, .131], p = .783), and heterogeneous (I² = 66.79%, Q(17) = 43.155, p < .001). Figure 9 depicts a forest plot of the effect sizes included in the meta-analysis. Egger’s test indicated no relationship between effect size and study precision (z = 1.053, p = .292).

Figure 9. Forest Plot of the Effect Sizes (Fisher’s z) for Studies Assessing the Association Between Media Multitasking and Task Management With Performance-Based Instruments.

Summary of Key Findings

Before discussing the outcomes of the review and meta-analysis, Table 2 provides a summary of the key findings.

Table 2. Summary of Key Findings.

Discussion

Building on findings first described by Ophir et al. (2009), over the subsequent decade, there has been a growth in the number of studies investigating associations between media multitasking and cognitive control. Assessments have primarily been conducted with a performance-based measurement approach, with less than a third involving self-reports. Noting the perspectives provided by previous reviews and meta-analyses (Uncapher & Wagner, 2018; van der Schuur et al., 2015; Wiradhany & Koerts, 2021; Wiradhany & Nieuwenstein, 2017), the aim of the present study was to provide a complete synthesis of existing findings across measurement approaches and cognitive control functions.

In relation to our first hypothesis, the results of the meta-analysis support the argument that those classified as having higher levels of media multitasking will indicate greater problems with cognitive control (i.e., more distractibility) relative to those classified as having lower levels of media multitasking. Notably, however, the effect size for this association is small and likely to be of minimal practical significance. Additionally, as would be expected with an analysis such as this that pools effects for different dependent variables and measurement approaches, the outcome was highly heterogeneous, indicating that this effect is likely to be moderated by various unmeasured factors. While the analysis supports the existence of an association, it does not speak to any causal effect between media multitasking and distractibility.

For our second hypothesis, both the moderator and sub-group analyses confirm that associations between media multitasking and cognitive control differ based on the measurement approach adopted. This outcome extends van der Schuur et al. (2015) who, based on a qualitative review of studies published prior to 2015, identified this distinction. Additionally, where effects were shown, the association was larger for studies using self-reports than performance-based assays.

Considering our research question, across measurement approaches, no associations were found for assessments concerning interference management or task management, while small associations were found for studies assessing associations between media multitasking and sustained attention, working memory, or inhibitory control. For associations with diminished working memory performance, this outcome is primarily driven by outcomes derived from performance-based assays—the only function for which a statistically significant relationship was found. Conversely, for associations with sustained attention outcomes, the result is primarily driven by associations with self-reported outcomes. While many meta-analytic associations were not statistically significant, where appropriate, the available evidence does not support the notion that increased media multitasking improves task-switching abilities or executive functioning in general. However, while many observed associations are small, the available evidence also suggests that it is not associated with attenuated abilities to manage tasks.

Further research is required to determine if the nature of the media involved in a media multitasking situation explain, at least some of, the variability in outcomes. For this, rather than the MMI (or similar measures), researchers will need to measure specific instances of media multitasking. Wilmer et al. (2017), for instance, argue in addition to different media and their modalities, the combined cognitive load of their simultaneous use is an important consideration for understanding the cognitive effects of frequent media multitasking. Additionally, a further factor that may explain some of the heterogeneity in outcomes, extending from the use of different measures of media multitasking, is the different ways in which media multitasking has been conceptualised. Researchers have variously considered the concept to refer to the simultaneous use of two or more media, media use alongside non-media activities, or some combination of the two. Accordingly, while all included studies used either the original MMI or a variation on its form, many who modified the measure used different combinations of activities (mediated and non-mediated) to arrive at their measure for media multitasking.

With a broader sample of studies, the findings of the present meta-analysis generally support the two previous meta-analyses in this domain (Wiradhany & Koerts, 2021; Wiradhany & Nieuwenstein, 2017), which separately analysed limited samples of either performance-based or self-reported assessments of associations. In the first, Wiradhany and Nieuwenstein (2017) analysed the results described in 12 studies using the same or similar tasks to Ophir et al. (2009), finding a small effect. In this earlier study, no moderating effect of cognitive control function was found. Later, Wiradhany and Koerts (2021) conducted a series of meta-analyses of self-reported associations between media multitasking and four aspects of everyday executive functioning. The magnitude and of the associations found in the present study confirm those reported in this earlier study. Consequently, across previous reviews and the results described in the present study, the divergent outcomes between measurement approaches are consistent.

A key question, raised previously by van der Schuur et al. (2015) and Wiradhany and Koerts (2021), is why the disparity in results between the two measurement approaches exists? While both ostensibly assess cognitive control, as Toplak et al. (2013) note, there tends to be only small associations between the approaches. Therefore, commensurate results should not necessarily be expected. Performance-based assays assess the underlying processing efficiency of an executive function in an artificial setting. Self-reports, by contrast, relate to reflections on action in context. Performance is grounded in context and experience, and is related to goals and intentions. Performance-based measures, in their assessments of functional efficiency, disregard the role of goals in directing behaviour and attention. This difference is characteristic of the distinction between typical performance situations and optimal performance situations (Toplak et al., 2013). As typical performance situations are unconstrained by requirements to maximise performance, reflections on actions in such situations concern goal prioritisation and the extent to which behaviour requiring cognitive control typically adheres to goals and intentions. Optimal performance situations, conversely, are constrained by requiring participants to strive to maximise their task-performance within the bounds of the task, irrespective of their own goals.

Notwithstanding these distinctions, there is much debate on the extent to which the two approaches assess cognitive control. Barkley and Fischer (2011), for instance, claim that self-reports offer increased ecological validity. Conversely, Snyder et al. (2015) argue that such measures are problematic because they are influenced by contextual factors. Lin (2009, p. 15521), however, notes that “what happens in lab experiments does not often represent a complete picture of what happens in real life [...] the distractions in experiments are not necessarily distractions in real life.” Irrespective of their validity as measures of cognitive control at a functional level, self-reports can capture the extent to which individuals perceived their behaviour to typically adhere to goals, and provide an indication of reflections on combined everyday executive functioning.

Given the results of the analysis, it appears that effects cannot solely be attributed to functional associations. Rather, the role of goals, context, habits, individual differences, stimuli salience, and perceptions of task primacy all bare on possible associations between media multitasking behaviour and cognitive functioning. Previously, researchers have argued that media multitasking occurs due to goal-conflict (van Koningsbruggen et al., 2018). The outcomes of this analysis may point to the proposition that, among other unmeasured factors, goals might not only account for media multitasking itself but, at least in part, for the effects thereof. This would seem to support the strategic hypothesis put forward by Ralph et al. (2015). Differences in media multitasking levels and effects may be indicative of general behavioural strategies or patterns.

Recommendations for Future Research

Determine Sources of Heterogeneity in Outcomes

Many of the analyses were shown to be particularly heterogeneous. In-line with the differential susceptibility to media effects model (Valkenburg & Peter, 2013), further research is required to determine the sources of heterogeneity in outcomes and investigate if various individual, technical, or contextual differences might explain or moderate associations between media multitasking and cognitive control. These could include, for instance: age (Voorveld & van der Goot, 2013), polychronicity (Magen, 2017), self-regulation (Szumowska et al., 2018), and sensation seeking (Sanbonmatsu et al., 2013), among other factors. In the present study, measurement approach and cognitive control function were assessed as moderators. A post hoc analysis of the moderating role played by age category indicated the potential for a difference. While further investigation is needed to follow up this outcome, no effect of media multitasking measurement form was found. Notably, however, the present study only considered studies using self-reported measures of trait media multitasking that resemble the original MMI measure.

Interpret the Differences Between Measurement Approaches

As this meta-analysis has shown, there are stark differences between studies adopting performance-based assessments and those adopting self-report assessments. There is a need to understand, in the context of media multitasking, why such differences have emerged and what they mean for our theoretical understanding of media multitasking and its possible outcomes. As noted previously, a number of potential explanations for these differences have been provided. Further investigation of which of these explanations are indeed valid is required. In this regard, investigations should focus on media multitasking outside of a laboratory context and endeavour to understand the role of goals, context, individual differences, and stimuli salience among other factors. While some proportion of the differential outcomes may occur due to methodological artefacts, we need to first determine if other factors account for the discrepancy and, furthermore, work to understand the mechanisms through which media multitasking might affect cognitive control at either of the two levels of assessment.

Address Causality and Theoretical Mechanisms

All included assessments were based on cross-sectional designs. While this is certainly a function of the eligibility criteria, it is also a reflection of how research in this domain is typically conducted. With few exceptions (Baumgartner et al., 2017; Ralph et al., 2018; Szumowska et al., 2018) research in this domain is cross-sectional. As such, the available evidence cannot be used to determine if media multitasking precedes, is an outcome of, or interacts reciprocally with cognitive control. As noted previously, there exists a variety of hypotheses potentially accounting for the observed associations. To test these hypotheses, researchers need to focus more explicitly on developing theoretical accounts of the mechanisms underlying observed associations and, building on this, employ designs that will enable greater insight into the factors that may drive media multitasking and related effects. In particular, it is necessary to conduct appropriately powered experiments which will shed light on whether increased media multitasking does indeed cause changes in cognitive control, or if observed cross-sectional performance-decrements are due to other factors.

Conclusion

Notwithstanding the value of the study findings for research in this domain, there exist several important limitations which require attention. First, the review explicitly targeted studies adopting a cross-sectional design. Therefore, any inferences about causality are limited. Second, there exist varying conceptions for how to group and categorise the executive functions. While some previous reviews have used different groupings (Wiradhany & Koerts, 2021; Wiradhany & Nieuwenstein, 2017), we based our categorisations on those of Uncapher and Wagner (2018). Consequently, given different categorisations, the results may differ. A third limitation relates to the outcome metrics selected and included in the analysis for performance-based assays. Noting that most tasks produce many metrics that can be interpreted in relation to various outcome standards, we chose to base our selection of metrics on those included in Wiradhany and Nieuwenstein (2017) based on the justification that these metrics are those most relevant to distractibility. Additionally, given the needs of the analysis, we elected to not include any interaction effects. It is conceivable that other metrics could reasonably have been selected and included in the meta-analysis and that this would likely have an effect on the outcomes. Finally, in the hypotheses posed, an implicit direction of association was adopted (i.e., frequent chronic media multitasking causing observed deficits in cognitive control). However, it is acknowledged that, other factors might account for this relationship and, in particular, differences in cognitive control might explain media multitasking levels. More research is required to not only address this limitation but, more broadly, to understand the factors that cause media multitasking, patterns of media multitasking behaviour itself, the outcomes (both positive and negative) of media multitasking and, if necessary, remediation approaches for these outcomes.

Ten years on from Ophir et al. (2009) the picture is not any more transparent. Based on the papers reviewed in this study we are no closer to understanding ‘cognitive control in media multitaskers’. While subsequent research has, in some instances, replicated Ophir et al. (2009), overall, at least in the methods most closely resembling the original study, the results have not stood the test of time. However, complicating matters, studies adopting a different measurement approach than this first investigation have consistently produced results supporting the claim that media multitasking is negatively associated with everyday executive functioning. Given the importance of understanding media multitasking and its effects on our attentional functioning, there is a need for further study.

Footnotes

¹ Acknowledging the first use of the term trait media multitasking by Ophir et al. (2009) and its subsequent use in this domain, we use the term to refer to media multitasking tendencies over time (i.e., an individual’s general tendency to media multitask) as opposed to state media multitasking which refers to media multitasking in a particular situation or moment in time.

² Noting Webster and Watson (2002), forward searches involve identifying articles that cite already included records, while backwards searches involve working through the reference lists of the articles already included. While the latter can be conducted with the full texts, the former was conducted using the Google Scholar search engine.

³ We use the term record to denote a published/unpublished work, a study to refer to an investigation on a sample that does not overlap with other samples, and an assessment to refer to a single comparison between media multitasking and a relevant outcome of interest.

Appendix

Electronic Database Search Strings

Web of Science

TS=((media OR computer OR phone) AND (multitask* OR switching OR task-switch*) AND (cognit* OR attention* OR distract* OR "cognitive control" OR "executive function*" OR focus*)) AND PY =2009 -2019

Results: n = 2123 (26-07-2019)

Scopus

TITLE-ABS-KEY ((media OR computer OR phone) AND (multitask* OR switching OR task-switch*) AND (cognit* OR attention* OR distract* OR "cognitive control" OR "executive function*" OR focus*)) AND PUBYEAR > 2008

Results: n = 3798 (26-07-2019)

Pubmed

((((media[Title/Abstract] OR computer[Title/Abstract] OR phone [Title/Abstract])) AND (multitask*[Title/Abstract] OR switching[Title/Abstract] OR task-switch[Title/Abstract])) AND (cognit* OR attention* OR distract* OR "cognitive control" OR "executive function*" OR focus*)) AND ("2009/07/01"[Date - Publication] : "3000"[Date - Publication])

Results: n = 196 (26-07-2019)

PsycARTICLES

Abstract: media OR Abstract: computer OR Abstract: phone AND Abstract: multitask* OR Abstract: switching OR Abstract: task-switch AND Abstract: cognit* OR Abstract: attention* OR Abstract: distract* OR Abstract: "cognitive control" OR Abstract: "executive function*" OR Abstract: focus* AND Year: 2009 To 2019

Results: n = 14 (26-07-2019)

Additional Scopus Search:

TITLE-ABS-KEY (( media  OR  computer  OR  phone  OR  television )  AND  ( multitask*  OR  switchig  OR  task-switch* )  AND  ( cognit*  OR  attention*  OR  distract*  OR  "cognitive control"  OR  "executive function*"  OR  focus* ) )  AND  PUBYEAR  >  2008 )  AND NOT  ( TITLE-ABS-KEY ( ( media  OR  computer  OR  phone )  AND  ( multitask*  OR  switching  OR  task-switch* )  AND  ( cognit*  OR  attention*  OR  distract*  OR  "cognitive control"  OR  "executive function*"  OR  focus* ))  AND  PUBYEAR  >  2008  AND  PUBYEAR  <  2020 

Results: n = 338 (11-10-2020)