Skip to main content
SearchLoginLogin or Signup

A Multiverse Analysis of the Associations Between Internet Use and Well-Being

Volume 5, Issue 2. DOI: 10.1037/tmb0000127

Published onMay 13, 2024
A Multiverse Analysis of the Associations Between Internet Use and Well-Being
·

Abstract

Internet technologies’ and platforms’ potential psychological consequences remain debated. While these technologies have spurred new forms of commerce, education, and leisure, many are worried that they might negatively affect individuals by, for example, displacing time spent on other healthy activities. Relevant findings to date have been inconclusive and of limited geographic and demographic scope. We examined whether having (mobile) internet access or actively using the internet predicted eight well-being outcomes from 2006 to 2021 among 2,414,294 individuals across 168 countries. We first queried the extent to which well-being varied as a function of internet connectivity. Then, we examined these associations’ robustness in a multiverse of 33,792 analysis specifications. Of these, 84.9% resulted in positive and statistically significant associations between internet connectivity and well-being. These results indicate that internet access and use predict well-being positively and independently from a set of plausible alternatives.

Keywords: well-being, internet technology, technology effects

Funding: Andrew K. Przybylski’s research is currently supported by the Huo Family Foundation and the Economic and Social Research Council (ESRC; ES/S00324X/1) and was recently supported by the ESRC (ES/T008709/1). In the preceding 5 years, Andrew K. Przybylski has also worked on research grants provided by the John Fell Fund, the Diana Award, and the children’s charity Barnardo’s. These research grants were paid to Andrew K. Przybylski’s employer, the Oxford Internet Institute. During this period, Andrew K. Przybylski has also engaged in unpaid consultations with several organizations, including United Nations Children’s Fund, the Organisation for Economic Co-operation and Development, Meta Inc., UK Interactive Entertainment, U.K. Research and Innovation, the U.K.’s Department for Culture, Media and Sport, the Office of the U.K.’s Chief Medical Officer, the Office of the U.S. Surgeon General, the U.K.’s Academy of Medical Sciences, and the U.K. Parliament. There were no financial products or benefits resulting from these consultations. The funders and these organizations had no role in study design, data collection and analysis, decision to publish, or preparation of the article. Matti Vuorre’s research is currently supported by the ESRC (ES/W012626/1) and the John Fell Fund and was recently supported by the Huo Family Foundation (awarded to Andrew K. Przybylski).

Disclosures: The authors report no conflicts of interest.

Author Contributions: Matti Vuorre contributed in conceptualization, data curation, formal analysis, investigation, methodology, project administration, software, visualization, writing–original draft, and writing–review and editing. Andrew K. Przybylski contributed in conceptualization, funding acquisition, investigation, methodology, project administration, resources, supervision, visualization, writing–original draft, and writing–review and editing.

Open Access License: This work is licensed under a Creative Commons Attribution 4.0 International License (CC BY 4.0; http://creativecommons.org/ licenses/by/4.0). This license permits copying and redistributing the work in any medium or format, as well as adapting the material for any purpose, even commercially.

Correspondence concerning this article should be addressed to Matti Vuorre, Tilburg School of Social and Behavioral Sciences, Tilburg University, 5037 DB Tilburg, Netherlands, or Andrew K. Przybylski, Oxford Internet Institute, University of Oxford, Oxford OX1 3JS, United Kingdom. Email: [email protected] or [email protected]


Video Abstract


The increasing adoption and use of internet-enabled technologies and platforms has spurred debate about their potential effects on people’s psychological well-being and functioning. Social scientists have shifted their focus from other topics such as violent video games and television-based technologies to new and emerging platforms and handheld digital devices (Orben, 2020). Large technology firms such as Meta (2022), Google (2022), Apple (2018), and TikTok (2021) have reacted to concerns and released a host of “digital well-being” tools, such as applications and notifications that make it easier for users to learn how much time they spend on a given platform or with a given technology. At the same time, health professionals (Office of the Surgeon General, 2021) and officials in many countries are working to enact new regulations (Department for Culture, Media, and Sport, 2022) to ensure internet and technology platforms protect user well-being. However, even after considerable scientific attention, an understanding of the fundamental associations between internet technology adoption and use and well-being remains elusive, and results of scientific studies on this topic are decidedly mixed (Appel et al., 2020; Best et al., 2014; Dickson et al., 2019).

The rise in tools, advice, and regulation aimed at addressing well-being is interesting, in part, because of how the studies informing this debate are done. For example, despite the fact that the challenge is often framed as a worldwide issue, the geographic and demographic scope of the evidence base is narrow and not well mapped onto worldwide trends (Ghai et al., 2023). In the past decades, the expansion of access to the internet has accelerated the most (International Telecommunication Union, 2021) in places where social scientists are the least likely to study their effects (Ghai et al., 2022, 2023). Similarly, while studying technology adoption and well-being across countries requires frequent and high-quality measures of both factors, few, if any, investigations successfully combine the two (see, e.g., Vuorre & Przybylski, 2023). Measurement quality and consistency vary significantly, and the most widely cited international studies either lack a longitudinal component (Byrne et al., 2016) or have long 3-year (Organisation for Economic Co-operation and Development, 2018) or 4-year (Inchley et al., 2020) intervals between data collections. Finally, because most of the debate surrounding the global impact of internet technologies is focused on younger people, little, if any, global data reflect associations between technology and well-being across the life course. This important lack of context means that it remains an open question of who, where, and when internet technologies and connectivity might be influencing people’s well-being. Without knowing this, it is impossible to deploy limited resources to capitalize on benefits or redress harms. To our knowledge, no research has directly grappled with these issues and addressed the worldwide scope of the debate.

Considering this impasse, our overarching research goal was to estimate the extent to which internet access, mobile internet access, and active internet use predicts psychological well-being on a global level. To this end, we analyzed data from a series of cross-sectional samples of 2,414,294 individuals from 168 countries from 2006 to 2021. We studied eight indicators of well-being: life satisfaction, the extent to which individuals reported experiencing daily negative and positive experiences; two indices of social well-being; physical well-being, community well-being, and experiences of purpose.

Because of the large number of predictors, outcomes, subgroups to analyze, and potentially important covariates that might theoretically explain observed associations, we sought out a method of analysis to transparently present all the analytical choices we made and the uncertainty in the resulting analyses. Multiverse analysis (Steegen et al., 2016) was initially proposed to examine and transparently present variability in findings across heterogeneous ways of treating data before modeling them (see also Simonsohn et al., 2020). We therefore conducted a series of multiverse analyses where we repeatedly fitted a similar model to potentially different subgroups of the data using potentially different predictors, outcomes, and covariates.

Our current research questions, then, were as follows: (a) To what extent does well-being differ between individuals who report having access to, or using, (mobile) internet? and (b) How robust are these differences in well-being across different internet adoption predictors, well-being outcomes, subgroups, and model covariate specifications?

Method

We analyzed data from the Gallup World Poll (GWP; Gallup, 2022). GWP is a nationally representative (of each country’s civilian, noninstitutionalized, adult [15+] population) continuous annual panel survey of approximately 1,000 individuals from each of 168 countries, conducted from 2005 to 2022. Gallup conducts the surveys through 1-hr interviews, either face-to-face or via telephone, with questionnaires that are translated into the major conversational language of each country. See Gallup (2022) for the GWP methodological details. We used data from 2006 to 2021 because, in 2005, the internet questions were answered by fewer than a quarter of respondents, and the 2022 data only contained 5,000 responses from three countries. We refer to the 2006–2021 data set below. The total sample size for this analysis was 2,414,294, with 168 countries. The sample was 53.1% female, and the interquartile range of age was 26–54. This study and methods therein were approved by the University of Oxford Central University Research Ethics Committee (SSH_OII _CIA_21_084).

Internet Access and Use

The key variables that we considered as predictors measured the respondents’ access and use of the internet. GWP has surveyed these with four items, with varying coverage over time. First, GWP measured internet access with “Does your home have access to the internet?” from 2006 to 2015, where the mean percentage of nonmissing values was 96.9%. From 2016 to 2021, a similar question measured overall internet access with “Do you have access to the internet in any way, whether on a mobile phone, a computer, or some other device?” (mean valid responses: 99.3%). Because these two questions were so similar, and to extend the range of data coverage, we combined these two into one variable that indicated internet access.

Second, GWP asked about mobile internet access with “Can your mobile phone be used to access the internet?” from 2017 to 2021 (mean valid responses: 86.5%). Third, GWP measured internet use with “Have you used the internet in the past seven days, whether on a mobile phone, a computer, or some other device?” from 2015 to 2021 (mean valid responses: 64.5%). All of these items had binary “yes”/“no” response options, and respondents were given the option to decline an answer or report that they did not know. We coded the latter two as missing values. We show summaries of these measures in Table A1.

Well-Being Outcomes

We focused on eight measures of well-being, broadly defined in the GWP: life satisfaction, negative and positive experiences, and social life satisfaction. In addition to these four outcomes, which were included in the GWP from 2006 to 2021, we studied four indicators from the Gallup-Sharecare Global Well-being Index (GWBI), which is a “barometer of individuals perceptions of their own well-being” (Gallup, 2022, p. 68).

Life satisfaction was measured with a single item:

“Please imagine a ladder, with steps numbered from 0 at the bottom to 10 at the top. The top of the ladder represents the best possible life for you and the bottom of the ladder represents the worst possible life for you. On which step of the ladder would you say you personally feel you stand at this time? ”

and respondents chose a number from 0 to 10. We rescaled this item to range from 0 to 1.

Negative and positive experiences are both measured through a set of five yes/no items. For negative experiences, respondents answered the prompts, “Did you experience the following feelings during a lot of the day yesterday? How about physical pain/worry/sadness/stress/anger?” For positive experiences, the items were “Did you feel well-rested yesterday?”; “Were you treated with respect all day yesterday?”; “Did you smile or laugh a lot yesterday?”; “Did you learn or do something interesting yesterday?”; and “Did you experience the following feelings during a lot of the day yesterday? How about enjoyment?”

In addition, we included two variables intended to measure individuals’ “social support structure and opportunities to make friends” (“Someone in your life always encourages you to be healthy” and “Your friends and family give you positive energy every day”). We included these items as a proxy for social well-being (hereafter, “social life”). While we recognize their potentially limited validity, we thought that, from the variables available, they best approximated an absence of loneliness, an important and much-studied aspect of well-being in connection to digital technologies.

In 2013 through 2015, the GWP database also included a (Gallup-Sharecare Global Well-Being Index) GWBI, which measured (among others) experiences of purpose (“liking what one does each day and being motivated to achieve one’s goals”), community well-being (“liking where one lives, feeling safe and having pride in one’s community”), physical well-being (“having good health and enough energy to get things done daily”), and social well-being (“having supportive relationships and love in your life”). Experiences of purpose were measured with prompts “You like what you do every day” and “You learn or do something interesting every day.” Community well-being was prompted with “The city or area where you live is a perfect place for you” and “In the last 12 months, you have received recognition for helping to improve the city or area where you live.” Physical well-being had prompts “In the last seven days, you have felt active and productive every day” and “Your physical health is near-perfect.” Social well-being was measured with the same items as the above “social life” index, but instead on a rating scale: All GWBI items included the prompt:

“Thinking about your life in general, please rate your level of agreement with each of the following using a five-point scale, where 5 means you STRONGLY AGREE and 1 means you STRONGLY DISAGREE. You may choose any of the numbers 1, 2, 3, 4, or 5. ”

While these scales are not psychometrically validated, Gallup cites prominent scientists as having helped with their development (Gallup, 2022, p. 5). While this statement does not make up for these scales’ lack of validation, we believe that the extensive scope of the data set, across both time and countries, makes them uniquely valuable objects of study.

Covariates

In addressing our second research question, our aim was to approach the independent contributions of internet access and use on well-being. To that end, we adjusted for plausible (and available) covariates in our models that might otherwise mask or bias any independent contributions of internet access on well-being or create spurious associations. We chose six variables to represent such potentially confounding factors that have been previously considered important in the literature on well-being (Diener et al., 2018): the respondent’s income (e.g., Lucas & Schimmack, 2009), educational (Lawless & Lucas, 2011), work (Luhmann et al., 2015), and relationship statuses (Diener & Seligman, 2002), their ability to meet basic needs for food and shelter, and whether or not they reported having health problems (Lawless & Lucas, 2011).

The GWP reports respondents’ monthly household income in their local currencies. GWP used hot deck imputation for individuals who reported an income range (∼15%) or who did not provide responses (∼15%). GWP then converted those values to international dollars using the World Bank’s purchasing power parity conversion factor, with the intent of making income estimates comparable across all respondents and countries. In our analyses, we log-transformed income. The GWP coded each respondent’s educational status to one of three categories: elementary education or less (up to 8 years of education), some secondary education (9–15 years), and tertiary (education beyond high school). We used this variable as a continuous predictor. Since 2009, GWP measured work status with five levels related to the quantity of work, but we recoded this as a binary employed versus not variable. Relationship status was measured with a question about marital status with six response options (single/never been married, married, separated, divorced, widowed, domestic partner). We recoded this to a binary in relationship versus not variable to reduce computational complexity, to not inflate the number of multiverse specifications with very similar specifications, and because there is no a priori reason to expect that most of the categories would differ. Basic need satisfaction was measured with two items querying whether the respondent had difficulties in providing for their food and shelter in the past 12 months. Respondents also answered, “Do you have any health problems that prevent you from doing any of the things people your age normally can do?” with a binary yes/no response format.

In addition to these covariates, we identified meaningful subgroups in the data. Previous research has highlighted important differences between age groups and sexes in their levels of well-being and use of internet technologies and associations between the two (Kelly et al., 2018; Kreski et al., 2021). We therefore conducted our analyses separately for each sex and age group. However, to reduce the computational complexity of our analyses and because there are no strong a priori reasons to assume large differences between adjacent ages, we split the continuous age variable into six categories (15–24, 25–34, 35–44, 45–54, 55–64, and 65+).

Data Analysis

Our general data-analytic approach was a regression model predicting one outcome (e.g., life satisfaction) from an intercept and one predictor (e.g., internet access). Where the outcome consisted of multiple items, we took the mean. In addition, we within-country centered all our predictors so as not to include between-country differences in the coefficients, which consequently indicated contrasts between individuals who (e.g.) had access to the internet and those who did not within a given country. In addition, because the data were nested within countries, we specified a multilevel model where the intercept and coefficient of the internet predictor varied randomly over countries. To be conservative, we also added random intercepts and slopes over years and the country-by-year interaction because any associations might be heterogeneous over time (Vuorre et al., 2021). We addressed our first research question about average contrasts with a multilevel model of well-being outcome y for observation i, country j, year k, and year by country l, on internet predictor x, specified as

yijklNormal(μijkl,σ2),μijkl=α0+β0j+γ0k+δ0l+(α1+β1j+γ1k+δ1l)(xijklx.jkl),βNormal([00],Σβ),γNormal([00],Σγ),δNormal([00],Σδ).y_{ijkl} \sim \text{Normal}(\mu_{ijkl},\,\sigma^{2}),\\ \mu_{ijkl} = \alpha_{0} + \beta_{0j} + \gamma_{0k} + \delta_{0l} \\ + (\alpha_{1} + \beta_{1j} + \gamma_{1k} + \delta_{1l})(x_{ijkl} - {\overline{x}}_{.jkl}), \\ \beta \sim \text{Normal}\left( \begin{bmatrix} 0 \\ 0 \\ \end{bmatrix},\,\Sigma_{\beta} \right),\\ \gamma \sim \text{Normal}\left( \begin{bmatrix} 0 \\ 0 \\ \end{bmatrix},\,\Sigma_{\gamma} \right), \\ \delta \sim \text{Normal}\left( \begin{bmatrix} 0 \\ 0 \\ \end{bmatrix},\,\Sigma_{\delta} \right).

In this manner, α1 is the difference in well-being indicator y between individuals who had access to the internet and those who did not, for the average country and year.

For our second, primary, research question, we then conducted a multiverse analysis over different covariate permutations on this base model and subgroups in the data. We analyzed all the possible ways in which the covariates could be included in this model (including no covariates) as fixed effects, leading to 64 different covariate specifications. In addition to model specifications and subgroups, GWP recommends using model weights to adjust for demographic representativeness in some analyses, and so we conducted the analyses both with and without model weights. Our multiverse therefore consisted of all the distinct combinations of outcomes, predictors, age, sex, covariate combinations, and whether model weights were included or not, leading to 33,792 specifications. We used R (R Core Team, 2022) for analyses and the lme4 package for estimating the multilevel models (Bates et al., 2015).

Data Availability Statement

We used the proprietary GWP data set, which is available to subscribing institutions through the Gallup website. All our code and a simulated GWP data set are available at https://doi.org/10.5281/zenodo.7774923. We did not prespecify the sample sizes, but instead used all the available data in the Gallup database (with the exclusions detailed below). This study was not preregistered.

Results

Our first research question concerned the average differences in well-being between individuals who had access to (mobile) internet or had used the internet in the past 7 days and those who did not. We display the results of this analysis in Figure 1. The associations between internet access and well-being were consistently positive. For the average country, individuals who had access to the internet reported on average approximately 0.08 units greater life satisfaction, positive experiences, and social life satisfaction and 0.06 units lower negative experiences than individuals who did not have access. Results regarding the more temporally restricted (2013–2015) GWBI outcomes portrayed a similar picture: Individuals with internet access reported approximately 0.08 units greater experiences of purpose, 0.1 unit greater physical, 0.02 units greater community, and 0.08 units greater social well-being than individuals without access.

Figure 1

Scatterplots of Well-Being Means (Y-Axis) for Individuals Who Responded No or Yes to the Internet Adoption Question (X-Axis) and Their Differences (Y–N)
Note. Columns indicate different well-being outcomes, and rows are different internet adoption measures (overall internet access, internet use, and mobile phone internet access). Empty points are countries’ means, solid points and lines are means of the countries’ means. Numbers in each panel’s left bottom corner indicate the regression coefficient (yes–no) and its 95% confidence interval. The GWBI outcomes did not overlap in time with mobile internet data. GWBI = Global Well-being Index; Y = yes; N = no.

Results across the other two internet access and use predictors were of the same sign and comparable magnitude. Being an active internet user was associated with 0.03–0.08 unit increases in life satisfaction, positive experiences, social well-being, and physical well-being, and with a 0.04 unit decrease in negative experiences. The overall association between being an active internet user and community well-being was not significantly different from zero. Mobile phone internet access predicted increases between 0.06 and 0.07 units.

The estimates discussed above and in Figure 1 refer to percentages of the scale: To put these magnitudes to another context, the well-being outcomes’ standard deviations across individuals, countries, and time ranged from 0.24 to 0.33. In standardized terms, the observed differences were therefore quite small (e.g., the median life satisfaction difference was 0.36 SDs between individuals who had access to the internet and those who did not) but not negligible.

Moreover, Figure 1 shows individual countries’ observed difference scores as small points; they were very consistently in the same direction as the average contrasts, indicating that this difference held similarly across most countries. These results showed uniformly across the eight well-being outcomes that individuals who had access to or actively used the internet reported meaningfully greater well-being than those who did not.

However, although the estimates in Figure 1 were surprisingly uniform across outcomes and predictors, they did not yet address our second main research question: The robustness of this association across different analysis specifications and subgroups and the extent to which internet access and use might independently predict individuals’ well-being. To best answer this question while at the same time appreciate the theoretical uncertainty in how, where, or for whom to study this association, we next turned to our main multiverse analysis.

We summarize the results of this multiverse analysis in Table 1. First, for life satisfaction, the median sample size across specifications was 59,606 individuals. Answering “yes” to an internet access or use question was associated with a median 0.04 unit increase in life satisfaction. The central 50% of the distribution of associations was within the 0.03–0.05 interval. The association between internet access or use and life satisfaction was positive in 96.38% of model specifications. Numerical results for the other well-being outcomes were of similar magnitude, and the total proportion of specifications that resulted in negative associations was only 0.45%.

Table 1

Summary of Multiverse Analysis of Associations Between Internet Access (or Use) and Well-Being

Association magnitude

Proportion of significant association

Outcome

K

N

Mdn

IQR

Negative

Not significant

Positive

Life satisfaction

4,608

59,606

0.04

[0.03, 0.05]

0.0%

3.6%

96.4%

Negative experiences

4,608

59,664

−0.02

[−0.03, −0.02]

86.7%

13.3%

0.0%

Positive experiences

4,608

59,588

0.05

[0.04, 0.05]

0.0%

0.0%

100.0%

Social life

4,608

60,131

0.04

[0.03, 0.06]

0.0%

3.0%

97.0%

GWBI purpose

4,608

28,102

0.06

[0.04, 0.07]

0.0%

19.0%

81.0%

GWBI physical

4,608

28,155

0.04

[0.03, 0.05]

0.0%

14.0%

86.0%

GWBI community

3,072

14,056

0.02

[0.01, 0.02]

4.9%

42.6%

52.5%

GWBI social

3,072

14,048

0.05

[0.02, 0.06]

0.0%

39.6%

60.4%

Total

33,792

40,478

0.04

[0.02, 0.05]

0.4%

14.7%

84.9%

Note.K is the number of specifications, and N is the median sample size across specifications. “Total” indicates quantities across well-being outcomes (we reversed associations with negative experiences for this aggregate number). IQR = interquartile range; GWBI = Global Well-being Index.

Figure 2 shows that across all model specifications, the multilevel model’s estimated association between internet access and use for the average country was very consistently positive. Each estimated association in Figure 2 identifies a model specification (e.g., the coefficient for internet access for 15–24-year-old males, adjusting for health problems and income, estimated with weights), which are ordered on increasing association magnitude.

Figure 2

Multiverse Analysis Results of Associations Between Three Internet Use Predictors and Eight Well-Being Outcomes
Note. The estimated associations are ordered by increasing magnitude; the x-axis denotes the ordered number of (Predictor × Age × Sex × Covariate Combination × Weights) specification. The small shaded areas to the right of each panel show the kernel density estimates of the associations’ point estimates and the median association with a colored point. CI = confidence interval.

Whereas Figure 1 showed that the overall average associations were consistent across internet adoption predictors and well-being outcomes, Figure 2 highlights a surprising consistency in that association across demographic subgroups and, moreover, in model covariate specifications. If these associations were spurious associations caused by any of the covariates we considered, we should observe clusters of nonsignificant or reversed estimates for specific covariate sets. This was mostly not the case: Figures A1 and A2 show additional “dashboards” that indicate the covariate specifications for each estimate in Figure 2, and we did not discern any clear patterns among them with respect to the covariate specifications.

However, Figure 2 shows a notable group of negative associations between community well-being and internet adoption among otherwise mostly positive relationships. To examine these differences further, Figure 3 focuses on variability in the associations across demographics and covariate specifications. The association between community well-being and internet adoption was particular to active internet use (rather than access) and individuals in the youngest 15–24 age category.

Figure 3

Associations Between the Internet Predictors and Well-Being Outcomes by Sex and Age, Ordered by Covariate Specification
Note. The specifications are ordered such that greater numbers indicate more covariates in the model. Estimates are means across weighted and nonweighted models.

More generally, Figure 3 also shows that while increasing the number of covariates in the model did tend to decrease the magnitudes of the estimated associations, they typically remained positive. This observation shows that even after adjusting for all possible combinations of the covariates that we considered, the relationships between internet access or use and well-being remained positive. In turn, this suggests that the contributions of internet access and use on well-being were independent of the covariates we selected for, and thus might indicate causal relations. (Although we highlight the evidence here for causal claims is less than thin.)

Discussion

The debate over internet platforms’ and technologies’ effects on individuals’ psychological well-being remains a central topic because of their potentially global consequences. While past results on this topic have been mixed, the overwhelming majority of studies have examined convenience samples from the global north, thereby ignoring the fact that the penetration of the internet has been and continues to be a global phenomenon. In this study, we examined associations between internet use and access and a broad variety of well-being indicators in a representative sample of 2,414,294 individuals across 168 countries within an age range that spanned from late adolescence to older adults. We found that on average across countries and demographics, individuals who had internet access, mobile internet access, or actively used the internet reported greater levels of life satisfaction, positive experiences, experiences of purpose, and physical, community, and social well-being, and lower levels of negative experiences.

The main thrust of our analysis was to examine to what extent this positive association between internet adoption and well-being was sensitive to different demographic groups and modeling decisions. Furthermore, we attempted to estimate the unique contributions of internet on well-being by examining this association across 64 different permutations of sets of plausible covariates that might otherwise create or mask associations. Across 33,792 of such model specifications, 84.9% of the associations were significantly positive, and only 0.4% were negative.

We did, however, observe a notable group of negative associations between internet use and community well-being. These negative associations were specific to young (15–24-year-old) women’s reports of community well-being. They occurred across the full spectrum of covariate specifications and were thereby not likely driven by a particular model specification. Although not an identified causal relation, this finding is concordant with previous reports of increased cyberbullying (Przybylski & Bowes, 2017) and more negative associations between social media use and depressive symptoms (Kelly et al., 2018; but see Kreski et al., 2021). Further research should investigate whether low community well-being drives engagement with the internet or vice versa.

Nevertheless, our conclusions are qualified by a number of factors. First, we compared individuals to each other. There are likely myriad other features of the human condition that are associated with both the uptake of internet technologies and well-being in such a manner that they might cause spurious associations or mask true associations. For example, because a certain level of income is required to access the internet and income itself is associated with well-being, any simple association between internet use and well-being should account for potential differences in income levels. While we attempted to adjust for such features by including various covariates in our models, the data and theory to guide model selection were both limited.

Second, while between-person data such as we studied can inform inferences about average causal effects, longitudinal studies that track individuals and their internet use over time would be more informative in understanding the contexts of how and why an individual might be affected by internet technologies and platforms (Rohrer & Murayama, 2021).

Third, while the constructs that we studied represent the general gamut of well-being outcomes that are typically studied in connection to digital media and technology, they do not capture everything, nor are they standard and methodically validated measures otherwise found in the psychological literature. That is, the GWP data that we used represent a uniquely valuable resource in terms of its scope both over time and space. But the measurement quality of its items and scales might not be sufficient to capture the targeted constructs in the detailed manner that we would hope for. It is therefore possible that there are other features of well-being that are differently affected by internet technologies and that our estimates might be noisier than would be found using psychometrically validated instruments. Future work in this area would do well in adopting a set of common validated measures of well-being (Elson et al., 2023).

Fourth, the validity of self-reported measures of technology engagement is found lacking, as self-reported quantities of use correlate only modestly with actual use, as measured, for example, by apps installed on smartphones (Parry et al., 2021; Wu-Ouyang & Chan, 2022). In our study, we used reports of whether an individual has access to or has used the internet in the past week, which may suffer from these biases. Nevertheless, we believe it is more difficult to be mistaken in answering those questions than, for example, questions about average hours used in the past year.

Finally, we further highlight the tentative-at-most nature of our results with respect to causal effects of internet access and use on individuals’ well-being. Causal inference from observational data, such as that studied here, is notoriously difficult (Rohrer, 2018; VanderWeele et al., 2016). Critical theoretical assumptions must be made and properly applied in the statistical models in order to approach unbiased causal estimates, steps that we did not take in the current work. We nevertheless remain hopeful that the clarity with which we hoped to address this issue will provide a solid foundation for future work on internet technologies’ causal effects (Grosz et al., 2020).

To overcome these limitations, we think the best way forward for this field is to expend more resources on collecting larger and more representative longitudinal data sets that include validated measurements of the constructs that researchers care about. In addition, these data sets should include accurate data on individuals’ engagement with internet technologies in lieu of self-reports. Fortunately, both of these data are already collected; large cohort-based surveys in many countries already track individuals’ psychological states over time, and internet platforms are infamous for collecting detailed data on their users’ behaviors. A significant but potentially fruitful challenge then would be to marry those two streams of data and use them in transparent and independent scientific inquiry for a more detailed understanding of internet technologies in individuals’ lives.

Appendix

Supplementary Results

We show the total sample percentages answering “yes” to the three internet adoption metrics in Table A1. Table A2 shows the Pearson correlations between all variables across demographics, time, and space. Figures A1 and A2 show the full specification curve analysis plots, including the “dashboards” in panels B.

Table A1
Total Percentages Answering “Yes” to the Three Internet Adoption Measures

Year

Internet access

Internet use

Mobile access

2006

20.5%

2007

23.8%

2008

28.0%

2009

28.9%

2010

35.8%

2011

40.3%

2012

40.6%

2013

41.9%

2014

52.7%

2015

49.3%

85.6%

2016

55.7%

89.2%

2017

55.5%

89.2%

63.9%

2018

60.1%

90.1%

69.2%

2019

62.6%

90.5%

72.0%

2020

82.3%

94.5%

86.4%

2021

73.7%

93.4%

81.2%

Table A23
Pearson Correlations Between Key Variables

Variable

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

1. Internet access

2. Internet past 7 days

.02

3. Internet phone

.66

.28

4. Income

.19

.07

.16

5. Health problems

−.15

−.11

−.21

−.11

6. Relationship

−.03

−.05

−.06

.06

.01

7. Food shelter

.07

.04

.09

.15

−.13

−.01

8. Life satisfaction

.15

.05

.12

.15

−.13

.00

.18

9. Negative experiences

−.05

−.04

−.09

−.10

.23

.01

−.22

−.21

10. Positive experiences

.12

.06

.11

.10

−.14

−.01

.13

.26

−.38

11. Social life

.09

.04

.09

.10

−.09

−.02

.12

.24

−.17

.23

12. GWBI purpose

.14

.07

.12

−.15

.00

.15

.33

−.22

.39

.22

13. GWBI community

.02

−.00

.04

−.03

.04

.07

.16

−.12

.21

.16

.30

14. GWBI physical

.17

.12

.13

−.39

−.00

.15

.26

−.28

.34

.18

.44

.25

15. GWBI social

.13

.04

.11

−.11

.03

.11

.23

−.16

.28

.25

.42

.22

.36

Figure A1

Multiverse Analysis Results of Associations Between Three Internet Use Predictors and Four Main Well-Being Outcomes


Note. (A) Estimated associations, in order of increasing magnitude, are shown separately for each of the four main well-being outcomes. Points and intervals are point estimates and 95% CIs. The small shaded areas to the right of each panel show the kernel density estimates of the associations’ point estimates and the median association with a colored point. (B) A dashboard of model specifications. Each point in the small panels indicates the value of the grouping variable or covariate specification for the model indicated on the x-axis.

Figure A2

Multiverse Analysis Results of Associations Between Three Internet Use Predictors and Four GWBI Well-Being Outcomes


Note. (A) Estimated associations, in order of increasing magnitude, are shown separately for each of the four GWBI well-being outcomes. Points and intervals are point estimates and 95% CIs. The small shaded areas to the right of each panel show the kernel density estimates of the associations’ point estimates and the median association with a colored point. (B) A dashboard of model specifications. Each point in the small panels indicates the value of the grouping variable or covariate specification for the model indicated on the x-axis. GWBI = Global Well-being Index.


Received September 15, 2023
Revision received December 5, 2023
Accepted January 2, 2024
Comments
1
?
Jim Crum:

This is fascinating and well documented. Thank you for starting the process of teasing this apart with data.