Have few, if any, studies on the health effects of particulate air pollution measured the actual exposure to particulates?

Question

Air pollution is a major health concern worldwide. One of the major problems is the emissions caused by traffic such as nitrogen dioxide and the small particulates known as PM 2.5 (which are very small carbon particles associated with vehicle engines, especially diesels).

The UK, for example, recently estimated that the combined effect of vehicle emissions could be causing more than 30,000 early deaths per year (see this report, for example).

There have been some questions about the accuracy of some of these estimates (see this question Does particulate pollution in outdoor air kill tens of thousands every year? ).

But statistician William "Matt" Briggs suggests in this video on the ecological fallacy (see his specific comments using PM 2.5 studies to illustrate the fallacy at about 1:34 in the video) that essentially none of the major studies have actually measured exposure to PM 2.5s in the populations they study, instead they have all used proxies for that exposure (so we could be seeing effects that are ecological not actually caused by the pollution and so overestimating or underestimating their harms). See also his blog entry about the video which has some references such as this one, a major study on mortality vs PM 2.5s which he claims contains a double ecological fallacy with both the outcome and the exposure been measured by proxy models.

For those wo don't want to watch his video here is a quote from one of his blogs describing his interaction (presumably as a statistical advisor) with the California Air Resources board:

Yours Truly was involved in a critique of a study submitted to the California Air Resources Board (CARB) which claimed to have discovered a correlation between air pollution (X; particulates of a certain size) and heart disease (Y). A weak, barely there finding of statistical “significance” was enough to embolden CARB to create new and enhance old air pollution regulations in order to “save lives.” Yet X was never measured.

At a very few places, particulate measures were taken for a limited time. The air pollution in these places was then crudely extrapolated to areas in which it was not measured. Finally, the extrapolated air pollution nearest the address of the study participants (where they lived at one time, ignoring moves) was taken as the exposure; this was their W. Nobody knows how much air pollution anybody was actually exposed to.

The sequel to this story is fascinating. I submitted written critiques where they were discussed at a CARB meeting. One panel member thanked me, called me learned, and took my criticisms of the epidemiologist fallacy seriously. But it was judged that—and here you must laugh—because the fallacy was so common that it led to many results referenced by CARB, that this current study was no different. And therefore acceptable.

Given the perceived importance of air pollution to health it seems to be a fairly significant claim that we might not be measuring the true effects but a false ecological effect instead. This makes his claim an important one to test: Have few if any studies on the health effects of PM 2.5 particulates directly measured the exposure to the particles?

Clarification of the claim and the evidence required to refute it

It is clear from the comments that this question has been severely misinterpreted by many readers, often for reasons that have nothing to with the claim itself and everything to do with who made it. Briggs has heretical views on climate science (which seems to be the source of much commentator ire) but is also a professional statistician with published textbooks like this one, so dismissing his complaint because he doesn't know anything about the topic is an extremely weak skeptical response.

And a little clarification (from the Encyclopaedia of Epidemiology) of why the ecological fallacy matters in epidemiology (my emphasis):

Exposure assessment is a critical component of epidemiologic research, and unfortunately for many studies, it is this component that introduces many limitations. Exposure assessment has been defined by Last (2001) as the ‘process of estimating concentration or intensity, duration, and frequency of exposure to an agent that can affect health’. It involves preferably quantitative, but often qualitative, procedures to estimate and assign an individual’s past or current exposures...

...Ecological fallacy: "The ecological fallacies in the three examples above arise from assuming that all individuals in each ecological group have the same summary measure (the mean value) of the group without accounting for possible confounding by other variables and for the unobserved heterogeneity of individuals in each group.

In view of this, when statistically significant association is found between exposure and health outcome at group level (usually aggregate data are easily assessable as they already exist, having been previously collected for other purposes), then individual-level data should be collected to obtain the joint distributions of exposures and outcomes. This would make it possible to test the ecological hypothesis thus generated so as to corroborate or refute the putative ecological association at the individual level. This is because for causal inference, individual data are required to account for population heterogeneity and confounding bias"

The Briggs argument is, essentially, that what is recommended here has not, usually, been done for epidemiological studies on particulates.

More importantly, this claim is, in principle, easy to refute (despite repeated claims in the comments that it can't be).

The claim is this. Many studies exist that try to address the relationship between particulate exposure and health. We can observe population health. And we can compare that to exposure to pollution. So far, so good. The problem is that, he alleges, many of these studies use data on exposure that is unreliable. He claims few if any studies use better exposure data.

Often these use models that estimate that exposure based on zip code of resident or even, county-level averages. He says we need better measurements of the real exposure. For example, a persons' exposure might be much worse if they travel to work via a busy road. Or far lower if they work in an air conditioned office 8 hours a day. Or the indoor exposure could be far lower than the typical outdoor exposure in their neighbourhood.

Some commentators claim that we simply can't use better exposure measurements. But portable meters exist so could easily be used to validate the models. Briggs is not demanding that we run unethical randomised controlled trials exposing people to different levels of pollution: he is questioning whether we have measured the actual individual exposures to the pollutants reliably. The issue is whether the input data on current epidemiological studies is reliable.

And this is, or should be, easy to refute. If there are good studies showing that a particular model of exposure to PMs shows a strong relationship to measured individual exposures, then his claim can be dismissed. If there are strong, convincing reasons why this is impossible to do, that would be a weaker, but still useful, challenge to his claim.

In short, he seems to be questioning whether we have reliable data about exposure in current studies on the effect of PM 2.5 particulates. This seems to be a perfectly reasonable question and one that should be easy to test.

Answer 1

If anything, some studies suggest that the present measurement methods underestimate the effects:

A reanalysis of the data by Willis et al, restricted to people who live closer to the monitor, reported a doubling of the estimate slope per unit exposure, suggesting substantial downward bias by classical measurement error.

---

The OP argues below that

There are perfect easy and non-invasive ways to assess human exposure to PMs without autopsies. You can have people carry portable meters that measure their actual daily exposure to the particulates rather than use some modelled exposure based on their residential zip code. The Briggs complain is that most studies do the second not the first making the exposure estimates easier and cheaper, but far, far less reliable in their models of the effect of the pollution on people.

It's not cheap to put "portable meters" on hundreds of thousands of people. There's a tradeoff between sample size and the accuracy of PM measurement. The study of Kloog for example (previous link) looked at 468,570 deaths.

There are some papers, e.g. a 2015 review which discuss the tradeoff between the number of sensors and the accuracy of each.

The main question, which arises when discussing about PM monitoring, regards the balance between pros and cons of the use of many compact and low-cost instruments vs. a limited number of more expensive and accurate professional devices. Actually, many low-cost sensors are easy-to-use, have small dimension and require limited power to operate continuously. They may provide to individuals the opportunity to monitor the local air quality. The low-cost makes them also suitable for large scale applications and the construction and exploitation of high-resolution maps obtained from a large number of single measurements. The drawbacks of low-cost sensors lie in their high noise, low stability, and limited accuracy although the latter can be improved by an appropriate calibration and an optimized data processing. Moreover, these sensors allow only to measure the total suspended particles (TSP) (Budde et al., 2014). On the contrary, many professional devices are extremely accurate, allow monitoring of various properties of airborne particles [...] and have been optimized for mobile monitoring. Some are certified by the governmental environmental agencies for official air quality measurements. However, at present, their high cost makes them unsuitable for large-scale investigations and for pollution imaging.

Also you cannot simply have a few guys go around and measure air pollution everywhere with just a few mobile device and be done because there's substantial temporal variation:

Another critical issue related to PM mobile monitoring arises from the high temporal and spatial variation in PM concentrations in urban areas.

Yes there are some smartphone based attachments now that fit into the first category (of lower accuracy) devices. The first paper using a phone that could also measure PM2.5 seems to have been published in 2018:

In this study, we comprehensively evaluated the first, and the only available, mobile phone—BROAD Life—equipped with air pollution sensors (PM2.5 and VOC), to answer the question whether this technology is a viable option in the quest of reducing the burden of disease to air pollution. [...]

At lower ambient concentrations of particles around 10 ug m-3 and 20 μg m-3 for PM2.5 and PM10, respectively, the phone’s response was below its noise level, suggesting that it is not suitable for ambient monitoring under relatively clean urban conditions.

Also in 2018, there have been some preliminary papers (meaning e.g. the usability of the portable devices) on what appears to be a planned epidemiological twin study.

The purpose of this study was to use the design-feedback iterative cycle to improve the usability of a portable PM2.5 monitor. This methodological paper describes the testing and refinement of the device for use in an epidemiologic study of personal air exposure measurements and clinical and biological outcomes in a large sample of twins recruited from a community-based registry. Although this study only reports on the usability aspects of the personal air pollution monitor, members of our team have published on the performance attributes of the sensor components for measuring PM2.5 and other endotoxins [citations].

I could not find any actual epidemiological data published from that ongoing endeavour though. A summary of the actual twin study design has been posted though.

But there have been some calibration studies by other groups. The aforementioned paper cites these and actually has the academic equivalent of the Briggs critique (as construed by the OP)

most epidemiologic studies have not measured true personal exposures. Instead, they have relied upon measurements made at central monitoring sites as exposure “surrogates,” resulting in considerable concern within the exposure assessment community as to the impact of such exposure error on disease estimates [13]. More sophisticated geospatial models have been used to try to capture spatial variations within urban areas [14] but often, the assumption is that the modeled ambient concentration at a subject’s residential address is a reasonable estimate of personal exposure, which is also incorrect because individuals are exposed to multiple locations in the course of daily living; for example, in the few studies that have used personal exposure monitoring instruments, substantial variations were found among individuals living within the same urban area and even within the same neighborhood [13,15,16]. Moreover, individuals tend to spend close to 90% of their time in indoor environments [17], and this is often not considered in air pollution epidemiologic studies. Recent meta-analyses of the issue have concluded that characteristics of the participants and their microenvironments can greatly affect the representativeness of such proxies and that greater attention is needed to evaluate the effects of measurement error [18,19].

Of the those last two papers/meta-analyses, only one is in English.

So:

• I have not found any epidemiological studies that use personal PM2.5 monitors; a twin study is ongoing apparently

• There is one 2010 meta-analysis of 18 studies trying to calibrate/correlate "ambient" (meaning sparser) models with personal PM2.5 data; the pooled subject-sample is quite small, 619 participants. The results were quite heterogeneous; the personal-ambient PM2.5 correlation varied a lot between studies (median 0.54; range 0.09–0.83).

---

As for another top-voted comment/why-not (not from the OP):

It doesn't seem difficult to measure the actual rate of particles in people's bodies and how they change over time - this is basic medical research and can be done on people who has volunteered to have research done to bodies after they die.

Alas, until very recently our poor "basic medical research" again only allowed us to measure very coarsely how particles move in live subjects. And you need that to measure changes. One can get a better measurements post-mortem, but then you only have one measurement per subject, as they no longer breathe.

What has been done in this respect is to produce models of particle deposition:

The deposition process has been studied for decades, and a thorough investigation on the deposition of particles is provided in the report by the International Commission on Radiological Protection (ICRP) (10). Based on the experimental data collected in the report, an ICRP model has been built to evaluate the deposition of particles from 1 nm to 10 µm in distinct anatomical regions in lungs, and the model has been widely used in studies of the health effects of air pollution (11, 12). The model is established on the assumption that the particles deposit uniformly in each generation of the airway in lungs. However, the deposition process and patterns are affected by the particle size. According to current understanding, kinematics of particles in the acinar region are dominated by the gravitational sedimentation for large particles (larger than 2.0 µm), while Brownian motion influences small ones (smaller than 0.1 µm) (13, 14). For intermediate particles in the size range of 0.1–1 µm, transport is dependent on the local irreversible kinematics within the alveolar cavities (13, 15?–17). Because of the different kinematics, the deposition process and patterns are more complicated than those predicted by the average model based on uniform deposition. This raises an important question: How can we know the real deposition rate in a local acinar region when the deposition pattern is nonuniform? It is still a challenge to measure the pulmonary deposition of particles in vivo. Radiolabeled particles are commonly used, but the spatial resolution is too coarse to distinguish and count the particles in a local area (10, 18). High-resolution deposition patterns in a lung can be obtained, but only from visualization in killed animals (19).

Only very recently (2019) has it become possible to measure at high resolution particle movements even in animals:

In this work, inhalation experiments are performed in vivo to obtain the deposition patterns and the deposition rates of the particles in local areas of a mouse lung using a fluorescent imaging method. Fluorescent polystyrene latex microspheres with the sizes of 0.2 µm (PM0.2) and 2 µm (PM2.0) are used, respectively, to simulate the small and large components of PM2.5 in the air. The concentrations of the particles in the air are 17.5, 175, and 1,750 µg/m3, corresponding, respectively, to the air quality levels of good, high pollution, and ultra-high pollution conditions according to National Ambient Air Quality Standards (NAAQS, 1997). The deposition of the particles in lungs are observed by a two-photon imaging microscope through a small surgical incision in the rib cage, and the deposition rates of particles in local areas are measured from the images in time sequence.

It would be thus possible in the future to correlate more precisely, in small-scale longitudinal studies in the laboratory, pattern deposition to other disease biomarkers. But some guy lecturing on youtube while waving a cigar will surely dismiss such future studies as (a) using a non-human proxy, (b) not using the actual disease outcomes. Asking for studies correlating such detailed measures of PM deposition directly with typical epidemiological measures, like say deaths in a population, is frankly a very tall order.

As a final point which connects the opening comment/claim of section to the main topic of the ecological fallacy: it is reasonably understood that low temporal resolution also produces a form of ecological bias/fallacy:

Focusing on the consequences of temporal aggregation of exposure, we show that an estimate obtained from a time-aggregated semi-ecological design can correspond to very different underlying time-varying exposure and risk scenarios. Further, distinguishing which of these is correct is not possible from the semi-ecological data alone.

And in air pollution studies, temporal resolution is even an issue for confounders like smoking:

as the effects of long-term exposure to air pollution are fairly small, an important consideration is the adequate control of potential confounding variables. Of particular concern is the need to control for time-dependent confounding, such as smoking patterns (which may be associated with an individuals exposure). [...] Villeneuve et al. [26] attempted to consider time-vary smoking in the context of the Harvard Six Cities Study. Unfortunately, inconsistencies in reported smoking histories in follow-up interviews of participants did not allow the use of a time-varying smoking variable.

(And nobody else apparently even tried that, as to the date of that 2007 paper.) Epidemiology is really the dismal science. I really need a smoke break now.

Answer 2

TLDR: Partialy correct. The epidemiological have large margins of errors. However measuring PM 2.5 more directly wouldnt really improve the results since the uncertanity of the exact composition of the PM 2.5 results in large margins of errors anyway.

So, first of all, are there studies on the health effects of particulate matter? Sure, lets take a look at this one. It states

Overall, these results provide a substantial molecular evidence base for potential positive health effects of SSAs at environmentally relevant concentrations through the mTOR pathway.

SSAs are aerosols, which is just another word for particulate matter see also Wikipedia or the EAA.

Confused? May i invite you to join study on PM-2.5mm (not micrometer)? Participants ingest various forms of PM-2.5mm, such a sugar, salt, sand or potassium cyanide. Oh, hold, its stupid assume properties of different compunds behave the same, just because they happen to have roughly the same size?

Some types of PM are well studied and known to be dangerous, e.g. asbestos. Others, like the areformentioned SSAs are healthy. Briggs doesnt seem to understand the concept, there is no such thing as the PM 2.5. I mean, what kind of PM does Briggs want measured? Tyres alone are made out of dozens of different types of material.

As a statistician Briggs should know, that these studies are not claiming that PM, NOx or whatever is causing "premature deaths" (i´ll ignore the fact, that the concept of premature deaths can only be applied to statistical twins), instead they are attributed to the corresponding markers. It doesnt matter if you measure PM 10, PM 2.5, NOx or whatever, as long as there is a direct causal relationship between the cause (traffic in this case) and the marker you use as a measurement (and you dont have to much outside interference from other effects producing those markers). You measure the marker(s) and corrolate them with the effect (cases of morbidity and mortality in this case). Use one marker, attribute all cases of morbidity and mortality to that marker, use multiple marker, allocate the cases of morbidity and mortality across them. That doesnt change the amount calculated lost life due to traffic in any way. And neither does it mean that that particular marker was the cause.

Its fairly safe to assume that a lot harmful effects are caused by various nanoparticles (such as Benzo(a)pyrene). Well, so how many premature deaths are attributed to these nanoparticles? Technically they belong to the PM, but they are so small and light, they arent even a rounding error, in other words, nill.

On the other hand some markers, such as NOx are known to be harmless at the concentrations observed in traffic (otherwise we would have loads of studies on high risk groups, e.g. cooks with gas ovens, and an ideal test group in that case as well, cooks with electric ovens). However its arguably better as a marker as there are fewer natural sources of NOx interfering. (Well, the cynic might say its because otherwise people would call for bans on electric cars. Those batteries are heavy. More PM from abrasion from the tyres).

So, summing the whole thing up. There are loads of studies on various particles that belong to the "PM group", but there is such thing as THE PM. And since you cant even compare PM from one source with PM from another source the whole argument doesnt make any sense.

Update: Since Briggs is specificly talking about a correlation between dementia and PM 2.5. Here is a little thought experiment that should convince him otherwise. Lets assume we equip several hundred people in district A and district B with some kind of mask that measures inhaled and exhaled PM 2.5.

The PM 2.5 level measured for group A is higher by a factor of 2-3 than for group B. Would this be a "good" experiment? This experiment could easily find a highly significant negative correlation between dementia an PM 2.5.

Lets say district A is uptown, a lot of people have those fancy fireplaces (lots of indoor PM 2.5) and drive large, modern cars, with a fairly high percentage of electric cars. Let district B be downtown, with lots of old cars. If something causes dementia, the cause will most probably have entered the brain, und thus the blood stream beforehand. This pretty much rules out the larger particles. Modern cars emit far fewer nanoparticles than older cars (see figure 13 on page 25 of the GasOMeP study). Such an experiment could easily yield worthless data, even with "correct" measurements of PM 2.5 levels. Relying on the law of large numbers to even out discrepencies simply makes a lot more sense.

Answer 3

Yes there are studies that have measured peronal exposure to air pollution with portable devices (mostly Black carbon) e.g:

• Personal exposure to Black Carbon in transport microenvironments
• Street characteristics and traffic factors determining road users' exposure to black carbon

Also some recent studies have observed directly the uptake of black carbon particles by humans, e.g.:

• Ambient black carbon particles reach the fetal side of human placenta
• Children’s Urinary Environmental Carbon Load. A Novel Marker Reflecting Residential Ambient Air Pollution Exposure?