Why I Changed my Mind about Systematic Evidence Maps (+ 2 new publications)

The goal of most environmental health research is to understand how a specific exposure impacts human health. In essence, the focus is on discovery.

The goal of risk assessment, however, is to use existing data to quantify population risk from exposures, with the aim of informing policy and regulations. Here, the focus is on evaluation and synthesis of the published research.

Over the last decade, systematic review has been gaining traction in the environmental health field as a way to transparently and objectively evaluate and synthesize evidence across multiple studies on a particular question (e.g., is there a link between developmental exposure to PFOA and fetal growth?).

But, because systematic review is focused on a specific research question, it is actually too narrow to inform the early stages of chemical assessments and evaluations – which often need to start by broadly scoping the entire evidence base before prioritizing topics for in-depth analyses.

A (relatively) new approach: Systematic Evidence Maps

A more useful tool for this context is a systematic evidence map (SEM). SEMs utilize the same systematic and transparent structure as systematic reviews, but they have a broader (and more neutral) goal: to characterize the entire evidence base for a certain topic and map the features of the data through visualization tools (e.g., Tableau). In contrast to systematic reviews, SEMs do not draw any conclusions.

Example SEM from The PFAS Tox Database

When I first joined the EPA’s Integrated Risk Information System (IRIS) program, everyone seemed to be working on SEMs (since they are now often part of the IRIS assessment development process). Coming from my training in academia, I was not particularly excited about these “descriptive” products. The focus on tallying features of the evidence base (“X” number of epidemiological studies on developmental outcomes, “Y” number of experimental studies on developmental outcomes, etc) seemed pretty dull, to be honest.

But after almost two years into my job here, I’ve definitely come to see their value.

SEMs, like many other steps in the chemical assessment process (e.g., study evaluation, data extraction, etc), are not at all sexy. But they can substantially increase the transparency, efficiency, and credibility of assessments. (And that’s the goal, right – more trustworthy assessments? Yes, indeed.)

The benefits of SEMs

You can think about SEMs as very detailed scoping outlines developed from structured literature searches. By looking at an (often) interactive SEM (here’s one recently published for naphthalene), we can quickly identify data gaps, decide whether a new or updated assessment is needed, and/or refine the assessment priorities. This process ultimately saves us time and ensures that our assessments focus on topics with enough evidence to draw conclusions. (Of course, lack of data does not mean lack of risk, but we can’t develop robust assessments on topics with minimal data.)

The National Academy of Sciences, Engineering, and Medicine (NASEM) has also highlighted that SEMs could be valuable in themselves as publicly accessible databases of all of the research on a certain topic. (Check out this great one on PFAS from a few years ago!). Community groups, NGOs, and local, state, or federal agencies could use a published SEM as a starting point for their own particular goals and needs.

Advancing coordination

The idea that one group’s work to develop an SEM could be used as a starting point for another group’s projects is very exciting to me. It sounds so simple, but there is just not enough data sharing in the environmental health field. Sharing SEM content, which is broad enough to support downstream work in a variety of different contexts, could save a lot of time and resources.

This type of coordination is something that I’ve thought a lot about since publishing an infographic depicting the existing – and very complex – landscape for chemical evaluations and assessments. There’s no need to re-create the wheel for each assessment if we can start from a common and trusted SEM. Then, each program could take that body of work and use it for their particular statutory needs.

To further support this type of collaboration and coordination, the IRIS program has just published our SEM template (and an associated introductory article with relevant context). It’s not the most thrilling publication (especially to those outside the world of chemical assessments). But trust me – it is exciting! Using unified methods and reporting approaches can advance interoperability of SEMs and promote harmonization across the environmental health field.

We have a lot of chemicals to evaluate, so the more we can work together, the better.


Related publications:

Thayer, Kristina A., et al. “Use of Systematic Evidence Maps within the US Environmental Protection Agency (EPA) Integrated Risk Information System (IRIS) program: Advancements to date and looking ahead.” Environment International (2022): 107363. https://doi.org/10.1016/j.envint.2022.107363

Thayer, Kristina A., et al. “Systematic Evidence Map (SEM) Template: Report Format and Methods Used for the US EPA Integrated Risk Information System (IRIS) Program, Provisional Peer Reviewed Toxicity Value (PPRTV) Program, and Other “Fit for Purpose” Literature-Based Human Health Analyses.” Environment International (2022): 107468. https://doi.org/10.1016/j.envint.2022.107468

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s