Scientific Document Review at the Centers for Disease Control and Prevention: The CLEAR Approach

John K. Iskander, Angela Calugar, and Richard D. Peavy are with the Office of the Associate Director for Science (OADS), Centers for Disease Control and Prevention (CDC), Atlanta, GA. Anne Sowell is with the National Center for Chronic Disease Prevention and Health Promotion, CDC.

Correspondence should be sent to John K. Iskander, MD, MPH, Centers for Disease Control and Prevention, OADS, 1600 Clifton Rd, MS D-50, Atlanta, GA 30333 (e-mail: vog.cdc@0ixj). Reprints can be ordered at http://www.ajph.org by clicking the “Reprints” link.

J. K. Iskander conceptualized and drafted the editorial. J. K. Iskander, A. Calugar, and R. D. Peavy performed the underlying work and critically revised the editorial for important intellectual content. A. Sowell provided important content revisions and critical review of the work.

Scientists at the US Centers for Disease Control and Prevention (CDC) publish an average of 50 peer-reviewed articles per week, 1 in addition to numerous other widely disseminated materials. Review of scientific content at CDC is a key process by which the agency maintains high scientific standards. Formal CDC review processes, referred to as clearance, recently have come under scrutiny 2 but have been upheld as crucial to maintaining the agency’s scientific reputation. 3

To help this process, particularly for short-term midcareer fellowships at CDC beginning in 2012, a review framework was developed empirically following the review of hundreds of scientific articles and incorporated into the curriculum. To facilitate its assimilation, the framework was captured in the mnemonic CLEAR, standing for Clarity, Logic, Ethics, Agency, and Relevance. This novel approach provides insight into the scientific review process at CDC, but also could be applied more broadly to improve the quality of public health science.

Our framework includes a three-level comment structure ( Table 1 ) that prioritizes comments for author response. The most important distinction is between comments that require action by the author to correct errors of fact or policy misstatements (level 1) and comments intended to improve clarity or editorial changes (levels 2 and 3), for which the author has some discretion in responding. Use of the three-level comment system allows reviewers to refine their reviewing style and focus on communicating level 1 issues to the author.

TABLE 1—

Three-Level Comment Structure for Scientific Document Review at the Centers for Disease Control and Prevention

We describe the five overarching principles encapsulated in CLEAR and provide examples of how each translates into specific aspects of scientific document review.

CLEAR

All comments are grounded in improving scientific communication.

Clarity refers to factors that enhance reader comprehension. Clarity can be improved by reducing sentence length, splitting lengthy paragraphs, and limiting use of jargon or acronyms.

Logic describes whether the flow of ideas can be followed and whether the conclusions follow from the reasoning or facts presented. The information should move from definition of the problem, through description of the methodology, presentation of the results, to a discussion of how the results address the problem. Even non–subject matter reviewers should be able to assess whether the authors are clearly describing study methods, including case definitions and inclusion or exclusion criteria, and whether inferences drawn are reasonable and discussion of contradictory or inconsistent results is adequate.

Ethics considerations include not only acknowledgment of institutional review board or other human participant protection review, but also whether readers may perceive broader ethical concerns. If the work involved vulnerable populations (e.g., pregnant women or incarcerated persons), additional explanation of protections for these participants may be needed. If the work raises privacy or confidentiality concerns, removal of potentially identifying details or censoring of data may be warranted.

Agency, in the context of the CLEAR framework, refers both to whether the authors’ organization or agency can support the work described in the document (i.e., Are the findings or recommendations placed in the appropriate policy context?) and to whether it is clear that the authors are speaking only on their own behalf (i.e., they are the “agents” of the work) or on behalf of their organization.

Relevance addresses whether the information is of practical use to the targeted audience. Relevance may refer to increased awareness or recommended actions related to emerging conditions and their risk factors, use of new diagnostic criteria or laboratory tests, or reporting of specific conditions to public health authorities. In the case of a journal supplement or a group of related articles, the framework can be extended to C-CLEAR, with the additional C standing for Context. Context refers to viewing the article within its broader scientific, policy, and programmatic framework.

REVIEW OF MMWR ARTICLES

As an example integrating the five CLEAR principles, we outline the review of Morbidity and Mortality Weekly Report (MMWR) weekly articles conducted within CDC’s Office of the Associate Director for Science. This level of review dates to 1960, when Alexander Langmuir, chief epidemiologist at CDC, routinely reviewed all content before publication. 4 One week before publication, articles are shared internally for agency-level clearance. All articles contain a single overriding health communication objective (i.e., take-home message), typically within the first paragraph. 4 Reviewers ensure that the single overriding health communication objective is sufficiently clear for a professional audience (C) and, if needed, is repeated as part of the article’s conclusion and summary. Reviewers check for consistency of data presented within the article and between text and figures or tables (L). Any MMWR content considered to be human participant research must describe review by the institutional review board and, when appropriate, clinical trial registration (E). Because articles published in MMWR represent agency policy, 4 reviewers assess whether policy-related statements are consistent with CDC policy or recommendations (A). MMWR articles may present recommendations not originating from CDC, if they are clearly labeled as such. Reviewers verify that recommendations presented are relevant to clinical and public health practitioners (R). This might involve authors identifying risk factors, such as specific travel or exposure histories, associated with rare but serious emerging infections. 5

Reviewers use the three-level comment structure to submit their reviews for response by article authors. Final article reviews, which occur a few days before publication, focus on author responses to level 1 comments. This organized, collegial, and efficient process permits timely review and publication of four to eight articles per week on high-priority public health topics. During public health emergency responses, when rapid review and dissemination of evolving information are critical, this system is especially valuable. 6

Best practices for reviewers include providing comments that are specific and actionable and making clear the distinction between mandatory (level 1) and voluntary (levels 2 and 3) comments. Comments should use neutral, constructive, and respectful language. Specific wording suggestions should be limited in length and number. A single reviewer should synthesize potentially conflicting comments so that the review “speaks with one voice.”

PRACTICE-BASED EVIDENCE

We propose the CLEAR framework as practice-based evidence on scientific reviewing. Although CLEAR is not the official policy of the CDC or MMWR, in our daily experience, it is a helpful tool for teaching and mentoring reviewers of various levels of experience, including students of public health, interns, fellows, and midcareer scientists. We have deliberately chosen to use the mnemonic device CLEAR to represent this conceptual framework, both because it can be easily remembered and because it signifies clarity, a value to which all authors and reviewers should aspire. Because continuous improvement has been identified as a core function of public health governance, 7 we encourage testing and feedback on its utility in various settings.

CDC’s output of articles and recommendations is a key factor in establishing the agency as one of the leaders in public health science. Upholding rigorous standards of science through the review process provides important oversight and accountability for the agency. For reviewers, critically analyzing content while providing constructive feedback can contribute to development of professional competencies and scientific leadership skills.

An organized and constructive review of scientific content is not merely an aggregate of editorial suggestions. Reviewers of public health documents should seek to improve the clarity, logic, ethics, agency, and relevance of content they review. By doing so, they can add scientific value to documents, as well as making authors aware of broader policy considerations into which the reviewer may have greater insight. The CLEAR framework, which can be applied to both internal and peer review processes conducted in a variety of public health settings, has the potential to increase the speed with which actionable results are shared with health professionals, who can translate them into lives and monies saved.

ACKNOWLEDGMENTS

Funding was received from the Centers for Disease Control and Prevention.

We thank Mark Hoover, Susan Laird, James Stephens, Leonard Ortmann, Sara Beth Wolicki, Harold Jaffe, and the Morbidity and Mortality Weekly Report editors and authors.

REFERENCES

1. Iskander J, Bang G, Stupp E, Connick K, Gomez O, Gidudu J. Articles published and downloaded by public health scientists: analysis of data from the CDC public health library, 2011-2013. J Public Health Manag Pract. 2016; 22 (4):409–414. [PubMed] [Google Scholar]

2. Blank MB, Jemmott JB., 3rd The CDC clearance process: an obstacle to progress in public health. Am J Public Health. 2015; 105 (4):614–615. [PMC free article] [PubMed] [Google Scholar]

3. Cono J, Jaffe H. The CDC clearance process: supporting quality science. Am J Public Health. 2015; 105 (6):e1–e2. [PMC free article] [PubMed] [Google Scholar]

4. Centers for Disease Control and Prevention. Public health then and now: celebrating 50 years of MMWR at CDC. MMWR Suppl. 2011; 60 (4):1–124. [PubMed] [Google Scholar]

5. Chevalier MS, Chung W, Smith J et al. Centers for Disease Control and Prevention (CDC). Ebola virus disease cluster in the United States — Dallas County, Texas, 2014. MMWR Morb Mortal Wkly Rep. 2014; 63 (46):1087–1088. [PMC free article] [PubMed] [Google Scholar]

6. Bell DM, Damon I, Bedrosian S, editors. et al. CDC’s response to the 2014–2016 Ebola epidemic —West Africa and United States. MMWR Suppl. 2016; 65 (suppl 3):1–106. [Google Scholar]

7. Carlson V, Chilton MJ, Corso LC, Beitsch LM. Defining the functions of public health governance. Am J Public Health. 2015; 105 (suppl 2):S159–S166. [PMC free article] [PubMed] [Google Scholar]

Articles from American Journal of Public Health are provided here courtesy of American Public Health Association