Bureau adopts new procedures for external peer review of important research
Bureau experts conduct various types of research to develop evidence that can inform the decisions policymakers face. The strength of that evidence depends on the quality of the research. The Bureau’s research is often highly technical, and therefore assessing the validity of the research can be challenging to policymakers and the public.
External peer review of research is commonly used across disciplines to enhance the quality and credibility of the research. Under peer review, a research paper is given to an expert or experts in the same field, who carefully review the work and provide a thorough and objective critique of the work. The review typically covers several facets of the research, including whether the data and methodology used in the research are appropriate for the research question and whether the conclusions drawn from the analysis are consistent with the analysis.
Subsequent to peer review, researchers often incorporate the feedback of the reviewers to improve the quality of the work. Academic publications regularly use peer review to assess and elevate the quality of research paper submissions, which helps consumers of that research draw conclusions about the validity of the research.
The Office of Management and Budget (OMB) published a Final Information Quality Bulletin for Peer Review that provides guidance to federal agencies concerning the peer review of “influential scientific information” and “highly influential scientific assessments,” terms defined in the OMB Bulletin. While the approach to peer review detailed in the Bulletin is different in many ways from traditional academic peer review, the goal of the Bulletin is similarly to elevate the quality and credibility of important technical and scientific information disseminated by federal government agencies.
Consistent with the considerations set forth in the OMB Bulletin about the benefits of peer review, the Bureau has elected to subject its own important research to external peer review. To conduct the peer review, the Bureau will rely on its Academic Research Council (ARC). The ARC is a panel of outside researchers with expertise in consumer finance who advise the Bureau on its research practices and topics, and it is ideally suited to conduct peer review of Bureau research. Collectively, the members possess expertise in the subjects the Bureau studies and the methods the Bureau uses.
Given their extensive substantive expertise outside of the Bureau on research related to our mission, the ARC can provide objective feedback. Furthermore, given the members’ commitment to supporting the Bureau’s research, they can also be counted on to conduct a thorough review.
Materials produced as part of the Bureau’s peer review process will be shared with the public on a dedicated webpage as they become available. We anticipate posting the original research report; the Bureau’s request for peer review; the ARC’s peer review report to the Bureau; the Bureau’s response to the ARC’s review; and, if merited, a revised research report that addresses major concerns raised by the ARC’s review.
The Bureau is still determining the scope of research it will subject to this external peer review process. In the meantime, the Bureau is in the process of subjecting to peer review a report entitled Disclosure of Time-Barred Debt and Revival: Findings from the CFPB’s Quantitative Disclosure Testing, which was posted on the Bureau’s website on February 21, 2020. As of today, the original research report, the Bureau’s request for peer review, and the ARC’s peer review report to the Bureau are available on the peer review webpage. Other materials, as applicable, will be posted as they become available.
The Bureau believes that peer review of its important technical and scientific research will ensure the quality of its research. This in turn will strengthen the policy making that stems from the research, and it will give the public confidence that its policies are driven by the best available evidence.