SBA Defends Peer Review Process on Regs Study; ‘Offered the Study for Review’ to Experts Beyond the Two Who Actually Responded

by Ben Somberg

April 06, 2011

When the U.S. Small Business Administration issued a study last September claiming regulations cost the U.S. economy $1.75 Trillion in a single year, the agency trumpeted that the "report was peer reviewed consistent with the Office of Advocacy’s data quality guidelines."

But the peer review file included with the study was embarrassingly meager -- comments from all of two individuals. The authors, economists Nicole Crain and Mark Crain, ignored a fundamental criticism raised by one of the two reviewers that struck at the very heart of their estimates of economic regulatory costs. The second reviewer's complete comment had the sort of casual quality to it that suggested a somewhat less than thorough review. The review, in its entirety: “I looked it over and it's terrific, nothing to add. Congrats[.]"

When CPR Member Scholars issued a report in February critiquing SBA's study, they noted that the peer-review process was unimpressive. CPR co-author Sidney Shapiro sent a letter at the time to Karen Mills, the SBA Administrator, and Dr. Winslow Sargeant, Chief Counsel of the SBA's Office of Advocacy, calling on the SBA to withdraw the study and disavow its findings. The letter noted, among other problems, the inadequate peer review.

Dr. Sargeant responded with a letter in March, standing by the report and waving off the CPR report’s very thorough criticisms, although without responding to the specifics. There was perhaps little of particular surprise, except one bit about the peer review process. Wrote Sargeant:

Advocacy offered the study for review to a number of individuals with expertise in regulatory cost-benefit analysis, including former heads of Federal regulatory agency economic analysis offices and academics who have published in the field.

For Dr. Sargent, it was apparently just a minor detail that only two peer reviewers responded, as far as anyone knows, and one of the two peer reviewers took issue with the core methodology and the other devoted all of 11 words to a review. Today Shapiro wrote back to SBA, noting that while SBA apparently regards silence as an endorsement by peer reviewers, CPR does not.  

Shapiro also reiterated one of the primary criticisms from CPR: the public does not have access to the data, equations, assumptions, extrapolations, and calculations that are necessary to understand and verify the methodologies, data, and assumptions that were employed in report. Shapiro calls on SBA to withdraw its sponsorship of the report until the agency makes the full methodologies, data, and assumptions available to the public, allowing the study to be replicated.

The SBA study, remember, was not privately funded; this was taxpayer dollars at work. The public deserves an explanation.

Be the first to comment on this entry.
We ask for your email address so that we may follow up with you, ask you to clarify your comment in some way, or perhaps alert you to someone else's response. Only the name you supply and your comment will be displayed on the site to the public. Our blog is a forum for the exchange of ideas, and we hope to foster intelligent, interesting and respectful discussion. We do not apply an ideological screen, however, we reserve the right to remove blog posts we deem inappropriate for any reason, but particularly for language that we deem to be in the nature of a personal attack or otherwise offensive. If we remove a comment you've posted, and you want to know why, ask us (info@progressivereform.org) and we will tell you. If you see a post you regard as offensive, please let us know.

Also from Ben Somberg

Short Bio needs to be uploaded.

Simpler Government, or Secret and Unaccountable Government?

Somberg | Apr 04, 2013 | Regulatory Policy

Who Will Run the EPA?

Somberg | Apr 01, 2013 | Regulatory Policy

Mancini "Leads" OIRA as Deputy Administrator

Somberg | Mar 13, 2013 | Regulatory Policy

There is Now No OIRA Administrator

Somberg | Mar 11, 2013 | Regulatory Policy

Robert Glicksman Testifies in House Hearing on Regulatory Policy

Somberg | Feb 28, 2013 | Regulatory Policy

The Center for Progressive Reform

455 Massachusetts Ave., NW, #150-513
Washington, DC 20001
info@progressivereform.org
202.747.0698

© Center for Progressive Reform, 2015