Early in my professional career, a mentor told me that some of the most intense professional disagreements involve matters in which so little is actually at stake.

I was reminded of this upon reading the funny pissing match about the best way to “measure” “corruption.”

As highlighted in this post, the European Centre for Anti-Corruption and State Building and the Center for International Private Enterprise recently released a “new free corruption analysis tool” called the Corruption Risk Forecast (CRF) which uses a term “Index for Public Integrity” (IPI) to rank certain countries. According to the groups, “the CRF relies on 30 fact-based indicators directly linked to observed sources instead of subjective coding of non-numerical data, which varies from year to year. The data used in the CRF is granular and comprehensive, spanning from the accessibility of land or business ownership information to the online disclosure of government mining concessions.”

Thereafter, Robert Clark (the Manager of Legal Research at Trace – an organization which happens to also produce its own ranking of corruption called the Bribery Risk Matrix) took to the FCPA Blog and argued that the results of the CRF “may not be reliable.”

According to Clark:

“The specific problem [with the CRF] arises in how the makers of the Corruption Risk Forecast determine whether a trend is “significant.” Usually, statisticians may consider a difference to be significant if it places the data point more than one standard deviation away from the average score. But while the Corruption Risk Forecast uses standard deviations as a reference point, it deems a change significant if it is more than one standard deviation away from zero, rather than one standard deviation away from the average change.


But there’s a more fundamental flaw in the Corruption Risk Forecast’s construction. There is no reason to expect that a comparison across two arbitrary moments in time (2008 and 2020) will yield a reliable forecast of a country’s handling of corruption in the future. This may be why the Corruption Risk Forecast includes additional steps beyond the quantitative calculations: a “political change check” in which the editors evaluate the effect of “radical political events” within the previous four years, and a “societal demand check” that aims to capture the popular pressure for good governance.

Although both checks are underdefined and obscure in their application, they have an outsized effect on the final conclusions: By my count, they altered the forecast for about 18 out of 120 countries—a full 15 percent.

To be clear, there is nothing wrong with using expert evaluation to help build this sort of index. But it does undermine any claim that the Corruption Risk Forecast is fundamentally based on hard facts rather than opinion. More importantly, such evaluation should not be used as a backstop for a statistically flawed methodology. Until these errors and shortcomings are addressed and fixed, I cannot recommend relying on the Corruption Risk Forecast.”

In response to Clark’s critique of its CRF, CIPE took to its website and argued:

“[Clark’s] critique, which centers on whether the new forecast should use zero or average global change as a baseline for statistical significance, is addressed by the CRF’s methodology section.

The CRF’s focus on changes within countries due to corruption constraints and opportunities aligns with a statistical baseline (or “null hypothesis”) of zero change. The alternative would lead to confusing results. For example, take a hypothetical increase in a country’s press freedom evidenced by several new prominent newspapers that regularly publish critiques of government decisions. Assuming the change is above a certain threshold (again, see methodology), the CRF considers it a significant new constraint on public corruption. On the other hand, a baseline of average global change would consider such a change to be irrelevant if the dataset’s average country showed a similar increase in press freedoms. Thus, in its choice of methodology, the CRF asserts that each country’s corruption constraints matter independently of what is happening in other countries.”

Here is the issue about this pissing match about the best way to “measure” “corruption.”

Does anyone really care?

As has been stated on these pages for some time, the various “measures” of “corruption” really don’t tell anyone anything that they likely did not already know.

In other words, regardless of the ranking, matrix, index, etc. used they all generally show that countries like Denmark, Finland, and Switzerland are less corrupt than countries like Mexico, Costa Rica, and Poland, and those countries are less corrupt than countries like Yemen, Somalia, and Sudan.


The post Who Really Cares? appeared first on FCPA Professor.