Jack M. Mintz: The measure of a model is how well it predicts

COVID-19 mandate protesters block the roadway at the Ambassador Bridge border crossing with the U.S. in Windsor, Ont., on Feb. 9, 2022. PHOTO BY GEOFF ROBINS/AFP VIA GETTY IMAGES

 

With the trucker protest against mandatory health policies still in play, it is no surprise that the controversial Johns Hopkins economic study released last week made the front pages here. This is a far cry from Imperial College London epidemiology studies from almost two years ago that estimated lockdowns were needed to cut deaths by 98 per cent. The three authors, only one of whom, Hanke, is actually at Johns Hopkins, conducted a «meta-analysis» of 34 COVID studies focused on March 16-April 20, 2020. Comparing COVID deaths across jurisdictions, they concluded that mask mandates had the largest impact on mortality with rules such as shelter-in-place or school closings having only marginal effects.

Some estimates credit voluntary avoidance with 90 per cent of the total effect. Mandates did not regulate everything, of course. Not surprisingly, many epidemiologists took vigorous exception to these results, which are so different from their theoretical models, which were excluded from the analysis. In their view, the Johns Hopkins study cooked the books.

That requires analyzing past data to see whether their predictions were in the ballpark. This is not an easy task since observations, such as the number of COVID deaths, depend on many other factors besides lockdowns and these also need to be included in the analysis. One approach to create more confidence in a theory is to «backdate» models to see if their predictions are close to what actually happened. Set the model up with data that goes to 2010, say, and then see if it «explains» what happened between 2010 and now.

The Johns Hopkins authors were right to include in their meta-analysis only papers that had been tested empirically, not those that had only made theoretical predictions. Whether their conclusion about the effect of mandates is correct only time will tell. Another area where complex models are used to make predictions about real-world events is concerning the possible effect of greenhouse gases on average temperatures. These climate models are also based on theory and then used to make predictions by filling them out with data.

The predictions are highly dependent on assumptions and data accuracy. In the case of climate models, you want to see if they can predict the temperatures we’ve already experienced. The new models predict even higher temperatures in the future.

One of the first such models, built by the University of Chicago’s Arnold Harberger in the 1960s, looked at the incidence of the corporate tax assuming a two-sector economy producing goods and services using labour and capital. Many empirical papers have shown that at least half of corporate tax is borne by labour through lower real wages, more so in smaller economies like Canada. Modellers should be humble about predictions coming from complex mathematical models that haven’t yet successfully confronted the data. Despite what the headlines said, that’s the most important lesson from the Johns Hopkins study.

Read more.

Source: Jack M. Mintz | Financial Post