Statistics on statistics

Young pup Telegraph blogger — and assistant comment editor — Tom Chivers laments that only nine per cent of Britons believe statistics (H/T: Frank Davis).

For those who think I’m being ageist, even when I was Chivers’s tender age reading columns by my contemporaries, I did wonder about their wisdom. Give me a grizzled, craggy writer any day. His experience I can trust.

Chivers says that we cannot know the world ‘dispassionately’ unless it’s through the lens of statistics. Really? Anyone who has an interest in medicine and human life can see how manipulated many of these ‘studies’ and figures are. You can massage stats and create fear with any set of numbers, many of which haven’t even been peer-reviewed.

However, Chivers does not live in that universe:

The most depressing numbers are the following. One thousand and thirty-four British adults between 16 and 75 were asked to choose between the following statements:

Statistics are more important than my own experiences or those of my family and friends in helping me keep track of how the government is doing

My own experiences or those of my family and friends are more important than statistics inhelping me keep track of how the government is doing

Forty-six per cent chose the latter. Just nine per cent chose the former.

However, even those who are weak in maths can add 46% and 9%, coming up with, erm, 55%. What of the rest?

As Frank Davis writes:

So what about the other 45 percent? Did they not answer? Or ticked both options? Or started screaming, “I want to get out!” So that “nine percent of Britons” is really “nine percent of those asked, but who may not have answered”.   And it’s 16 percent of those who actually answered the question.

See? Statistics can show you anything and everything.

My favourite response following Chivers’s post comes from Telegraph reader sircomespect. Emphases mine:

Sorry Tom.

But having worked as a consultant for Government departments during the early 2000’s I can tell you that statistics are used for manipulated propaganda.

When the relevant, usually Labour party emplacement manager/supervisor/officer got control of a big profile issue, then anything goes.

If a stat can be twisted to fit, it will be. 

I saw rules manipulated and statistics doctored to make their ‘operations’ sound hugely successful.

If it was a DoT operation and they needed to show how their interventions were increasing employment, they would contact a company that was employing just to say ‘hello, how many people do you think you will employ this year?’ mark it as an intervention! and record the additional employment as a result of their interaction.

When I was doing a consultancy document for the NHS, the use of statistical manipulation was legendary.

At the time the emphasis was on smoking. There were two objectives – record how many people they had helped to quit and how many people were suffering through smoking ‘related’ disease.

You wouldn’t believe how wide a descriptor is the term ‘smoking related’ is. – Asthma? Did your parents ever smoke? Erm, I think so! BANG – Smoking related!

High blood pressure? – Have you ever smoked? ‘Well, I tried it in my teens!’ – Smoking related!

Do I ever trust statistics? Never. 

I have seen how they have manipulated data. I will never trust a Government statistic, especially not an NHS one.

On trade figures, another reader, nefti, explains the nonsense behind the UK’s foreign trade figures (emphasis in the original):

Allow me to demonstrate.

“Half of all our trade is with the EU”

Half.

50%.

But wait.

80% of our trade takes place domestically.

Within the UK.

And the remaining 20%?

About half goes through Rotterdam on its way to its eventual destination – the fabled Rotterdam Effect that is regularly used to mislead you.

Of that 20%, only about a third of those eventual destinations are within the EU.

So about 7%.

So next time you hear Nick Clegg say that “Half of all our trade is with the EU” you can reply………………

“Do you mean 7%?”

Elsewhere, nefti drew readers’ attention to an interesting American website called Shadow Government Statistics. Its author is economist William J. ‘John’ Williams, who has spent his career working with business owners and Fortune 500 companies. He writes (emphases mine):

One of my early clients was a large manufacturer of commercial airplanes, who had developed an econometric model for predicting revenue passenger miles. The level of revenue passenger miles was their primary sales forecasting tool, and the model was heavily dependent on the GNP (now GDP) as reported by the Department of Commerce.  Suddenly, their model stopped working, and they asked me if I could fix it. I realized the GNP numbers were faulty, corrected them for my client (official reporting was similarly revised a couple of years later) and the model worked again, at least for a while, until GNP methodological changes eventually made the underlying data worthless.

That began a lengthy process of exploring the history and nature of economic reporting and in interviewing key people involved in the process from the early days of government reporting through the present. For a number of years I conducted surveys among business economists as to the quality of government statistics (the vast majority thought it was pretty bad), and my results led to front page stories in 1989 in the New York Times and Investors Daily (now Investors Business Daily), considerable coverage in the broadcast media and a joint meeting with representatives of all the government’s statistical agencies.  

Nonetheless, the quality of government reporting has deteriorated sharply in the last couple of decades. Reporting problems have included methodological changes to economic reporting that have pushed headline economic and inflation results out of the realm of real-world or common experience.

Over the decades, well in excess of 1,000 presentations have been given on the economic outlook, or on approaches to analyzing economic data, to clients—large and small—including talks with members of the business, banking, government, press, academic, brokerage and investment communities. I also have provided testimony before Congress (details here).

An old friend—the late-Doug Gillespie—asked me some years back to write a series of articles on the quality of government statistics.  The response to those writings (the Primer Series available at the top-center of this page) was so strong that we started ShadowStats.com (Shadow Government Statistics) in 2004.  The newsletter is published as part of my economic consulting services. — John Williams

Statistics or experience. Which do you trust — numbers or your own lyin’ eyes?