Trusting statistics, part 2

You will (I hope) recall that I said in Part 1 that statistics were once state figures. Now governments being what they are, or were, there was more than a slight tendency in the nineteenth century to twist things just a little, to bend the figures a bit, to bump up the birth rate, or smooth out the death rate, to fudge here, to massage there, to adjust for the number you first thought of, to add a small conjecture or maybe to slip in the odd hypothetical inference.

It was all too easy to tell a few small extravagances about one's armaments capacity, or to spread the occasional minor numerical inexactitude about whatever it was rival nations wanted to know about, and people did just that.

By the end of the last century, though, statistics were no longer the mere playthings of statesmen, and we find Francis Galton explaining that the object of statistics "...is to discover methods of condensing information concerning large groups of allied facts into brief and compendious expressions suitable for discussion".

So while you can go on sniping or objecting about being reduced to a mere statistic, those poor old statistics are still doing a fine job.

It's a pity, though, they've such a bad image problem, especially if you're trying to convert a diehard smoker from his or her evil ways.

As somebody once observed, or should have done if they didn't, figures don't lie, it's just that liars can figure.  Presumably we don't set out to deceive ourselves deliberately: but could we use statistical information in such a way as to be unintentionally misled?  I think it's very possible.  Like fire, statistics make a good servant, but a bad master.

From Galton's time on, his meaning of statistics as some sort of numerical summary has become generally accepted, and the addition of "tests of significance" has added hugely to the number of statistics we can use.

So if the word "statistics" no longer means what it did when Mr. Disraeli didn't really make his comment, then it hardly seems fair to keep on giving statistics such a cruel and unusual treatment.

But as I implied before, I won't rush to the defence of your average number-abuser.  If somebody does a Little Jack Horner with a pie that's absolutely bristling with statistical thingummies and they produce just one statistical plum, I won't be impressed at all: the plum's rather more likely to be a lemon, anyhow.

There are several handy little tests I apply to any figures and statistics which come my way: either the figures pass or they fail.  These tests let me decide whether I'll take any notice of the figures or not.  Statistical tests are very useful, especially if somebody is trying to prove by statistics that X causes Y.

In the first place, I want to know if there is a plausible reason why X might cause Y.  If there isn't, then it's all very interesting, and I'll keep a look-out, just in case a plausible reason pops up later, but I won't rush to any conclusion.  Not just yet, I won't.

Secondly, I want to know how likely it is that the result could have been obtained by chance.  After all, if somebody claims to be able to tell butter from margarine, you wouldn't be too convinced by a single successful demonstration, would you?

Well, perhaps you would: certain advertising agencies think so, anyway.  So let's take another tack: if you tossed a coin five times, you wouldn't think it very significant if you got three heads and two tails.  Not unless you were using a double-headed coin, maybe.

If somebody guessed right three, or even four, times out of five, on a fifty-fifty bet, you might still want more proof.  You should, you know, for there's a fair probability it was still just a fluke, a higher probability than most people realise.  There's just about one chance in six of correctly guessing four out of five fifty-fifty events.

Now back to the butter/margarine dichotomy.  Getting one right out of one is a fifty-fifty chance, while getting two right out of two is a twenty five per cent chance, still a bit too easy, maybe.  So you ought to say "No, that's still not enough.  I want to see you do it again!".

Statistical tests work in much the same way.  They keep on asking for more proof until there's less than one chance in twenty of any result being just a chance fluctuation.  The thing to remember is this: if you toss a coin often enough, sooner or later you'll get a run of five of a kind, and much more often than you'll fill an inside straight at poker.

As a group, scientists have agreed to be impressed by anything rarer than a one in twenty chance, quite impressed by something better than one in a hundred, and generally they're over the moon about anything which gets up to the one in a thousand level.  That's really strong medicine when you get something that significant.

There.  Did you spot the wool being pulled down over your eyes, did you notice how the speed of the word deceives the eye, the ear, the brain and various other senses?  Did you feel the deceptive stiletto, slipping between your ribs?  We test statistics to see how "significant" they are, and now, hey presto, I'm asserting that they really are significant.  A bit of semantic jiggery-pokery, in fact.


And that's almost as bad as the sort of skulduggery people get up to when they're bad-mouthing statistics.  Even though something may be statistically significant, it's a long way away from the thing really being scientifically significant, or significant as a cause, or significant as anything else, for that matter.

As I said earlier, statistics make good servants but bad masters.  We need to keep them in their places.  But we oughtn't to refuse to use statistics, for they can serve us well.

A replica of the Broad Street Pump, located in about the right place.

The plaque at the base of the replica pump.

And naturally, for research purposes only, I had to have several
pints at the John Snow pub, just down the road from the replica
pump. Inside, one meets locals and retired epidemiologists and

anaesthetists: Snow was also a pioneer at using anaesthetics.

One of my household heroes is Dr. John Snow, the man who solved a cholera epidemic in London in 1853.  He did it by having the handle taken off a pump in Broad Street which was supplying polluted water.  The story interested me, and I ended up researching it rather more deeply than I needed to, and I learned about some interesting side issues.  Let me share one of them with you now.

During that same cholera epidemic in 1853, not ten minutes' walk from the Broad Street pump, in London's Middlesex Hospital, an unknown woman of thirty-three was helping to look after some of Snow's patients, and many other victims of the epidemic as well.  It offered her some relief from the tedium of middle-class Victorian era spinster life, but her decision was a world-shaking one, nonetheless.

To us, she's no mere unknown, for that quiet spinster was Florence Nightingale.  And while most people know her as the woman who founded the modern profession of nursing, there are just a few of us who know of her other claim to fame: as a founder of the art and science of statistics.

I'll come back to her in my next talk, and to whether the ABC is secretly driving you insane, and why all the podiatrists in New South Wales seem to be turning into public telephone boxes in South Australia.  Or why I think that's what is happening.

My grandmother was one Florence Evans.  Not an unusual name, they told me, lots of Evanses in Wales they said, so when I visited her native village of Manorbier, there was some doubt as to just which of several Florence Evanses I was talking about.

Still, after old Mrs Ogmore-Pritchard had eliminated the one who died at seventeen, and the one who died an old maid, she recalled the one who emigrated to wild colonial parts, and there was my Flo Evans.

As I say, Florence is a common enough name these days, but in 1820, it wasn't at all common.  Only Florence Nightingale carried the name back then, and that was because she was born in the city of Florence, in a room with, by the sounds of it, a truly marvellous view.

It was only later, when Miss Nightingale became world-famous as the founder of modern nursing, that other young girls were also named Florence, in honour of the Lady with the Lamp.

And yet, Florence the First, Florence Nightingale, could quite easily have turned into a fairly good mathematician: anybody with the steely resolve to break into nursing as it was in those days, when it was peopled by drunks and retired prostitutes, anybody game enough to take on all of that, could have done just about anything at all.

And certainly Florence had the interest in mathematics, and she had the ability.  Unfortunately, she bowed to her father's wishes, and abandoned her interests.  Or did she?  After her name was made famous in the Crimea, Florence Nightingale returned to London in 1857, and started to look at statistics, and the way they were used.

First, she prepared a pamphlet, based on the report of a Royal Commission, studying the Crimean campaign.  This little work, named "Mortality in the British Army", is generally believed to feature the first-ever use of pictorial charts to present statistical facts.  Graphs, in fact, the origin of all those rinky-dinky little diagrams, beloved of geography teachers, you know the ones, with wheat bags, or oil barrels or human figures lined up like little paper dolls, or skittles, or whatever.

In the following year, 1858, Miss Nightingale was elected to the newly formed Statistical Society, just as she turned her attention to hospital statistics on disease and mortality.

In essence, she said, you could never discover trends in the data if everybody went happily around, concocting their own special data in their own sweet ways.  You had to make everybody keep their figures in the same way.  And so she prepared her scheme, published in 1859, for uniform hospital statistics.  Her aim?  No less than to compare the mortality figures for each disease in different hospitals, a thing which just could not be done under the old methods.

As in other spheres, Florence Nightingale was a success here, too, so the Statistical Congress of 1860 had, as its principal topic, her scheme for uniform hospital statistics.

These days, we use rather more sophisticated methods.  It won't be sufficient just to say Hospital X loses more patients than Hospital Y does, so therefore Hospital X is doing the wrong thing.  We need to look at the patients at the two hospitals, and make allowances for other possible causes.  We have to study the things, the variables, which change together.

This series started with Part 1 and it is completed in Part 3.