I'm picking a few bits out of my next book but three, Science is Like That.
Lord Rutherford is supposed to have said “If your experiment
needs statistics, you ought to have done a better experiment”. Yet statistical
analysis reveals the underlying truths in complex situations, the sort of
messes that true physicists used to shy away from. It spoils the story a bit,
but Rutherford once sat in on Horace Lamb’s lectures on mathematical statistics
to improve his analysis of alpha particle deflections, a task which demanded some
serious statistical work.
Once upon a time, simple patterns were solved by simple
analysis, with simple mathematics revealing the laws that lay beneath the
patterns. By the 19th century, nothing was quite so simple any more. The
patterns were more complicated, and even physics needed statistics to help deal
with the large masses of data. Most medical and biological research, all social
science research and many other areas of modern scientific enquiry can only
work by using statistics.
While modern statistics owe more to Karl Pearson, R. A.
Fisher and J. B. S. Haldane, the first steps were taken by Adolphe Quételet,
and then carried forward by Florence Nightingale. Quételet was a brilliant
mathematician, who learned about probability from Pierre-Simon de Laplace while
studying in Paris, before he returned to his native Belgium to run a new
observatory there. While the observatory was being built, Quételet began
exploring the ideas of ‘social physics’ and ‘moral statistics’.
He saw that there were many predictable sets of data.
Crimes, suicides and marriages all involved individual free choice, but they
happened at predictable rates in different age groups, giving him the starting
point for his ‘moral statistics’.
Sad condition of the human race! We can tell beforehand how
many will stain their hands with the blood of their fellow-creatures, how many
will be forgers, how many poisoners, almost as one can foretell the number of
births and deaths.
—Adolphe Quételet, Treatise on Man, 1835.
Florence Nightingale makes an excellent case study, because
while we usually know her as a nurse who gained fame during the Crimean War, the
Lady of the Lamp, few people are aware that after this middle-aged spinster
returned to London in 1857, she used statistics to argue for better nursing.
First, she prepared a pamphlet, based on the report of a
Royal Commission, about the Crimean war campaign, where Britain and France had
fought Russia. Nightingale wanted to rally public support for nursing reforms.
The pamphlet showed where the problems lay, and her Mortality in the British Army, featured
the first use of pictorial charts to present data, those charts with tiny wheat
bags, or oil barrels or human figures lined up like so many paper dolls. She
hammered away again in 1858 in her Report
on the Crimea:
It is not denied that a large part of the British force
perished from causes not the unavoidable or necessary results of war…(10,053
men, or sixty percent per annum, perished in seven months, from disease alone,
upon an average strength of 28,939. This mortality exceeds that of the Great
Plague)…The question arises, must what has here occurred occur again?
In 1858, Nightingale was elected to the newly formed
Statistical Society and turned her attention to hospital statistics on disease
and mortality in Britain. You could never, she said, discover trends unless
figures were recorded in the same way. She prepared a plan, published in 1859,
for uniform hospital statistics. Her aim was to compare the death rates for
each disease in different hospitals, which could not be done without a
standardised recording system.
Others could also be counted as part-founders of
statistics. John Graunt published his Observations
on the Bills of Mortality of the City of London in 1662. This work has
sometimes been attributed to Sir William Petty, but George Udny Yule showed by
statistical analysis (how else?) that the sentence length in Observations did not match known samples
of Petty’s writing. Yule turns up again in chapter 5 (but you may have to buy the book to find out about that).
Graunt’s figures became the basis of the first life
insurance tables, but he also revealed that for a small fee, a death from “French-pox”
(syphilis) could be listed as “consumption” saving the family of the deceased
much embarrassment, while hiding a medical truth. Before the 19th century,
statistics were just numbers describing the state of a nation, and this is what
Mark Twain had in mind when he spoke of “Lies, damned lies, and statistics”.
After 1860, statistics began to take on a whole new
meaning, with a statistic becoming a summary figure for a large number of
measurements, a way of getting a handle on complex data. To experienced eyes,
the mean and standard deviation of a set of measures is a quick summary, though
lay people may still say statistics cannot be trusted.
The simple fact is that figures don’t lie, but liars can
figure. “Statistics” always need to be looked at carefully, but the use of
statistics in science is fully justified. Statistical analysis can reveal such
things as Burt’s fraudulent work on twins and inherited intelligence (chapter
9 of the book), or Mendel probably massaging his data, where he faked his data. Statistics
can also reveal amazing patterns, laws and truths.
Statistics would end up being the glue which tied
together evolution and genetics in the 1920s, helping biologists to understand
what was going on in large populations. In time, ecology would absorb pattern
analysis as a powerful tool, just as numerical methods would find a place in
biological taxonomy and classification. Tied in with this were tests of
significance in sets of results, tests
which provide an estimate of how likely numbers are to mean something.
It took statistics, wielded by epidemiologists, to prove
what people suspected in the 19th century, that tobacco causes lung cancer and
other diseases. You can trust statistics, if they are properly used. Mind you, in
the data set <1, 1, 1, 1, 2, 2, 2, 3, 3, 4, 10>, the mean is 3, the
median is 2, and the mode is 1—and the average statistician won’t tell you about
that!
Most modern scientific advances owe a great deal to
statistical analysis, often in the form of correlation coefficients. Now if I
can claim any special professional expertise aside from story-telling, it is to
be found in the application of statistics, and in particular to the honest and
dishonest uses of such statistics. I used statistical analysis to catch my
frauds.
But that's another story...