Some thoughts on educational research

I found a reply that apparently never went to an email list. It was written some years ago, but it remains valid today.  The matter under consideration was an item of research that concluded that "who" your parents are, as in being better educated, high socio-economic status etc etc was more important than what they "do" with their children in their formative years and so "proved" that success is dependent on those factors.

One of my good friends (her name is Barbara, which will identify her to a few) commented:

I am ... very sceptical about a lot of  "research" because, as we know,  much of it is undertaken by parties with vested interests in the results and these can be skewed to reflect those interests and sell their product.

I agreed with this, but from an expert viewpoint. My reply began by saying that my science teacher wife and I had been entertaining a rather delightful US environmental activist, writer and illustrator who had been staying with us for a few days, so I had not been reading my email, but oddly enough, reading to children was a topic of conversation that arose several times as we clambered around Sydney's many Aboriginal engravings sites.

We agreed that reading was a key, because we shared that sort of mind-set.  We even debated, not the truth of what we saw as self-evident, but how reading to small persons could be encouraged more, because we took its central role as a given. This is what tends to happen when educators meet: they agree on what is good, or potentially good, and look at ways to foster it.

I must say that I agreed strongly with Barbara about educational research -- and in the 1980s, I had more than my fill of contact with educational researchers seeking access to certain classes of data that were under my management in the NSW Department of Education. If they wanted our data, or access to our schools, they had to get past me.

This is Peter the ADHD, who is almost indistinguishable from Rikki-Tikki-Tavi who always had to run and find out: I care passionately about Aboriginal engravings, I care about wee beasties, rocks and stuff — and I am a dab hand at educational research, among other things. I also know a fraud when I see one, and I saw a few.

I will be blunt: I moved on to other and more engaging things when I found that most educational research is the codification of the bleeding obvious, and some journals are amazingly able to be worse than others. At one point when I was no longer with the NSW Dept of Education, I acted as the external member of a panel to select somebody who needed expertise in research methodology.

After hearing the applicants I argued for one bright young person getting the position, because I had asked about the Journal of Educational Research, probably the most popular "journal" in the Directorate where he was working and where I had once worked.

Most of the other applicants, when I asked them about their preferred sources for information, immediately named the  Journal of Educational Research, praising it for its lucidity and relevance.  The bright young person was different.

He had mentioned other journals, but not the JER, and when I asked him about it, he declared that he wouldn't touch it with a barge pole.  This was in contrast to the other suckers, who had all declared, unprompted, that it the best thing since sliced bread.

The BYP's answer accorded with my judgments of JER, based (among other bizarre things) on a learned piece that demonstrated that children took longer to read longer passages (the author did have the grace {or tail-covering skills} to admit that this result was unsurprising!)  The others on the selection panel saw other values in him, and he got the job by unanimous consent.

Surely the key thing with the alleged research on determining factors is that many of the variables are correlated. I used to be one of those austere number-crunching types who would engage in statistical jiggery-pokery to partial out confounding effects of inter-correlations, but that is probably even rarer now than it was then.  You have to do that, and do it the right way around.

I suspect that the better-educated and higher status etc etc parents are also the ones who take it for granted that they should read to their children.  Maybe some of them are too busy earning Nigel and Nigella's school fees, but others are right in the thick of it.

In the same way, some parents take it as a given that they will sit and watch television with their children -- and I suspect that this style of TV watching would not be counted as different from unsupervised watching by most "researchers".

"All science is either physics or stamp collecting" said Lord Rutherford, a little unkindly. Well, some educational research is stamp-collecting, and some of it barely makes it to the level of a sincere form of philately.  Please remember that, next time you see a report that research has shown that something is good while all competing forms are incontrovertibly bad.

Remember: research in education nearly always states the bleeding obvious — or at least that which was the bleeding obvious for the "researcher" when the "research" took place. There are, I have to concede, rare exceptions, but these are vanishingly rare.