The selling of PLATO

Caution: the following is quite long, and it is posted as an historic resource of interest to a small minority. It is here for evaluators, educators and number-crunchers who find fraud, forensic accounting and other dry stuff to be quite riveting.

This completes the fraud series that began with Keeping Savages in a Cage, and continued with Fraudo the Frog?.

Background

In 1981, I was handed a proposal from the Control Data Corporation, and because of my known funny head for making numbers sing, I was asked to give it a once-over. The whole operation proved to be a fraud which I foiled by writing what was referred to as "the acid drops" in 1981. Basically, CDC tried to con four government departments into buying an outdated  computer-based education system called PLATO.  They delivered a thick wad of "evidence" which was a complete load of garbage, as I showed, and as they say in assassins' circles, I did so with extreme prejudice (hence "acid drops").

What I found is known in general terms in fraud circles, but it has never been fully documented: that is the reason why this is now posted.

My "acid drops" paper was marked for no further distribution, but a slime-bag of my acquaintance loudly praised PLATO in 1985 at the Australian Association for Research in Education.  This was some years after I had shot PLATO down, and I knew this bloke for a complete shonk who had harmed a friend of mine by stealing her credit (and to a lesser extent, my credit). No names no pack-drill, but Colin, I know you are out there somewhere...

This ghastly barrow-boy needed to learn that this was not the best way to build your career.  So the following year I delivered my clinical demolition of PLATO to AARE, hoping he would be there, but he had skulked off back to England. My paper was delivered to a small circle of cognoscenti who basically nodded, and said "we thought as much".  The offer had been spurned, and nobody cared much any more.

I recently found the printed paper. The events happened more than 30 years ago, and I am applying the 30-year rule. I ran the paper through OCR the other night, and recovered it. Delightfully, in the same folder, I found the original "acid drops" paper, which I am sitting on, along with a large volume of evidence. If anybody is silly enough to even hint at legal action, they need to be aware that I can prove that some CDC people knew the claims were fraudulent but still made them. If you try to be a nuisance, I will escalate.  I can do that, you can't, and if you try, expect to pay for it.

In fairness, the known crooks were in America, and Colin was just a numpty. I am quite certain the Australian CDC people were blissfully unaware that the evaluation studies they handed us were bogus. Even an idiot would not have handed over the data they gave me, because the proof that I predicted would be there was easy to find. I played the role of admiring numbskull, and got the whole lot.

I have appended one (and one only) of the smoking guns at the end.  I am hanging onto the rest, to use as evidence.


* * * * *

The paper begins:

This is the story of a meta-evaluation that was completed in 1981. The object of the evaluation was PLATO, a computer-based education system, as it was used to teach basic skills to adults, the object of this report is to show where the simplest enquiries can sometimes lead. The evaluation relied heavily on gain scores derived from standardised tests: we ought really to start with these, so that we are all talking the same language.

In the first place, we need to be sure that the standardised test used is appropriate. Preparing a standardised test is both difficult, expensive, and time-consuming. It also takes a considerable amount of time to do. Researchers commonly look around for a test which appears to ask the right sorts of questions, and which has been tried out on the same sorts of people as those under study in the research. If such a test can be found, then the researcher may report results such as "9% of the students performed at or above the 9th grade level" (most standardised test are developed in the USA.

But is a standardised test the best way of assessing the performance of an adult basic education student? Galen (1980) quotes Otto Ford (Teaching Adults to Read, 1967) as saying "Everyone agrees that an adult can be frightened away from a basic education program by testing. The informal inventory in the hands of a sensitive teacher has none of the formidability of standardised tests." The main thing to be said for a standardised test is that it is a convenient and quick method of gathering the data required for a study or evaluation.

A proper measure of reading ability would involve sitting down one subject with an experienced teacher who would watch the subject read, listen to the subject reading, and ask questions of the subject, all before making an informed decision. A standardised test simply poses a set of (usually) multiple choice questions aimed at objectives which reflect reading ability. These indicators of reading ability are then used to allow us to make the (usually safe) jump to the (usually correct) conclusion that we have direct information on the reading ability of the subject.

But this can come unstuck if the subject has been coached in those skills and those skills only which are tested in the test. In this case, the assessed reading ability (i.e., the score on the test) would be too high. Again, if we are studying "slow learners", the subject's testable skills may be exactly the ones in which he or she is having difficulty. If the subject has learned to compensate in some way for these difficulties in some roundabout way, then his or her scores will be too low: the deficiencies are still there, but they do not affect actual reading performance any more.

A gain score is calculated when a standardised test has been given twice, once before, and once after some form of instruction. To assist this, most standardised tests are available in two or more equivalent forms. The difference in grade equivalent score (or raw score, for that matter) is then calculated, and attributed to the intervention of some forms of instruction. In logicians' circles, this is known as post hoc ergo propter hoc, and held in low esteem as a form of proof. Stake goes further in his criticism:

"The testing specialist sees not one but at least four hazards attendant to the analysis and interpretation of learning scores: grade-equivalent scores, the “learning calendar", the unreliability of gain scores, and regression effects. All show how measures of achievement gain may be spurious. Ignoring any one of them is an invitation to gross " misjudgement of the worth of the instruction." (p 210)

Much of the evidence which we will consider later depends on the interpretation of gain scores. It will thus be instructive for us to consider each of Stake's objections individually.

The grade-equivalent scores objection

It often happens that a difference of one, two or three marks in raw score is equivalent to the gain typically found between one grade and the next. If this is the case, then we must be wary of gains which are due merely to chance effects, or to the acquisition of one minor skill.

The standardised test most frequently used in PLATO studies is ABLE: the Adult Basic Learning Examination. While I have not been able to obtain copies of the test itself, I have located reviews, and those are most revealing.

Hieronymus (1972?) comments in general terms about the shortcomings of Levels 1 and 2 of ABLE, detailing problem areas, and concluding "This general criticism applies to a somewhat lesser degree to all of the tests in the battery with the possible exception of reading.“

He then goes on to criticise the reading test:

“The reading tests consist of short passages, in which the last word in most sentences is missing and must be selected from three alternatives. For this reviewer, this type of reading test has some serious shortcomings. Most of the passages consist of two or three sentences interrupted by missing words. The examinee must use the context of the remainder of each sentence to select the word which best fits the context. This type of item does not recognize the multi-faceted nature of reading comprehension. No emphasis is given to such skills as generalization, discerning the main idea, evaluating the purposes, attitudes, or intentions of the writer, etc "

Fry (1969) also finds fault with the test on these grounds:

"…there are only four items which cover the grade range 5.0-5.8, while three items cover the eighth grade. Hence, I believe that the man who wrote the front page and probably the advertising copy for this test should state that Level I is most suitable for testing groups with first- and second-grade ability and Level II is suitable for students with third- eighth grade ability with much greater discrimination at the lower end.

But Fry is also critical of the Level II arithmetic test:

"…Test 4, Arithmetic Problem Solving for Level II of the ABLE has a total of twelve items which give a grade level range of 3-9. This means that the student can gain or lose 1/2 year by simply getting one more item right or wrong."

Hall (1968) has a further criticism:

"Although the examiner is told that guessing is to be encouraged and no "correction formula" is to be used, the instructions to examinees are not sufficiently explicit on this point.“ (p271)

An even more serious problem is implied by the data disclosed by Nafziger et al: (1975):

"Reliability: split-half (odd-even) reliability coefficients adjusted by the Spearman-Brown formula are reported for grade 3 of the school group (.87 for vocabulary, .93 for reading, .95 for spelling), grade 4 of the school group (.89 for vocabulary, .93 for reading, .95 for spelling), the Job Corps group (.85 for vocabulary, .96 for reading, .96 for spelling) and a group of adult basic education students (.91 for vocabulary, .98 for reading, .94 for spelling)."*

[Footnote interpolated here, it having appeared at the foot of the page: * The context of Nafziger et al. is ambiguous: coefficients quoted are probably only those for Level 1. If this is so, the problem mentioned is a problem no more. Only a study of the test can tell for sure.]

These results are outstandingly high, and may well have been obtained by having tests made up of paired items: in the absence of a copy of the test, this must remain as the most probable explanation. If this proves to be a correct surmise, then students would tend to advance by two-mark steps, giving even more rapid gains on the vocabulary, reading and spelling test than for Arithmetic Problem Solving.

A specific objection to the ABLE grade—equivalents must also be raised: they are second-hand. The grade—equivalents on the Stanford Achievement Test of an elementary school sample have been used as the basis for the ABLE grade—equivalents. Hall (1968) reports that the correlation between the Stanford Paragraph Meaning subtest and the ABLE editing Level 2 subject is .58. In the following paragraph, he comments that “the authors wisely urge that local norms be developed by ABLE users.“ (p.273)

The “learning calendar" objections

When standardised tests are administered to a norming population, this is done at one time of the year. It is then taken for granted that 0.1 grades are gained in each of the nine USA school months, with a further 0.1 grade gain over the three month summer vacation.

Unfortunately for this assumption, as far back as 1968, Beggs and Hieronymus showed that there is a distinct loss of performance on many tests of skills over the summer vacation. Losses of two grades were quite common, and the trend was rather more marked in students of lower ability. ‘This loss is obviously retrieved in the early part the new school year, and augmented by the year's growth. Any teacher who tests students at the start and end of the year should be able to show a gain of about three grades during the year.

It is instructive to ponder the possible results of this effect operating on adults who have been absent from school for some years.

The unreliability of gain scores

Stake demonstrates that when two tests have reliabilities of 0.84, and correlation of 0.81, these being typical good values (but compare them with the figures for ABLE on page 61), the reliability of the gain scores will be 0.16.

The regression objections

The phenomenon of regression to the mean has been known for a century or so, but never sufficiently widely. when things vary, there are usually two main sources of variation. There are systematic causes, such as heredity, treatment, intelligence and so on, and there are chance factors such as assignment of teachers, diet, "luck of the draw" and so on.

Now in any test, some individuals will be at the high end of the distribution: this is because both the chance and the systematic factors have favoured them. Similarly, those at the "bottom of the pile" are there because both chance and systematic factors operated against them. If we take the "top" group and test them again, the chance factors (which are completely independent of the systematic factors) will, on average, neither advantage nor disadvantage the group.

Some will be favoured, some will suffer. But on the first test, most of them were favoured: that is how they ended in the top group. So the end result is that the "star" performers have given clear evidence of falling standards of exactly the sort that demagogues love to write. Or have they? Down at the bottom, the low group have shown an equivalent improvement. This is just the sort of growth that educational do-gooders love to clasp to their bosom and claim for their own.

We have now reached (I hope) that happy point where we may consider the claims and offers about PLATO that were laid before the educational community, confident that we have some the necessary gains of salt ready at our sides.

* * * * *

The marketing of PLATO passed in the late 70s to Control Data Corporation, and rather than just marketing the idea, CDC wished to sell programs as well as terminals and processing. The content area chosen matched a perceived need: basic skills, mostly for disadvantaged students of one sort or another.

Most of the studies seem to have involved one or more Control Data personnel, as do most of the available public documents. Through the good offices of Ms Lane Blume, Control Data Australia, I have been able to obtain a bound set of photocopies of what appear to be the papers collected by Dr Peter J Rizza, educational consultant to the Control Data Education Company at Minneapolis.

Some of these papers have authors, some do not. One is even labelled ‘NOT FOR PUBLICATION OR ATTRIBUTION" (Study 3). Tough luck, guys: I never agreed to that.

The papers total more than 300 pages, are incomplete, and quite possibly out of chronological order. I can only attempt to draw selections from these and the matching public documents, in the hope that a pattern will emerge. These papers deal with the PLATO system, as it was used to present the Basic Skills Learning System, or BSLS. These are supposed to be adult materials, but are they?

In January 1979, David F. Fry, Supervisor of Instructional Systems, wrote to Rizza and commented:

“Looking at the total BSLS package from the viewpoint of an instructional developer who has been shown the advertising claims and statements, I was a little disappointed. You should tell the brochure writers not to claim "multi-media package“ when the only other media provided is [sic] at best secondary and motivational. The texts were never "prescribed" nor were the video tapes, except for the first one. In my opinion the video tapes should not be used for adults. My students were embarrassed and uneasy when viewing the tapes. I had to use them in groups because the program never referred to them. The workbooks provided practice in working the problems, but were not adequate as alternate methods of instruction. They should be rewritten.“ (p.201)

This appears to imply that the materials were originally written for children: could it be that Control Data learned to hanker after a more lucrative market? Rizza and Caldwell are quite specific about the target, but while their paper is undated (other than a non-committal “1979"), the evidence of the ERIC Clearinghouse number implies a date late in 1979 (a point which will be reintroduced later in this paper).

Rizza and Walker-Hunter, dated January 1979, and so writing before Fry's letter, say the target population may be found “in a variety of settings: adult basic education centres, correctional institutes, and unemployment lines". Here we see less emphasis on an adult-centred system. Two of the major evaluation projects which were carried out in 1978 were centred on the use of BSLS in schools in Baltimore City and Florida. The claim that BSLS was written for adults does not appear to be wholly proven.

Study 1 in the CDC papers is actually a report on two studies carried out with adult learners in Baltimore City. Most, but not all, of the students showed gains in both reading and maths. 0f the 11 students who had completed the PLATO reading course, all had gained, with a mean gain of 0.8 grades. This is a depressed estimate, as two students in the post-test had reached the Grade 9 ceiling of the Level 2 ABLE test. The 13 students who had not completed also had a gain score of 0.8 grades, also a depressed estimate, for the same reason. On average, the non-completers had completed less than two of the five units on reading. If linear growth were predicated, this would imply an overall growth of 2.2 grades. The alternative possibility is that a Beggs and Hieronymus effect is working.

In mathematics, the completers performed better than the non-completers, and the relationship was roughly linear. The completers had gained 1.8 grades, the non-completers had gained 1.2 grades with two-thirds of the work completed. As the mean entry score of the completers was 6.3, and the mean entry score of the non-completers was 4.8 (grades), this result is surprising. Caldwell and Rizza (1979) state that the approach adopted by BSLS is a mastery one. If this is so, then all performers should come out at the same level, and so the lower group should show a greater gain. Of the 27 non-completers, 5* showed losses, one showed no gain, and four showed gains of less than 0.2.

[* The numeral 5 was missing in the presented paper in the previous line, but was found in the "acid drops".]

A summary of attrition levels is fairly impressive: of 135 enrollees, 8 are described as "dropped", while another 23 left under "extenuating circumstances“ (which are not defined). This is good, although possibly attributable in part to the novelty value of computer learning. Rizza and Walker-Hunter (1979) clearly see this as a strong point: "Attendance was good; the drop-out rate was only 6 percent…".

Study 3 (there is no study 2) also looks at the Baltimore Adult Learning Centre, and was received by Peter J Rizza (according to a stamp on the title page) on ' March 23, 1979. This is after the publication of Rizza and walker—Hunter (1979), and so it is not quoted there. The main interesting feature of this study is that some of the post-test scores exceed 9.0. A footnote on each page of the results tells us that

“Post-test scores of 9.0+ were estimated at the rate of 0.15 grade-level increase for each raw point above 53“. This did not need to be done with pre-test scores, since all students over 8.5 grades have been deleted from the study, and thus probably boosting the regression effect. The gain-scores are swelled by about 10% by this approach.

Study 4 is also on the use of PLATO in Baltimore, but this time, the users were school pupils in 7th grade. On page 90—a, we read "For 107 seventh graders, who averaged only thirteen hours each on PLATO, a mean gain score of 5.7 was found. (This was a raw score gain in terms of number of correct problems out of forty.)"

The test used was the Baltimore City Proficiency Test, and it was administered to all of the city's 6th and 8th graders, who showed gains of 4.5 and 3 respectively. (“The seventh grade test was not given system wide, so seventh grade comparison figures were not available ") If this means what it says, a separate test was used on each grade, so that gain scores cannot in any way be compared. And even if the same test is used, we do not have the norms to tell us what to expect of 7th grade. The author(s) use a t-test to show that the 7th grade result is significantly different from the 6th and 8th grade.

Page 95 shows us that students who had completed more of the PLATO course had higher gain scores. The possibility that both are influenced by some other factor (mathematical ability?) is not discussed.

One thing that can be said for this study is that there is probably not a Beggs and Hieronymus effect operating: the pre-test was in November, two months into the school year. In this context, it is curious to note that

“…almost all math students spent the majority of their time working to improve whole number skills. The forty-problem proficiency test used as the measure of achievement contained only four problems that required straight-forward whole number computational skills.“

A second paper appears to refer to the same study, but there are minor differences in the numbers. There are now 96 7th graders using PLATO, and their gain score is 6.13. There is also a control group of 47 with a gain score of 7.73 (in statistical terms, this is not significant: p = .14).

Results are also available for a senior high school group. The gain score for the control group was 3.11, while for the PLATO group it was 1.67. The PLATO gain appears to have come from the improvements for a few poor individuals:

The table, taken from the AARE paper.

Neither of these negative results is quoted by Rizza and Walker-Hunter (1979) or Caldwell and Rizza (1979). It should be stressed that these results are negative in that they do not show that PLATO instruction is significantly better; although they tend in that direction, these results do not show that PLATO instruction is significantly worse.

Study 5, on the other hand, is quoted (in part at least) by both Rizza and walker-Hunter and Caldwell and Rizza:

"Students at Stillwater gained an average of 1.6 grade levels in reading achievement and 2.16 grade levels in mathematics as measured by ABLE. Statistical analysis showed that gains in reading were significant even with small number of cases. (p .06)." (Rizza and Walker-Hunter, p.23).



Caldwell and Rizza supply this table:

The table, taken from the AARE paper.

These results refer to reading only, and several things need to be said. In the first place, the Stillwater control group consisted of three individuals, while the experimental group consisted of 5. Secondly, the PLATO group contains two individuals with post-test scores in excess of 9.0. Park's bibliography lists the Adult Basic Learning Examination as the only test used, and the date is given as 1967. This precludes the possibility that the more recent ABLE Level 3 was used as a post-test, so there appears to be no justification for scores greater than 9.0. Interestingly, 2 of the 3 members of the control group reached 9.0, but did not progress beyond, while the third member lost ground in both reading and mathematics. The gain must be reduced to a more conservative 0.7+.

The mathematics gain of 2.16 must also be reduced, since post-test grades of 12.4 and 10.1 appear. The best estimate becomes 1.26+.

Before leaving Caldwell and Rizza's table, it is worth quoting Park. Perhaps this explains the zero gain score for the "Fair Break" group:

“The Fair Break group all had access to terminals and the teachers were unable to provide facilities for a control group." (p.147) and “There were no controls in the Fair Break Learning Center..." (p.148).

[Interpolated comment: at this point in my presentation, I raised my eyebrows and said, very slowly, "There was no control group."  My audience got it, and I guess if you have read this far, you will have got it as well. Yes, I'm not a nice person.]

* * * * *


In Rizza and Walker—Hunter (dated January 1979, hence written in late 1978) we read: "At the Adult Learning Centre and the Fair Break Learning Centre, adults referred by city training programs were able to achieve measurable progress in both reading and math. Due to the lack of a control group, it was difficult to show the gains to be statistically significant." (emphasis added)

Park also undertook a similar small study at willow River, with 7 in the PLATO group and 3 controls. This produced anomalous results, and this may be why neither Rizza and Walker-Hunter or Caldwell and Rizza reported it. The PLATO students lost 0.3 grades in reading while the controls gained 0.2. The PLATO students gained 0.5 grades in mathematics while the control group gained 0.36. The total time given over to study for all students appears to have been only about six hours. One PLATO group pre-test score is stated as 9.2, but this not explained.

The Fair Break study has already been mentioned in the context of the control group that never was. The raw data make interesting reading, especially in the context of an internal Control Data memo from Peggy Walker-Hunter to Peter Rizza which is attached. A copy of Park's Table 4 is also attached.

The second paragraph tells us that Level 3 of ABLE was in fact used in the Fair Break project as a post—test, but not as a pre-test: "...it is still impossible to establish a grade level gain when the pre test is inaccurate." Again, in paragraph 3 we find that times were not recorded for the St Paul students: "...staff had to look at group records and guess at the amount of time spent in each curriculum. In some cases, it was just too difficult to determine,". No blanks appear in Park's Table 4, and the same figure (11 hours) is quoted by both Rizza and Walker-Hunter (published January 1979) and Caldwell and Rizza (1979, no month, but submitted to ERIC in late 1979, on the basis of clearinghouse accession numbers). Walker—Hunter's memo is dated 7th March, 1979.

[Interpolated comment: I was also submitting material to ERIC in 1979, and I kept meticulous records of my submissions, and as I had my (and Caldwell and Rizza's) accession numbers, I had a very good idea of submission dates. One of my submissions went off by air mail from Australia in late 1979, and their paper had a higher accession number, so it arrived later. These are the trivia that catch shonky operators out.]

In paragraph 4, Walker-Hunter writes "...there emerge only eight students with accurate pre- and post-test scores and time data in reading." (This was from a starting total of 38.) Then in paragraph 5, we read

"In view of this situation, I simply determined the average entry and exit level of the students (eliminating those with “9+" scores either pre or post) and computed the average grade level gain, ignoring time on task altogether.

The table, taken from the AARE paper.

There are several notable things in this quotation. The eight become fourteen, probably because time data have been ignored. And the post-test reading mean is 9.15, when all individuals over 9.0 have been deleted. "

It appears impossible to reconcile Walker-Hunter's quoted calculations with the results which are appended to her memo, or to Park's Table Four. Park's eight students would not, one would expect, have time data which are valid. There are seven students so noted in Walker-Hunter's data. Park's participant 3 appears to be Walker-Hunter's 022, and Park's 6 appears to be Walker-Hunter's 001. If this is so, then why are Park's pre-test scores given as 9.0 instead of 9.0+? Park's 8 looks a bit like 005, Park's 1 is like 009, her 5 could be 013, possibly her 7 is 002. But there are discrepancies, and the match gets worse as we proceed.

Rizza and Walker-Hunter had claimed

"Students gained an average of 1.8 grade levels in reading and 2.6 grade levels in mathematics. Both gains were statistically significant." (p.23)

This is much better than Walker-Hunter's 0.62 and 1.9, figures which do not seem to have been made available in any scholarly or promotional publication.

Interestingly, Caldwell and Rizza comment on the Stillwater and Fairbreak projects: "Each site utilized approximately twenty (20) students. .". Park's Table 2 (p.157) and Table 4 (p.159) shows that there are results for only eight (8) students in each case.

The most important point, though, is the discrepancy between the "experimental" and "control" groups in the Stillwater project. The "control" group had a significantly poorer performance on the mathematics pre-test (p = .008) than did the "experimental" group, even though the numbers were so small.

Study 7 was the work of Fairweather, which we encountered briefly. His most biting criticism was over hardware issues, but he also had doubts about the suitability of the material:

"Repeatedly, certain inmates needed convincing that the Basic Skills materials were designed for adults and that they were part of a continuum that led to the high school equivalency certificate. Although the inmates responded well to the animations the benefits of the graphics were offset by the perception that the materials were inappropriate for study by adults." (p.179)

This is one study which recognises the "...problems involved in using gain scores to evaluate a project of this sort..." (p181) but pleads that "...he did not have time to design a mancova program." (p.181).

One of the most unusual aspects of this study is that the researcher calculated a series of regression equations to fit PLATO study time to learning gains. The clear implication of these equations on page 183 (copy attached) is that one gains about one and one half grades on each of reading vocabulary and spelling before even touching the keyboard! In the case of spelling, a loss is incurred which increases with exposure to PLATO. If we ignore bizarre temporal theories, we are left with two possibilities. Geof Hawke (pers. comm.) argues that the linear regression model is probably wrong, and if it were correct, it ought to be forced through the origin. My own view is that the y-intercept indicates the operation of the Beggs and Hieronymus effect in adult learners. In view of the short term involved, natural maturation may be rejected.

Study 9 relates to remedial mathematics for college students unable to meet college requirements. An existing program, using hand-held calculators. The challenge to PLATO here was to match the results of a well-thought-out program designed to meet certain objectives which might not be found in the BSLS system. The result was that PLATO came off second-best, except in the area of ABLE word problems in arithmetic. (It has not previously been mentioned that there are two separate ABLE arithmetic scales.)

This was curious, in that the "Calculator Basic" students were drilled in word problems, while the PLATO students were drilled in computation.

This report does not offer sufficient data for any real analysis, but it appears that when pre-determined objectives are to be taught, PLATO may prove relatively inefficient.

Studies 10 and 11 relate to schools use, and have no data of note.

In conclusion, at the time of my study, there was not one study which compared PLATO with an equally expensive traditional system. There was not one study in which a properly controlled comparison took place. There was not one study which was written up, complete with data, in the professional literature. And there was not one valid study showing PLATO to be better than traditional approaches. The potential is there, but I do not believe that it has yet been realised.

REFERENCES

Beggs, Donald L., and Hieronymus, Albert N., Uniformity of growth in the basic skills throughout the school year and during the summer. Journal of Educational Measurement 5(2), 1968, 91-97.

Caldwell, Robert M and Rizza, Peter J. A Computer Based System of Reading Instruction for Adult Non-readers. ED 184 554, 1979.

Control Data Corporation: Basic Skills Learning System: Evaluation Report: May 1979. (No other details supplied.)

Fairweather, Peter (1978): See Control Data Corporation.

Fry, Edward 8., untitled review, excerpted in Buros, O.K., The Seventh Mental Measurements Year book. New Jersey: The Gryphon Press, 1972.

Fry, David F. (1979); See Control Data Corporation.

Galen, Nancy: Informal Reading Inventories for Adults: An Analysis, Lifelong Learning: the adult years, 3(7), 1980, 10-14.

Hall, James N., The Adult Basic Learning Examination. Journal of Educational Measurement, 5(3), 1968, 271-274

Hieronymus, A. N. Review of Levels 1 and 2, Adult Basic Learning Examination in Buros, O.K., The Seventh Mental Measurement Yearbook. New Jersey: The Gryphon Press, 1972.

Nafziger et al. Tests of Functional Adult Literacy: an Evaluation of Currently Available Instruments. Portland, Oregon: Northwest Regional Education Laboratory, 1975.

Park, Rosemarie ( ): See Control Data Corporation.

Rizza Peter J. and walker-Hunter, Peggy, New Technology Solves an Old Problem: Functional Illiteracy. Audiovisual Instruction 24(1), 1979, 22-23, 63.

Stake, Robert E. Measuring What Students Learn, in House, Ernest R. (ed.) School Evaluation: The Politics and Process, Berkeley: McCutchen Publishing Corporation, 1973, pp. 193-223.

Walker-Hunter, Peggy (1979): See Control Data Corporation.

The above material is a lightly-edited and annotated version of a paper delivered to AARE in 1986 in Melbourne. Tables are taken from original material in my possession.

Disclaimer:

This text was converted from the paper read to AARE using OCR, and in a late stage of checking, the phrase "post hog ergo propter hog"  was detected. After a struggle with my conscience (I decline to say who won), I amended this.  I remain uncertain that the initial version was not more apposite, and I suspect this may well be the view of the majority of those who have read my account of such an inept and fraudulent evaluation.

Are you one of the players?

By the way, if you were involved and you are thinking of taking legal action, this is just a small sample of what you will have to justify. I have left your name out, for now, but that doesn't mean I don't know it. If you take action in any way whatsoever, to annoy me, I will mount a truth and public benefit defence and name you.

The choice is yours, the pleasure will be mine.

Here is a table that will show you what went down, and as a sample of what I hold. If you know your numbers, this shrieks. If you don't, look at the average of the reading pre-test scores, look at the alleged gain scores of participants 1, 2 and 4. This was either incompetent or fraudulent, and letting the raw data out shows gross stupidity.

Fraudo the Frog?

This was the very first Ockham's Razor talk that I ever did, back in 1985, as near as I can recall — I wrote it on my trusty Commodore PET, with 32k of RAM. I found something odd, rang Robyn Williams, expecting that he would do an interview with me, and instead he asked me to write a script. "If it goes 13 minutes and 20 seconds, we'll use it in Ockham's," he said. "If it;s longer or shorter, we'll use it in the Science Show."

It seems that with intro and outro and other bits, the ideal time for a 15-minute talk is 13 minutes and twenty seconds: for a fast talker like myself, that is about 2300 words: enough to develop a reasonable thesis, and so I have been drawn, over the years, to return to the format, over and over again. But this one has always been a favourite: it appeared in the second collection of Ockham's talks that was printed, in 1987. I have slowed down my delivery rate a bit over the years, so I now aim for 2150 words,

This is the second of a short series on fraud that I am posting to close out August. The first was on a hoax that I pulled — there is a family relationship between hoaxes and frauds, the difference being the intent of the perpetrator. The third is an account of a major fraud that I uncovered in 1981, which I am publishing under the 30-year rule, later this week. A fourth case study covers my use of hoaxing to counter a fraud in 1985, not long after this piece went to air. That one will not be released until next year, again under the 30-year rule. There was one other case. That one involved criminal actions and some violence (not to me). It will never appear in print, but I can occasionally be coaxed to expound on it over a beer, if only to explain why I went into a safer line of business.

  * * * * *

All tribes have their myths. The young are brought up on these myths, and they are expected to live by these myths. The strongest of all the myths of the Science Tribe is the one about the Scientific Method. The elders say that a scientist starts out with a particular idea or rule about how things work, variously called a Theory, a Law, a Principle, an Hypothesis, or even a Conjecture.

By careful consideration, the scientist is then able to make certain predictions about what will happen if the idea is right, given some new assumption. Then all that is needed is a couple of quick experiments to see how the scientist's predictions stand up. Before long the rule, along with its assumptions, can be judged, and given either the All-Clear, or the Order of the Boot.

Not that the All-Clear necessarily means that the rule is correct, they explain. All we can really say is that such-and-such seems a bit more likely, or seems to be an acceptable approximation. Newton's Laws are still Laws, Einstein notwithstanding, because Newton lets us send rockets to the Moon, Mars, Halley's Comet, and beyond, without needing to say, "left hand down a bit to allow for relatively".

In the real world, the myths don't work. Laws don't spring, fully-formed, from the sweaty brows of scientists. Laws start out with somebody doing a bit of data-snooping.

Data-snooping involves making lists of measurements, and poring over them to see if there is any mathematical relationship or pattern or trend that might give us a hint about the rule that lies beneath the measurements. We assume that there is some rule, but in the absence of any real knowledge, we must try all the tricks.

Measured values, their squares and cubes, square roots and cube roots, their products, progressions, logarithms, sines, tangents, and other exotic mathematical functions are all thrown in, even fractions involving combinations of functions. Heavy stuff, but well worth it, if only we can make a breakthrough.

If there is a pattern, the next step is to explore the relationship further. Are there any missing values in the range of values studied? Can we extrapolate beyond the range? If we can, we must predict some values, and then go looking for them.

Johann Balmer did this when he found a relationship linking four of the hydrogen lines in the visible spectrum. It seemed that there should be another line, right on the edge of the ultraviolet, a line of which he had no prior knowledge. When he checked, the predicted line was there, just where he said it would be, and Balmer's rather odd little equation was confirmed. Score one point to data snooping.

Sometimes, though, the confirmation can be misleading, as happened in the case of Bode's Law. Now before we start, Bode's Law isn't a Law, and it wasn't even Bode's: other people had said it before him. Those problems apart, it isn't a bad sort of Bode's Law: in fact, it is the best Bode's Law that we'll ever have.

Start with the numbers zero, one, two, four, eight, and so on, triple each number, add four, and then divide by ten. This recipe gave Bode a series of numbers that closely matched the orbits of the known planets, measured in astronomic units. One astronomic unit, of course, is the distance from the Earth to the Sun.

Bode's values had no logical cause, there was one value with no matching planet, and there were slight discrepancies, but this was basic data-snooping, and the fit was quite impressive. Mercury is at 0.39, while Bode predicted 0.4, Venus is at 0.72, Bode predicted 0.7, and Earth is exactly at Bode's 1.0 position.

Mars is at 1.52, while Bode's Law says 1.6, there was nothing found at Bode's next point, 2.8, but Jupiter is spot on at 5.2, and Saturn is at 9.54, not far off Bode's value of 10 astronomic units. It seemed a terrible pity about that gap at 2.8. Maybe that was why most people ignored Bode's Law at first.

Then Herschel found Uranus 19.2 A.U. from the sun, close to Bode's next value, 19.6. So people looked at the gap between Mars and Jupiter again, and found the asteroid Ceres, at 2.77 A.U. Later, other asteroids were found, leading people to believe for a while that the asteroids were the remains of a broken-up planet.

Bode's Law, as I said before, is not a Law of Science. Even though it's an elegant pattern, and even though it predicted unknown events, both inside and outside the range of observations, Bode's Law was of no great use, and the pattern could not be tied in with any theory of Why Things Are.

Maybe we could ignore Neptune which fails to fit the pattern, squeeze Pluto into the next value, and look for Planet X at the value after that, but we don't. There seems to be no future in doing it, and it's hard to fiddle with figures which are accurately and publicly known.

That's the problem with data-snooping, though: there will always be a temptation to bend the facts. The data are often derived from experiments under the investigator's control, and if it helps to make the data fit better, well why not? We can even give it a more likeable name: let's call it fudging.

Now fudging can vary from unconsciously biased observation through massaging the data to outright fraud. Sometimes the fudging is legitimate, as when R.A. Millikan, of Oil-Drop Experiment fame, practised and practised until he got his technique right, and his results consistent. Millikan ignored his early results, and said so. You couldn't really say he cheated.

Mendel rejected one set of results, repeated the experiment, accepted the second set of figures, and said so. Mendel also fiddled mildly with a few other things and said nothing. However charitably you look at it, Mendel came perilously close to cheating.

Pierre Dulong

Dulong and Petit massaged their data, and faked some more. They said nothing about this: there can be no doubt at all that they cheated. This is a serious charge, so I shall devote the rest of this talk to proving my case.

Dulong and Petit came on the chemical scene in 1819 with a law that linked atomic weights and specific heats. All elements, they said in effect, had the same heat capacity, about 25 joules per mole per Kelvin. This was a useful approximation at the time, but it hasn't been needed much since about 1830. Maybe that's why their skulduggery has gone unremarked till now.

Their Law was useful in the 1820s because proper chemical theories needed the accurate atomic weights, or at least accurate comparative atomic weights. Structures, formulae, valency, the Periodic Table were all impossible until these basic values were worked out.

In the early days, the atomic weights of metals were determined by a variety of methods, including electrolysis of soluble salts, and the reduction of metal oxides to metals. The problem of multiple valencies meant that there were conflicting values available for a number of metals. Nobody knew which values to take: was copper twice as heavy as oxygen, or was it four times as heavy? There seemed to be no solution.

Up bobbed Dulong and Petit with a law that made the measurement of atomic weight a simple matter. The Gallic duo noted that specific heats were easy to determine, and advised that the product of atomic weight and specific heat was a constant for all elements. They even presented a table of values to show how this worked out.

Now their claim was better made for metallic elements: the only non-metal in their table was sulfur. On their data, it sort of fits, but with modern data, it sticks out like a sore thumb on the butcher's scales.

Their data are not immediately examinable: for one thing, the atomic weights are all given in terms of oxygen, which is taken to have a value of one. So when I started to examine Dulong and Petit's figures, I suspected nothing more than a bit of minor fudging, and I certainly found that quickly enough.

Putting it simply, where the product of atomic weight and specific heat is too low on modern data, Dulong and Petit's specific heat value is too high. Where the product is higher than the average, their specific heat value is too low.

This could be excused by saying that the measured values for specific heat in 1819 weren't fixed: even at the end of the nineteenth century, published values varied quite a lot. There are just two give-aways in the table, two uncontestable whistle-blowings that tell us what was really going on.

The first of these is the information on cobalt. This, we are assured, has a relative weight of 2.46, and a specific heat of 0.1498. Translating the relative weight, we get an atomic weight of about 39.36. That wasn't right, I thought, reaching for the CRC Handbook. Sure enough, it gives an atomic weight for cobalt of 58.93, almost 50% higher.

When I found this, I looked again at the specific heats quoted by Dulong and Petit and blow me down! the modern figure was only two-thirds the one they used. How fortunate they were, two cancelling errors like that! Curious, too, seeing that cobalt has valencies of 2 and 3, so that the error in the relative weight is almost, well, totally predictable. What a shame that the same can't be said for the error in the specific heat.

In 1981, I had the pleasure of unravelling a fraudulent set of evaluations of a computer-based education system, just at the time when I was first reading Eugene Kamin's marvellous expos‚ of Cyril Burt's work.

I came to the part where Kamin suggests that "a benign Providence appears to have smiled on Professor Burt's labours". I fell in love with the expression, and I had used several variations on it. Now seemed to be the time to revive that phrase. I started to look more carefully at the rest of the data.

I am not, I must confess, familiar with the chemistry of tellurium, and I wasn't really expecting another find. But there it was: the relative weight of tellurium, 4.03, gives us an atomic weight of 64.5, about half the accepted modern value of 127.6. And would you believe it? The specific heat had gone from today's .048 to .0912, almost double. Funny, that, especially as tellurium has valencies of 2 and 4.

I still can't explain why they had problems with cobalt: measuring its specific heat should be quite straightforward. But I think I can shed a bit of light on the tellurium problem. You see, tellurium is unpleasant stuff, and it gets absorbed through the skin, ever so easily. So do its compounds. And once you've absorbed the tellurium, you exhibit something that the CRC Handbook calls "tellurium breath". You smell of old garlic, and it lasts for months. So I wouldn't really blame any experimenter who chose to avoid tellurium in the lab, and no bad jokes about French food either, thanks.

So there we have it. Just to nail it home, I calculated the correlation coefficient of the specific heat fudge factors against the shift needed to get a perfect fit, and got a correlation coefficient of -.71, which is significant at the 1% level, and consistent with the other evidence.

I concluded that the two scientists massaged their data. They made two errors in atomic weights, each consistent with a wrong guess at the valency of a metal, and in each case, the specific heat was grossly wrong, and just happened to cancel out the errors. This was even less likely than the convenient little errors elsewhere. And if they guessed wrongly on cobalt and tellurium, how many of the other figures were just luckier guesses?

Overall, I think that their practices stink worse than tellurium breath, but does it matter? Their "Law" helped the early chemists to bypass with confidence a sticky problem, so even if they did cheat, it served a useful purpose. I can't help wondering, though: how many of our cherished and established facts of science were born of similar fraud?

Additional data, not in the talk, for obvious reasons

The things to look out for here, the tell-tale signs of cheating are in bold:

* the 'fudging' of the specific heats of lead and sulfur, to make them fit better, and

* the serious errors in the figures for tellurium and cobalt, matched by errors the other way in the specific heats.

Keeping savages in a cage

I am slowly collecting together my past radio broadcasts, so they have a permanent home. This one went to air in about 1995. It is a true story in all respects, but I have since moved on.

People who know me well often have trouble trying to pin down my profession. Indeed, I sometimes have trouble defining that myself. Every two or three years, I change, I move on to new horizons, or sometimes, I even move back to old ones.

You see, there're quite a few things that I enjoy doing, and I believe in seeking out jobs that'll let me do as many of them as possible. Mainly, I enjoy anything to do with the sciences, teaching young people, messing about with computers, and writing. Most of my shifts in employment have been associated with a varying urge to concentrate on one or another of these four interests.

So for the past three years, I've been an education officer at the Australian Museum, but now I've returned to classroom teaching. I've got my own students, for better or for worse, and the chance to watch them develop under my guidance. Further, I've elected to work in the area of Computing Studies, introducing students to the effective use of the computer as a means to higher ends, rather than as an end in itself.

As part of this, I have been working with my Year 9 students on the development of writing skills on the word processor. They have been taught to write in a certain way by their English teachers, and now I'm teaching them an entirely different process. The word processor is a useful tool for writing, but only once you realise that it is neither a pen and paper, nor even a typewriter. It requires a very different approach to planning, drafting and writing.

Their task has been to write news stories about local events for a network called Global Village News, part of a worldwide educational network called K12Net. When their stories are ready, we send them by modem over a telephone line to a local centre. Over the next couple of days, that centre connects with other centres around the world, and our stories trickle through at off-peak rates, sponsored by the local electricity supply people in New South Wales. Students in the USA, New Zealand, Germany, Canada, South Africa, Aruba and beyond, can read our stories, just as we read theirs.

Several of the boys in my class decided independently to tell the story of a man who died in the Corso, the main street of Manly, just down the hill from our school. The man's body lay there for a considerable time before anybody realised the man was dead. "We thought he was just sleeping it off", people said of the man, apparently a heavy drinker.

One of the students was appalled at what seemed to be a complete lack of attention or compassion on the part of passers-by. "How could people do it?" they asked me. "How could people be so uncaring, so lacking in curiosity?"

"People," I assured them, "are capable of ignoring many strange and curious things." The bell had gone, and a few students had stayed behind to finish things. As we were in our own time, I sat down to reminisce about the time I kept and displayed two savages in a cage at the Australian Museum.

It was late 1992 when I did this. It was busy, just before Christmas, with everybody busy doing last-minute shopping, and the Sydney Biennale was on as well, so it was hardly surprising that my students had never heard of the visit to Sydney of two Guatinaui, colourful savages from the island of Guatinau, a little-known island in the Caribbean.

Coincidentally, I said, the visiting display reached us in the week of the bicentenary of Bennelong and Yammerawannie being taken off to London to be shown to King George the Third. "Yammerawannie?" asked my students, "Who was that?"

He was, I explained, the less successful exhibit, for he died of pneumonia in Britain, and so never returned to his native shore. Still, I added, King George was quite taken with them. It probably made a change for George from being an exhibition himself.

"Because he was a Royal?" one of them asked.

"Because he was mad," I replied. "People used to go to Bedlam, the insane asylum, to laugh at the inmates, and a mad king was even more fun. people used to say that when he had an argument with a tree, the tree usually won."

They tittered dutifully, but one of them looked a bit concerned. "I'm glad we don't accept that sort of behaviour today . . ."

"True," I said, "but we do accept displaying savages in a cage still. Nobody gets upset by that, or not in my experience." Then I started to explain how two outrageously colourful Guatinaui, from the island of Guatinau, were displayed in a golden cage at the Australian Museum for three days.

I was open with my students from the start that the display was a hoax. It was a Biennale event, undertaken by two performance artists, intended to provoke a public reaction, but so far as the Australian public was concerned, the "Year of the White Bear" was the genuine article. Like any good hoax, everything was over the top, so that the more perceptive would see through our hoax, but at no time would we admit that it was anything other than genuine. The possessions of the savages included a laptop computer, but even that could be glibly expalined away.

At the end of 1992, the International Year for the World's Indigenous Peoples was just a few days away, and three young Koori men took it in turns to watch over events, ready to intervene if anybody became too worked-up about the display, but their presence was never needed. Nobody, but nobody, objected.

The "savages" are two American performance artists, Coco Fusco and Guillermo Gomez-Pena, and I first heard of them in a Radio National program in mid-1992, talking about people's racial stereotypes, and how they, as Latinos, were perceived by Hollywood and Anglo-America generally, and what it was like to be inside the cage.

As soon as I heard that they were coming to the Museum, I volunteered to be one of the guards outside the cage. It was a role that required a story-teller with a consummate flair for telling lies and tall tales with a straight face, I said. Faced with that consideration, the Museum management waited an unflattering three milliseconds, referred briefly to my petty cash claims, and signed me on. They also accepted Karen, an American lady who was working as a volunteer in the Museum's Arachnology section.

Coco and Guillermo wore, when on display, swimming costumes, sneakers and sunglasses, while she had a wig and yellow face paint, and he wore a Mexican wrestler's mask. They also wore collars, to which we would attach a chain when they left the cage ("for their own safety", we would explain officiously)

They also added various bits and pieces to the kit, but that was their basic ensemble. It was more than enough to make them unrecognisable when they went out at night, or when Coco spoke on radio as an "anthropologist" travelling with the specimens.

Inside the cage, Coco and Guillermo were unable to communicate with the public, and so were free to observe public reactions from behind their Raybans. The guards, on the other hand, were to present the public face of the display. The "savages" could speak no English, but they could communicate with us by sign language. If they wanted to hear a conversation, they would signal to be fed, if they wanted to brief us, they would signal to be taken to the toilet, and then talk to us outside, before we led them back to the cage.

As "guards", we were dressed in white overalls, and we presented as two ignorant people who had picked up a certain amount of hearsay information, and who knew the general details. If pressed, we would encourage the visitor to read the text displayed in front of the cage. We were, you see, just the guards, and the anthropologists who had all the answers were away at that time. We were calculatedly provocative and patronising in our comments, as we had been instructed to be.

If attacked by unbelievers, we would simply say something like "I can assure you, sir (or madam) that the Guatinaui are just as genuine as their island, and you can see that in the map over there." Confronted with impressive cartographic evidence, said to be taken from the Encyclopaedia Britannica, most doubters were effectively silenced.

If challenged as to why the Guatinaui were wearing Raybans, or sneakers, or why the male specimen was wearing surf shorts, we were told by Coco and Guillermo to state simply that theirs was a syncretic culture which happily absorbed all sorts of elements from other cultures, if the people saw fit to adopt them.

As to how they had collected so much "outside" material while remaining undiscovered and unspoiled, that was for me to explain. It was, I would say, a result of the interplay between circumpolar effects derived from the Gulf Stream and an intermittent form of inadvertent Coriolis forces. Clearly, I would conclude, a situation which would result in currents going towards the island, but never away, so that drifting objects and castaways might be carried there, but nobody could ever leave. Faced with such a wealth of technical detail, most questioners wisely fled.

The third day was the Saturday before Christmas, and the crowds were light around lunchtime. Our specimens decided that we should take them for a walk through the busy shops, and down to the Art Gallery. We did so, but it was an almost total failure, as we saw it, for the video taken from ten metres behind showed almost no reaction. A teenager who works in the Museum told us later that she followed to see what happened, and that people seemed to react after about thirty seconds, which she put down to their preoccupation with shopping.

In the end all we got was somebody saying that was what he wanted to wear to the party, and a lady spruiking outside a bookshop, who asked what we were selling. When I explained that these were two savages who were on display in the Australian Museum, she told the crowd about this, straight-faced, but once again, nobody twitched or looked shocked.

Outside the cage, we had the "Biennale crowd", an in-group who came to watch the public being fooled, and who were all so cool about their heightened awareness. On one occasion, I launched into an impassioned speech about how we had hopes that the savages would eventually, given their natural sense of rhythm, become civilised like us, and capture their own savages to put in cages.

I put everything into it, playing to my audience, and discovering, too late, a couple in their sixties. But I needn't have worried: seeing me looking in their direction, they beamed their appreciation of what we were doing in helping the savages to rise to that level. Nothing, but nothing, upsets the Australian public!

Me with the savages. They specified the dark glasses:
it helped to avoid eye contact, they said.

One of the "givens" of quaint natives everywhere is that they are colourful, they tell amusing (to us, anyway) legends, and they dance exotic dances.

Our natives were prepared to pose for a photo with the public, using a Polaroid camera for one dollar. (The small fee was to be used to defray costs and to buy things to take back to their island.)


For the same fee, the female would perform an exotic dance to rap music, or the male specimen would tell a story about one of the totemic objects that he kept in a black briefcase which had syncretically become part of his culture.

We don't put savages in cages any more, I said. Not, I suspect, because we're more civilised, but rather more because we can go and see the natives in their native habitats, or we can sit at home and have them displayed upon our television screens.

But if we did still practise the exhibition of quaint people in cages, would we worry about it? I would have thought so once, but not any more. Especially after I heard Paul Berents speaking of how supinely we accepted the actions of the so-called Aborigines' Protection Board, right up until 1969, effectively abducting children and fostering them out to "more civilised" white families. After that, i could believe almost anything of modern Australian society.

One of the boys suggested that it might have been like that in Nazi Germany, that this might explain the events he had seen depicted in Schindler's list. I still prefer to think that we had an audience who saw the hoax for what it was, and played along with our piece of theatre. But I know that there were also those who accepted all that they heard and saw with complete acceptance. It worried me, I said.

"And when it was all over," asked one of the more perceptive boys, "what did it all prove?"

"Not a great deal," I had to confess. "Maybe it showed that we, as a culture, are prepared to accept all sorts of things, so long as they don't threaten our personal comfort. Australians were far more willing to accept this sort of behaviour than American and Spanish cultures."

"But what did Coco and Guillermo make of it all?" he persisted.

"I've no idea," I answered. "You see, by the time we got round to evaluating the experience, Coco and Guillermo had taken off to savour the quaint and exotic natives of Bali in their local habitat."

This one will also be found in one of the Ockham's Razor book collections.

Flash Jack goes droving

The thing about Flash Jack, he's a terrible liar, not that he'd call it anything as plain as that.  He'd just say he was a bit inclined to spin a yarn now and then, but other people don't always agree.  They'd tell you straight out that if Flash Jack bumped into the truth on a sunny day, he'd walk straight past, without so much as a flicker of recognition.

He was always a bit that way, but the tendency to exaggerate got worse when he was a bit past his prime.  Take the night when he was quite old, and some city bloke asked him if he'd ever been droving.  "Droving?" Jack said, "Droving?  I've drove cattle across every border in Australia, I've taken them from the east coast to the west coast, and from Cape York to Hobart."

Now this city bloke was no fool.  He even knew that there's a big lump of ocean, the Bass Strait, separating Hobart and the rest of Tasmania from Cape York and the rest of Australia, so naturally he challenged Jack, saying "Wouldn't your cattle have got a bit wet, mate?"

"Naaah," said Jack.  "We went the other way round."

Anyhow, you could see that this bloke, while he was pretty well bamboozled by Jack's reply, wasn't what you would call completely convinced.  In fact you could see he was pretty sure there was something fishy, and I don't mean tuna from the Bass Strait, neither.  So he kept asking these difficult sort of questions, and Jack kept answering them, and in the end the bloke gave up, but when he did, another bloke chipped in.  "What was the hardest droving job you ever did, Jack?"

Quick as you like, Jack answered that one.  "That'd be the time I took four hundred head through the Speewah back desert, across the sucking swamps, and then down to the big smoke."

"That doesn't sound like much of a job to me," said the bloke.  "That isn't exactly what you'd call a large mob, is it?"

"It is when the four hundred head are all 44-gallon drums," said Jack.

"Well, I don't believe that for a minute!" said the bloke, and a few other voices joined in, saying the same thing.

"It's the honest truth," said Jack.  "A mate of mine found out that all these drums were coming out full from the city, and then just being dumped anywhere, and back in the big smoke, they were worth a packet.  Anyhow, Smiling Annie was heading that way with her daughter Alice, and they reckoned they'd had a few domesticated drums in running with the chooks one time when they were staying in town in Bandywallop, and if we had a couple of those at the front, they said, the rest of the mob'd follow on as easy as you like.  Anyhow, we just started with a few drums, and built the herd up as we went along, but I'm afraid we had a lot to learn."

"What, about being truthful?" asks the bloke who'd started it all, but Jack just took that in his stride.

"No, about droving drums.  For a start, we should've taken ear plugs so's we could sleep at night — you don't need to put bells on them drums, I can tell you.  And we shouldn't have gone along parallel to the railway line out of Bandywallop, up towards the Speewah spur line.  It was an easy track to follow, nice and clear of obstructions, and fenced on the railway side, so we could keep them penned in, but we hadn't allowed for the train coming through.

"Scattered the drums all over the place, that did, and it took us three days to round them all up again, and I'll tell you now, the sound of several hundred fully-grown drums stampeding over stony ground is one I don't want to hear again.  Wouldn't've got them in that quick if Crooked Mick hadn't shown up with his dog and a couple of pups, and he later lent us the pups for the trip, luckily.  We wouldn't've stood a chance in the sucking swamps without them.

"Anyhow, we got them together again, ran 'em through Yandackworroby as easy as you please — Mick's pups had 'em marching in three files by then, so Alice and Annie and me stopped for a bit of refreshment while the pups paraded the drums through town.  We were feeling pretty pleased with ourselves till we got to the other side and found a stock inspector waiting for us, a real mongrel who reckoned them drums had to be dipped against rust and five different diseases before they went any further.

"No worries, though.  The pups dug a trench as soon as they realised what was needed, and Annie and Alice butchered a couple of poor-looking stray drums that'd joined the herd, just before Yandackworroby, and boiled up a mean sort of drum dip in them, stuff that'd make any self-respecting disease run away, stuff that stuck to the drums and kept the rust off as well, and poured it into the pups' trench.  Then we had to send 'em though."

"They would've floated, wouldn't they?" asked the troublesome questioner.

"I was just getting to that.  Yeah, they float all right, but if you tie their legs together, and wrap a rope four times round, then drag 'em though with the rope, that spins 'em and gets all sides.  You're right — there was no way we could make them go under that dip.  One good thing, though.  The pups took a strong dislike to this stock inspector, and the one that was doing the pulling would drag each drum over near the inspector while the other one nipped the leg tie and released the drum, so it was free to shake itself.  Catching that inspector for the first few was a bit hard, but after the tenth, he brushed against a tree and got stuck to it, and the rest of them were easy.  By the end, he was no longer really visible.

"Anyhow, we got them out on the other side, and the stock inspector being in no position to say much, we headed off.  Along the way, we collected a few mean-looking scrub drums, but they didn't seem to be doing no harm, so we let them come along.  Looking back, it was stupid, but we just didn't know, and things were really going well. You see, the pups had conceived this idea of making the drums butt each others tails, so they were stuck together in cylinders of fifteen or twenty drums each, which was great in the open country, but got nasty when we hit the Blue Mulga, but that was when the thunderstorm hit, and that stampeded them straight through the middle and out before the rain hit and let them split up.  That's why the road there goes so straight — it was them drums, boring through, that laid down the line of it.

"Well after that, it was fairly easy going till we hit the sucking swamps, when we had to rope them together so they'd go in single file.  That way, if one of them fell in, the ones in front just leaned forward, and the ones at the back leaned back a bit, and pulled the lost one out.  Then just as we got clear of the swamps, and thought we were on the home stretch, some of them drums began to calve, and we were stuck once again, surrounded by the strangest mix of billy cans, jerry cans, and even a watering can, all tangling around our feet, and they never stopped clanking.  I don't know where them scrub drums came from, but they'd obviously had a very mixed pedigree.  Anyhow, we had to wait until they grew up into 4-gallon drums, then we drove the lot down to the big smoke, and cleaned up.  But it wasn't easy money, I can tell you that."

Well, the people listening all swallowed it, but just about the whole thing was completely untrue.  Yes, Jack had been droving through there, so he described the country accurately enough.  That's true, but as for the rest, you can't drive 44-gallon drums across there, because they'd either get bogged, or they'd go lame from the desert sections, because they're too heavy.

In fact, all he ever took through there was a bunch of about three hundred big goannas for the goanna oil industry, and he'd borrowed a pup from Mick's dog, one that'd escaped the lazy genes, so Jack didn't even have to do much, not with the pup, and Alice and Annie along — that part's true enough.

And if the truth be known, Flash Jack couldn't handle a mob of big 44-gallon drums, anyhow.  So what he actually did was drive a really big mob of 4-gallon drums once, though not over that route, and he reckoned it was acceptable to call that a small mob of 44-gallon drums, and to embroider the route he took.  In my book, that makes him no better than a liar.


* * * * *

Note: there is a whole book of these stories, which I am currently pitching to publishers, but they will probably appear in an e-book.

There will be quite a number of these on the blog, all with the tags Speewah and Crooked Mick.

Writing Process Blog Hop

I first posted this entry about a week ago, while I was still on the tail-end of a madly (I use the term advisedly) creative and productive session.  I thought I was clear of everything, but it kept going, until about an hour ago. As I surfaced, I did what people with Y chromosomes don't do, and looked at the instructions.

Hmmm. I was supposed to answer four questions:

* What are you working on?
* How does your work differ from others of its genre?
* Why do you write what you do?
* How does your writing process work?

It appears to me that I did some of that, but I am now setting out to get this entry more into line.

The introduction stays, partly, and that part is in fine print:

I said I would do this quite a while back, and I left the email about it in my personal email folder, marked unread, so I would keep being niggled about it. I got involved because Sandy Fussell agreed to my request to be tagged by her.

Part of the plot plan for Mad Sheep (see below).

Keep in mind that I usually write non-fiction, typically around science, technology, mathematics and history, though every so often, I go off in another direction.  Like the Monster Maintenance Manual, which a Year 6 girl told me the other day was her favourite.

It's a bit of a weird book, but it was written for kids with brains like mine.

I think that when I write for younger readers, I need to extend their view of what is normal play, what other folks are doing, inside their skulls.

For example, I enjoy complex puns, but I was probably 12 when I found out that the word games I played in my head were called puns, and lots of people do them.

That brings me to:

How does your work differ from others of its genre?

Yes, I know, wrong order.  See the end of this section.

Probably I had a rather odd upbringing, as an only child with, let us say, authoritarian parents, but I was left to my own devices quite a bit.  Books were a convenient escape, but they didn't always tell me the  things I most needed to be told.

I tell my factual stories in an amusing manner because I know that real readers like it, and because it gets right up the noses of the pompous and po-faced. I am a conservative anarchist who does not approve of blowing people up. Why? Well, it's so much more fun deflating them, and I found when I was a bureaucrat that those I designate as The Enemy are left floundering by the surreal.


I got through it all largely unscathed, but there is always a strain of surrealist anarchy bubbling away in there somewhere, a still small voice, urging me slip the line "the fen is muddier than the sward" past an alert editor.  I am probably proudest of getting into an official Department of Education publication the information that a pedant is a footnote fetishist — in a footnote, naturally.

True, that isn't a pun, but since when did anarchist surrealists worry about rules?  The living people read my stuff, just to see what I will do next, and the walking dead, well, they aren't my concern.

What am I working on?I started, 18 months back, on a set of ideas for books in a series, all called Not Your Usual... and there might be, if I did them all, about 30 of them.  I will probably only do a few of the best ideas, and basically, I am getting towards advanced middle age, and I have a lot of partly-researched stuff that I thought I might as well share.

The plan was that they would go out as unvarnished, unedited e-books, and some of them may still come out that way, but two of them had more going for them, so I sent off two mss to Five Mile Press, just on spec, and they (a) knew me from earlier work and (b) liked the stuff I can do and (c) didn't like what I had done.  They were too brief, and written for younger readers, but they had promise.

Note that I have one advantage over new starters: I have been around for quite a while, and there's a good chance of finding somebody in the firm who knows me when I try a new publisher

Anyhow, they considered (a) and (b) and said they would take them if I made changes.  So I attacked them again, expanded the text, changes all of the mixed-up bits into prose that an editor cam make something out of, and submitted on one July 31 and one on August 13, each one day ahead of the deadline set out in the contract.

I haven't heard yet, but I am hoping that all will go well, and Not Your Usual Bushrangers and Not Your Usual Gold Stories will see print in 2015.

Once that was out of the way, I thought I might be able to get into doing this blog, but one of my other books, The Big Book of Australian History was to be reprinted, and we needed some changes. The original editing had been slipshod (to say the least of it!), and while a lot of the messy bits were cleaned up before printing, once we had sidelined that editor, there was one editor-inserted factual error that I had missed (I and the new editor got all of the others), and there were a number of updates that were needed, because this is history that comes right to the present.

So I got all the latest political and sporting changes done and sent off.


I will come back to current activities because that brings me to:

How does your writing process work?


For me, revising a manuscript involves making a PDF of the file which I transfer to my Samsung tablet. Then I read it and mark it up on the tablet, usually while riding a train to a country town and back, transferring the changes to the main copy on my desktop computer, converting the result to mp3 files that I listen to while closely reading it, making a new PDF that goes on my tablet and my wife's tablet, transferring mark-ups, reading through on-screen, making and listening to new mp3 files, and so on.

Why on a train?  Because I can, Sam-I-Am.  I am approaching advanced middle age, so I can travel all day on public transport for $2.50.  I walk down to the ferry, ride to town, reading as I go, get a train to Sydney's Central Railway, and take the first country train that leaves thereafter, usually Kiama or Newcastle.  When the time is ripe, I get off, have lunch, and retrace my steps.  I have an ipod loaded with classical music, and I work, uninterrupted.

Most ideas come to me as I am browsing old books, old newspapers, stuff like that. If I get curious enough, the curiosity is promoted to the level of Temporary Obsession, and I start recording details in a spreadsheet, an eccentric method that you can read about here.

Why do I write what I do?
Because I can, and because people will read it in sufficient numbers for me to be making a useful contribution to the future of our society, a fair, honest and open society, where bullies are shamed for their acts of bastardry. Karl Popper cared about the open society, but I will settle for a fair society, and if I can encourage people to think, I am doing good.

Because I know my stuff.  I have a curious mind that leads me to ask interesting questions, difficult questions that people of a certain calibre (the small-bores, let us call them) will find uncomfortable.

Because it brings joy. It brings joy to me, and it brings joy to the minds I touch,

Because it's what I do. Scribo, ergo sum: I write, therefore I am.

What I am going to do next.
Here is what I had in the original version of this blog entry at this point, though as I will explain, it turned out to be wrong.

I am of several minds.  I may do a book, a bit like the Monster Maintenance Manual. Called (working title) Sheep May Safely Craze, it is for YAs and the permanently immature (like me) geek market. It involves sheep that wear Viking helmets so swagmen will mistake them for mad cows and not eat them, only they aren't really sheep, they just look like sheep, and they have amazing powers, a jeep that functions as a time machine and a hovercraft, and a friend called Gordon who is the same species but looks like a minotaur.

It also features the phrase "taking Gandhi from a baby" and a virtual normality machine as well as monkeys, typewriters and some rats.  Not to mention some musical ravens who are hoping to get the sheep to take them to the 19th century so they can kill Edgar Allan Poe.  I have 20,000 words of that written, and a storehouse of bad puns. No samples at this stage.

My next serious book is probably going to be on really peculiar medical treatments like using bacteria to kill cancers (that one worked!), doses of millipedes, scorpion oil, and electric shock or onions for fun and cures (those generally didn't work). It will be called Not Your Usual Treatments. It also looks at things like magnets, leeches and patent medicines, and it is fully researched but only in the rough-cut stage.  We will see,

What I am really working on now.
Well, what has happened is that Not Your Usual Treatments took me over. It was a fat and ungainly 120,000 words of drafts and notes, so in the past week, I have kicked that into shape. It is still BIG, at 76,000 words, but there is now a coherent narrative, and the things I had in there "just because" have gone out. They are all in a discards file, so second thoughts are possible, but not to be encouraged.

That can now go onto the back-burner to marinate for a few weeks. Now Mad Sheep will come to the fore, and while I was writing this revision, I decided that I need to set the plot out in more detail in a spreadsheet. I will take some of the best pun-streams and put those in place


To help you pursue a few other strands in the writing process, I point you to an old friend, Robyn Tennant-Wood who is also Miss Ruby's Bookshop, an equally old friend, Lynne Kelly who writes excellent books on spiders, megaliths and other stuff. Robyn and Lynne, you are now tagged.

Crooked Mick makes a mistake

Now don't get me wrong.  Crooked Mick was just an ordinary bloke, and just as inclined to make a mistake as I am — maybe more, because I know of at least one occasion when he did just that, though it was more a mistake of judgement than anything else.

It was the drop bears that were behind it, of course, that and Mick's dog being so lazy.  And I suppose Mick not thinking things through carefully enough.

A few of the back paddock drop bears were turning nasty, and attacking the women again.  This wasn't a problem for Smiling Annie who would just look up as they hurtled down, and smile at them, which'd make them turn right round and hurtle back up again.  It's true — one of Annie's special smiles was enough to repeal all the laws of physics, so bending gravity a bit was easy.

And Alice had no trouble at all, since she'd been strangling drop bears with her hands since she was five.  We have a name for hands as big as Alice's: we call them bear hands, but that's just our little joke.  You need a few jokes like that in the bush, because nothing much ever happens.

Anyhow, those two were safe enough, and Gertie, Greasy Smith's youngest, she'd just charm them with this charm thing an old Aboriginal lady gave her to ward off the evil spirits.  This charm was a piece of carved and fire-hardened brigalow about four foot long, and knobby, and Gertie always said it worked just like a charm, and had done for all of the twenty years she'd been using it.

Some of the other women and quite a few of the men, though, found themselves threatened by the drop bears, so Mick decided to do something creative.  He went out with his dog and showed the dog how he wanted it to roll a wave of rabbits over the drop bears after he'd gone running through, tempting the drop bears to have a go at him, and come down onto the ground.

Now as I said, the dog was lazy, and it soon got fed up with rounding up enough rabbits to trample the drop bears to death, and because they never got them all, Mick was always wanting him to turn the rabbit wave around, and run it back through the bear trees once more.

So being lazy, the dog decided to fix the drop bears once and for all, and he taught four of the other dogs how to work the rabbit wave, then ride up over the top of the wave, land clear on the other side, and then turn the wave back the way it came.

The idea was just to go steam-rollering back and forth with this wave of rabbits, flattening any drop bears that were on the ground, and being cannibalistic, the surviving bears would come to the bait, as soon as you got one or two of them, so it was all a bit like priming a pump.

Well the first day worked a treat, and being a Sunday, nobody paid much attention to what the dogs were up to, and it was only later that Gertie happened to mention that she had seen a bunch of them working the rabbit wave, all on their own.

The next Sunday, though, was a different matter.  The dogs must've rounded up every rabbit in the Speewah back paddock, because the wave was running at ten foot high, piling up to twelve or thirteen in the shallows, and it was close to unstable all of the time, according to Truthful Lewis, who saw the thing from the top of a Speewah ironbark, where he'd been chased by a bush alligator, which was sitting at the bottom of his tree and leering hungrily at him.  The rabbits got the alligator as well, on their second pass, so Truthful was happy to just sit and watch what was going on.

As I said, the wave was close to unstable, because there was just too much mass in it, and the disaster came on the fourth pass.  What happened was the dogs got this really big wave going, a bit like when you keep pushing a kid on a swing, but the fourth pass was just too much, and too rushed, and the bunnies on the bottom were getting trampled by the ones on top, and that slowed the base down.

So when the dogs rode up onto the crest of a wave, ready to drop down and turn the wave back the other way, the unthinkable happened, and the wave turned into a dumper.  Well the dogs went over the top, down the front, and got dragged under and rolled over by the rabbit wave, they had their faces pushed into the sand, and generally got treated in a demeaning way.

So when they surfaced, snorting and sputtering behind the wave which was now beginning to falter, they were good and mad.  And when Speewah dogs get mad, they roar.  And when any sensible animal hears a Speewah dog roar, it gets going, which is why the rabbit wave re-formed and took off across the plain, with the dogs still roaring behind them.  They might have been as silly as rabbits, but that roaring soon got them sensible.

All the warning the people of Bandywallop got was a low rumbling noise as the rabbits came pouring over the plain, with the dogs roaring behind them.  Of course, they thought it was a stampede of scrub bulls, and that was enough to persuade them all to scramble up onto a large rock behind Mulligan's pub, carrying whatever they could.  So they were well placed to see how high the tide came.

Except, that is, for a bloke called Long Harry, whose legs were so short that he couldn't make it to the top in time, and just as people were reaching out for him, the wave hit, and he was carried away.  Luckily for him, old Mulligan used to keep some planks up on top of the rocks, and the people up there were holding one of the planks out to him.

Now Long Harry had been down the Big Smoke once or twice, and knew a bit about waves.  So when he realised he couldn't make it to safety, he yelled out to Mulligan to let go the plank, and he rode that wave of rabbits, always slanting out to the left, until the wave died away.

Then he stepped off, and brought the board back with him, walking for two days and two nights to get back to where Bandywallop ought to have been.  When he got there, there wasn't a bit of the town left that was as big as his board, and there hasn't been to this day.  All the people just packed up, and moved to Yandackworroby, where life in the bush is slow and uninteresting, the way it ought to be.

But none of it would've happened if Mick hadn't made a bad mistake of judgement.  And even then, there would've been no problems if the dog hadn't been lazy and strong-willed, and even more lacking in judgement than Crooked Mick.  But it was the laziness that made Mick's dog forget to stop and think.  No doubt about that at all.

At least it kept the drop bears under control for a few years.

* * * * *

Note: there is a whole book of these stories, which I am currently pitching to publishers, but they will probably appear in an e-book.

There will be quite a number of these on the blog, all with the tags Speewah and Crooked Mick.