Did Hattie get his statistics wrong II: Comments from John Hattie and Arne Kåre Topphol

Earlier this week, I wrote an entry here on my blog referring to and commenting upon an article written by Associate Professor Arne Kåre Topphol at the University College of Volda, concerning what seems to be an erroneous use of a specific statistical tool, the Common Language Effect size (CLE), in John Hattie's world-renowned and highly influential 2009 publication Visible learning.

Being as I am also a student at the teacher-training program at the University of Oslo, and, moreover, a member of the student association there – and further, being both curious to get to the bottom of this debacle, and concerned with the fact that references to Hattie's research bespatter our curriculum without (to my knowledge) ever having been questioned – myself and some of my fellow members of the student association contacted John Hattie himself, and presented him with the criticism advanced by Arne Kåre Topphol.

John Hattie

To our utter delight, Hattie took the time to answer our query; and when whereupon we presented Hattie's respons to Topphol, he kindly took the time to make a comment of his own.

We in the student association have been given permission by the two parties to publish their respective comments on our blog. Our request to Hattie, and his response, is published here. The follow-up comment from Topphol is published here.

I would like to emphasize that I am not on a mission to "dethrone" Hattie, or to try to find faults where none are to be found, or to exaggerate or blow out of proportion those that may be there. But I am in agreement with the main focal point of Topphol's initial criticism, which I also believe is the important element we should take away from this discussion:

That is, as a quantitative approach is becoming more and more dominant in research on schools and education, all of us depending on this research to make informed judgments on policy decisions or professional practices tend either to trust more or less blindly the results researchers present us, or else we do not possess the necessary statistical competency to assess the accuracy or validity of a result. I agree with Topphol that publishers, peer reviewers and researchers themselves have a responsibility to make sure that their use of statistical tools is trustworthy. However, I believe that this is a responsibility that has a wider currency, and that there is also a need for more methodological literacy in general – among students, teachers, educators, and, probably more pressing than anywhere else, among policy makers and politicians.

Hence, I would encourage everyone being a stakeholder in the discussion on education in general and best teacher practices in particular to (i) read (i.e. actually read) Visible learning, (ii) read Topphols article, (iii) read the aforementioned comments from the respective parties, and (iv) make up your own mind on this particular issue. Good luck!

See also Did Hattie get his statistics wrong?