Clever Hans

Some types of experiments can be prone to a type of error than is often called the Clever Hans phenomenon, where involuntary clues are given by the person conducting the experiment without being aware of these cues prompting the behaviour of the subject.

This problem has been raised as a reason to distrust unconscious priming experiments. Bruce Bower has an excellent blog posting on this subject and I recommend reading it. (here) I have written about experiments that may have been faulty in this way in the past. I have no way of knowing the small details of how experiments were done and therefore no way of knowing whether to be suspicious of any.

So to repeat what I have said in this blog may times – the important evidence is not individual experiments, but consistent results over many labs and many methods. It is not a thread we are after, but a fabric.

I assume that there are well known papers that suffer from some Clever Hans unintentional flaws. I assume that are also flaws in many statistical analyses. I assume that there is a problem with the publication of replicating (or more importantly, unreplicating) papers. Scientist should clean up their act somewhat by, for instance, using the blog to post unpublished replication attempts of psychological studies. But I am trying not to throw out the baby with the bath water and still feel that there is a lot of evidence in favour of embodied cognition.

Here is a sentence from Bruce Bower’s blog. Cesario, who studies how responses to the same prime vary in different physical settings, says that priming critics are threatened by evidence that complex thinking doesn’t require conscious thought. It is probably true that they are motivated critics, and it is a good thing that there are motivated critics. People who find it nearly impossible to believe something have the motivation to find any and all flaws they can in experimental evidence.

Those of us sitting on the outside, with perhaps a more open mind, will just have a wait patiently for the fog to clear. I have not been completely convinced either way but if I had to bet money, my money would be on embodied cognition.

Leave a Reply

Your email address will not be published. Required fields are marked *