You are currently browsing the thoughts on thoughts weblog archives for the day 02/10/2009.
| M | T | W | T | F | S | S |
|---|---|---|---|---|---|---|
| « Sep | Nov » | |||||
| 1 | 2 | 3 | 4 | |||
| 5 | 6 | 7 | 8 | 9 | 10 | 11 |
| 12 | 13 | 14 | 15 | 16 | 17 | 18 |
| 19 | 20 | 21 | 22 | 23 | 24 | 25 |
| 26 | 27 | 28 | 29 | 30 | 31 | |
- 20/05/2012: Perceiving the whole individual person
- 17/05/2012: Mind-pops
- 14/05/2012: A beta version of the brain
- 11/05/2012: Unconscious cognition and control
- 08/05/2012: Seeing auras
- 05/05/2012: Power of self-directed speech
- 02/05/2012: The buena vista theory of consciousness
- 29/04/2012: Neural correlates of beauty
- 26/04/2012: Conducting consciousness
- 23/04/2012: Controlling focus of attention
- May 2012
- April 2012
- March 2012
- February 2012
- January 2012
- December 2011
- November 2011
- October 2011
- September 2011
- August 2011
- July 2011
- June 2011
- May 2011
- April 2011
- March 2011
- February 2011
- January 2011
- December 2010
- November 2010
- October 2010
- September 2010
- August 2010
- July 2010
- June 2010
- May 2010
- April 2010
- March 2010
- February 2010
- January 2010
- December 2009
- November 2009
- October 2009
- September 2009
- August 2009
- July 2009
- June 2009
- May 2009
- April 2009
- March 2009
- February 2009
- January 2009
- December 2008
- November 2008
- October 2008
- September 2008
- August 2008
- July 2008
- June 2008
Archive for 02/10/2009
Dead fish and voodoo
02/10/2009 by admin.
For a little while there having been questions about the statistical processing of fMRI scans. These play along side less sensible questions. We have:
-
It is no clear exactly what causes the changes measured with fMRI and so the results due not have meaning.
-
The way scan results are reported in the press as ‘areas for x’ is misleading.
-
There are problems with the significance levels for large numbers of statistical tests.
-
There are problems with the independence of statistical tests.
The first two are red herrings. We do not have to know exactly how an effect is caused as long as we have a general idea and correlation with other effects. The scans do have meaning – changes in the amount of neural activity. What way the popular press report research has nothing to do with the validity of the work. Calling new things ‘miracle drugs’ does not have any effect on how well or badly they may treat a disease.
The statistical problems were reviewed recently by C. Chatham at Developing Intelligence (here). He reviews the Dead Fish experiment of Bennett and the Voodoo Correlation paper of Vul. Here is his conclusion:
To summarize, the dead fish study is a point about first-pass analysis, which almost every paper I’ve ever seen does correctly. The papers that don’t always note that the result failed to pass multiple comparisons or cluster correction, and typically discuss those results with caution. On the other hand, “voodoo correlations” is a point about nonindependence in statistical tests. This has not always been done correctly, and has not always been reported clearly. Moreover it primarily affects only a subset of correlations between brain and behavior - and not the vast majority of work in fMRI, which has to do with task-brain relationships.
I believe that a good protection against being taken in by reports of research that turn out to be shoddy is to be very, very slow to accept or reject any single result. What convinces me is not any particular chain of logic or single result but a fabric of results by various people, with various methods and various view points. When such a fabric builds up, then it can be accepted.
Posted in Uncategorized | 1 Comment »