Probable, true or truthy

How do we know what is true? How do we decide what to believe? I am not going to present a logical, all-encompassing answer here, but just some helpful observations and my aims in this blog.

1. Most ideas are possible and the level of probability ranges from very close to false to very close to true. In most scientific papers this is quantified as the ‘level of significance’. So a result that is said to be significant has a fairly high probability but not an overwhelming one – say in the region of 95% or more. If I read 20 papers this week, all with significant results at the 95% level, I should not be surprised if one of them turns out to be off base. This is not a failure of the scientific method or a scandal; it is the normal workings of experimentation. No matter how good the results look, they still may be a produce of chance.

2. Some ideas are subjected to many, many experimental tests (different types of tests by different people over many, many years) and even though each experiment has a probability that is not amazing, the total probability of all of these experiments having been the product of chance becomes just too small to be taken seriously. This is what is meant by a ’scientific fact’ and it is quite different from the idea of absolute truth but is very trustworthy.

3. Results are one thing but their explanation is another. The same results could be explained by different theories. Theories are judged by how large they are, how many ‘facts’ they explain and how accurate they explain those facts. In other words how useful and predictive they are. No one expects scientific facts to change but they do expect theories to change, grow, die, merge and so on. However, some theories are trusted to a high degree. They work well and there seems no real problems with them. These strong accepted theories form the foundations of the sciences. For example Biology is based on the Cell Theory, Evolution, the Central Dogma of molecular biology and so on. This theories have moved with the times but their core ideas last. To take down one of the central theories of a science is to tear the whole strong fabric of understanding. ‘Scientific truth’ is used to mean these strong theories.

4. Thus a single experiment with a significance of even 1% or 0.01% is not going to change a scientific fact let alone a strong scientific theory. It takes a lot of results, contradictions and puzzles to bring down a strong theory or even an excepted fact. It happens but rarely.

5. Some ideas that are treated as scientific because of their context and language but they are not. Pundits and even scientists (when writing books and articles rather then journal papers) will say things that appear to them self-evident but for which they have no shred of evidence. These ‘truthy’ statements are not part of science but of literature, speculation, journalism, politics or even bullshit. What an individual has to do to avoid falling into a truthy trap is not to think about probabilities but about their own biases. If someone says something that seems self-evident to you, you will not notice the lack of evidence. If they say something you find hard to believe then you will want to know what their evidence is. Lack of explicit or implicit evidence does not make something false, it just makes it a personal opinion of some particular person rather than a piece of science.

6. New and complex experimental methods are particularly likely to produce result that are not repeatable. New and complex subjects of inquiry are particularly likely to be mis-interpreted using models that will not stand the test of time. Today these problems are especially true of neuroscience – a new area of inquiry with new methods and few consensus theories (but many old ideas from pre-scientific thought).

This is the reason that I try as much as I can to give an indication of the evidence behind the ideas that I deal with in my posts. The attempt, although not perfect, is to be clear about what ideas are my opinions (worthy as they are, I think) as opposed to scientific results, facts and theories.

As well, this post is a warning not to take individual results too seriously. A single paper may not turn out to be repeatable or the interpretation of what the experiment is actually measuring may change. It is the accumulation of evidence that counts. We want to build a fabric not a chain of evidence.

This entry was posted on 26/12/2010 at 08:05 am and is filed under Uncategorized. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response or trackback from your own site.