xkcd.com // CC 2.5
I certainly believe that scientific research is important. Research uncovers new knowledge and prunes away facts that are not accurate. However, in our society, research is also a coinage to justify views of reality. A Biblical scholar might invoke a sentence from the Bible before holding forth on his own interpretation or opinions. In a similar manner, a scientific study might be cited or a scientist quoted to justify that something is real before jumping off into one’s own thoughts, opinions, theories, or justifications. If a scientific result can be invoked, we can believe that something is true. Is there an unconscious? Freud said so, but he’s out of date. Are we intrinsically social beings? Evolutionary theorists argue. Does meditation really result in an altered state of consciousness? If I present results from research, preferably using a high tech measurement like a brain scan, or if I can come up with a theory that uses words like “neural nets” or “neurotransmitters,” then I can believe all of these things.
What’s wrong with this? Isn’t this science doing its job of uncovering truth? There are two things wrong with this. One is that not all knowledge is scientific knowledge. The second is that scientific results are often portrayed inaccurately in our society.
With regard to the first point, I’ll just give a few examples. von Bertalanffy, a systems theory scientist, wrote that even a physicist will chase his (sic) hat when the wind blows it without knowing the mathematics determining which way the hat will blow. Einstein famously said that not everything that was important could be measured, and not everything that could be measured was important.
But what I really want to talk about here is the second point. We are inundated with scientific results in newspapers, websites, and other places. Most often, a brief summary of research is followed by broad generalizations about what the research means. However, the outcome of research is not simple facts. Experiments are complicated things that must be evaluated by readers and understood in context. When I was a graduate student in psychology, every class included practice in critiquing research.
To understand research, certain mathematical ideas are important. “Statistical significance” is important to both accurate interpretation of research and to inaccurate or misleading reports. If you’ll bear with me, I’ll run through what I mean. Suppose you have a coin. If you toss the coin 100 times, it will come up heads about 50 times, not exactly 50 but close. Why? That’s just the way the world we live in works, there are laws of probability. Since there are two possible outcomes—heads or tails—each will come up about half the time. If I toss my coin 100 times and it always comes up heads, I’ll probably conclude the coin is biased. Why? Because it just doesn’t happen; it’s extremely improbable, in the world we live in, that an honest coin would do this.
July 5 – As 
