Baloney detection kit

7 03 2008

In the previous entry I listed the factors that, in my opinion, determine whether or not we actually believe in something. However, I also made it clear that many of those factors are unreliable and, as a result, using them will lead us astray.

So, what should we do to avoid being deceived? The answer is that we should stop utilizing many of the “default” psychological mechanisms we normally use to evaluate statements and we should instead start educating ourselves in the art of “critical thinking”. 

In order to critically analyze the validity of a claim, these are the most important factors to be taken into account (based on the “baloney detection kit“, from Carl Sagan’s book “The Demon Haunted World”):

General principles

  • Wherever possible, there must be independent confirmation of the facts.
  • Encourage substantive debate on the evidence by knowledgeable proponents of all points of view.
  • Arguments from authority carry little weight (in science there are no “authorities”).
  • Spin more than one hypothesis – don’t simply run with the first idea that caught your fancy.
  • Try not to get overly attached to a hypothesis just because it’s yours.
  • Quantify, wherever possible.
  • If there is a chain of argument every link in the chain must work.
  • “Occam’s razor”: if there are two hypotheses that explain the data equally well, choose the simpler.
  • Ask whether the hypothesis can, at least in principle, be falsified. In other words, is it testable? Can others duplicate the experiment and get the same result?
  • Control experiments should be conducted, especially “double blind” experiments where the person taking measurements is not aware of the test and control subjects.
  • Check for confounding factors – separate the variables
  • Common fallacies of logic and rhetoric

    • Ad hominem – attacking the arguer and not the argument.
    • Appeal to authority – claims must be proven with facts, not with names.
    • Argument from adverse consequences – putting pressure on the decision maker by pointing out dire consequences of an “unfavourable” decision.
    • Appeal to ignorance – absence of evidence is not evidence of absence.
    • Special pleading – logical reasoning makes no exceptions, not even for god.
    • Begging the question – assuming an answer in the way the question is phrased.
    • Observational selection – counting the hits and forgetting the misses.
    • Statistics of small numbers – drawing conclusions from inadequate sample sizes.
    • Misunderstanding the nature of statistics – President Eisenhower alarmed that fully half of all Americans have below average intelligence!
    • Inconsistency – e.g. military expenditures based on worst case scenarios but scientific projections on environmental dangers ignored because they are not “proved”.
    • Non sequitur – “it does not follow” – the logic falls down.
    • Post hoc, ergo propter hoc – “it happened after so it was caused by” – confusion of cause and effect.
    • Meaningless question – what happens when an irresistible force meets an immovable object?
    • Excluded middle – considering only the two extremes in a range of possibilities (making the “other side” look worse than it really is).
    • Short-term vs. long-term – subset of excluded middle – why pursue fundamental science when we have so huge a budget deficit?
    • Slippery slope – subset of excluded middle – unwarranted extrapolation: If we allow abortion in the first trimester, next thing we’ll know we will be killing newborn babies.
    • Confusion of correlation and causation.
    • Straw man – caricaturing or stereotyping a position to make it easier to attack..
    • Suppressed evidence or half-truths.
    • Weasel words – e.g. use of euphemisms for war such as “police action” to get around limitations on Presidential powers. As Talleyrand said: “An important art of politicians is to find new names for institutions which under old names have become odious to the public




    One response

    22 09 2008
    Critical Thinking « Ubi Dubium ibi Libertas

    […] For more regarding critical thinking, you can also read my entry “baloney-detection kit“. […]

    Leave a Reply

    Fill in your details below or click an icon to log in: Logo

    You are commenting using your account. Log Out /  Change )

    Google+ photo

    You are commenting using your Google+ account. Log Out /  Change )

    Twitter picture

    You are commenting using your Twitter account. Log Out /  Change )

    Facebook photo

    You are commenting using your Facebook account. Log Out /  Change )


    Connecting to %s

    %d bloggers like this: