Debunk This! How to Look with Critical Eyes

British statesman and author Benjamin Disraeli is famous for many things, but the follow cautionary phrase might be his most-famous quote:

There are three kinds of lies: lies, damned lies and statistics.

We all know that statistics can be manipulated and cherry-picked to support different points of view. We also know that statistics by themselves just say something factual about the world.

Statistics give us no clue of the way the world ought to be.

Most often, we don’t get our statistics from raw-source data, but rather within the context of some type of academic study or "expert" presentation.

When studies cite statistics, it’s generally not a coincidence when they support a particular point of view.

Now, as a thinking person, you should always approach a study with two things: openness to being persuaded, and a skeptical mindset that questions everything.

To help you do both of these tasks, the Economist recently published an article with the appropriate title, "How to debunk a study."

***

The first thing to understand when debunking a study is that humans are pattern-seeking animals.

As such, we tend to find and uncover patterns in all kinds of data, and to make connections with seemingly disparate concepts.

This tendency no doubt helped keep us alive as a species for at least 100,000 years as modern man. But pattern recognition can be both an asset and a liability.

According to the Economist:

Scientists use statistical tests to sniff out sense from the data, but even their tests can sometimes turn up apparent relationships where there are none.

To guard against this tendency to make connections where connections might not exist, scientists use what’s called the "p-value."

This [p-value] is the probability that the test would report the same relationship (or a stronger one) even if it were run on random data with no underlying pattern. A lower p-value is better, as this makes it less likely the pattern came about for no reason.

The Economist then goes on to recommend the following strategy for analyzing any study:

A simple way of debunking any paper is to recalculate the results of the original tests (a "strict replication"), hoping to spot an error in either the original calculation of a correlation or the p-value.

While I think this makes a lot of sense from an academic standpoint, I realize that this kind of recalculation of data from a study isn’t what we are all going to do when confronted with the myriad data we are exposed to each day.

So, to debunk the claims of so-called experts in just about any field (including expert studies), we can apply some more basic tools that will get us where we want to be a lot faster than any recalibration of academic data.

***

The first and possibly best way to debunk any study or pronouncement from any authority is to apply something called Occam’s razor.

The principle states that, among competing hypotheses that predict equally well, the one with the fewest assumptions should be selected.

Another way of saying this is that the simplest explanation is usually the correct one.

For example, is it more likely that the so-called "Face on Mars" is just a rock formation that appears (to pattern-seekers) like a human face? Or that is this actually a sculpture done by some alien intelligence?

Both of these are plausible explanations, but the simplest here is that we as pattern-seeking humans "recognize" the shape as resembling a face.

A similar tool in our debunking arsenal can be taken from philosopher David Hume.

Hume’s "What is more likely?" question here applies to any study or claim by an expert.

For instance, what is more likely: David Copperfield can actually make a skyscraper disappear, or that it is some type of optical illusion?

Applied to the realm of the financial field and investing, what is more likely: that stocks will continue to move 20% higher after nearly seven years in a bull market, or that we will first see a correction that takes the market back at least 10% off the all-time highs?

Using Hume’s tool, I think it’s logical to conclude the latter.

***

Of course, there are many other ways to debunk studies and/or expert claims. But rather than tell you what I think they are, I want to know what you have to say.

If you have a way to help determine the veracity of a claim, please let me and our readers know.

Send me your thoughts and let me know by leaving a comment on our website or by sending me an e-mail.

***

U.S. stocks traded slightly higher Friday, after data on industrial production climbed 0.6% in July. There were also upward revisions of 0.1% each in February, May and June.

Meanwhile, the University of Michigan’s consumer sentiment index edged slightly lower to a reading of 92.9 in August from 93.1 in July. The number was in line with expectations.

•  China raised the reference rate on the yuan by 0.05%, after devaluing it twice earlier in the week. The new rate is 6.4 yuan to $1, although it can continue to fluctuate.

•  Explosions at China’s Tianjin port have disrupted the flow of cars, oil, iron ore and other items through the world’s 10th-largest port. Although domestic WTI crude gained 27 cents today, the international Brent benchmark slid 17 cents.

•  Oil prices were higher Friday, despite an increase in the total number of U.S. oil rigs. The closely watched Baker Hughes rig count rose to 672 this week, up 2 from a week ago.

•  Longtime public-television favorite "Sesame Street" will now be brought to you by the letters H, B and O. But the children’s show relocation didn’t help shares of HBO parent company Time Warner (TWX), which drifted more than half a point lower in today’s session.

•  John Kerry was in Cuba today for the reopening and flag-raising at the U.S. Embassy in Havana. Speaking of debunking the incredible, our story yesterday about Kerry’s "Dangerous Lie or Frightening Prophecy" drew some great comments from Afternoon Edition readers. We’ll share some of our favorite feedback next week, so please send your thoughts over the weekend!

•  Social Security turns 80 years old today. But Nilus Mattive uncovered evidence that it won’t see its 100th birthday in its current state, which he shared in Tuesday morning’s edition.

Good Luck and Happy Investing,

Brad Hoppmann

Publisher

Uncommon Wisdom Daily

P.S. Just this week alone, I’ve seen significant signs that a new financial collapse, one that I have been warning my readers about for months, is nearly here. Take a quick minute to get the facts … plus our tips on how to protect yourself before this situation escalates … in this FREE investor video.

Your thoughts on “Debunk This! How to Look with Critical Eyes”

  1. Man is NOT an animal. That should be obvious. You have set yourself in opposition to the Bible.

  2. Ask which is the cause and which is the effect. Correlation suggests possible cause, but people often have the cause and effect swapped. For example heavy eating correlates to obesity, but obesity is not the cause– it is the effect.

  3. My definition of an expert is an Ex is a has been. A spert is leak under pressure. I also use what I call my rules of 1/3. 33% will 33% may and 33% won’t. This rule can be applied to all things in life

  4. I’m 89 years old, which seems to have given me a perspective I lacked in my younger days. The tool I use the most is; examine and evaluate the source, using the following criteria:
    1) does the source speak, IN GENERAL, in a balanced, mature manner?
    2) does the source handle objections to his claims in a sober, respectful manner?
    3) does the source have a good record of PAST predictions?

    Bill

    P.S. I’m predicting an economic and social “holocaust” probably within months to a year. B.

  5. I have added a fourth demention to the Disraeli quote, “Lies, damned lies and statistics.” I add: the Media’s biased interpratation of these statistics! And the always great test of all of this: Correlation does NOT equal causation!!

Comments are closed.