Wednesday, May 09, 2012

Argumentum Ad Googlum

So, in listening to a podcast this morning, I came across an argument that I hear far too often, what I call "The Argumentum Ad Googlum"

(I'm not the first person to coin this term, but all the other versions I've found so far are silly, so I'm taking it back.)
I don't know either lol.

It's an argument used by both believers and sceptics alike, so I'm not pointing the finger at anyone in particular. Essentially the argument goes something like this:

If you do a search on Google for "Homeopathy is a scam" you get 955,000 results, but if you Google "Homeopathy is real" you get over 8,000,000 results. Therefore more people believe that homeopathy is real than those who believe it isn't.

This is wrong on several levels.

On the face of it, it's a subset of the Argumentum Ad Populum, also known as the Bandwagon Fallacy: the more people that support a notion, the more likely that notion is to be true. Suffice to say that's nonsense.

But there's a second layer to this argument's wrongness: the premise is bad. In two different ways!

The first way is essentially publication bias. All this tells you is that more people included the words "homeopathy is real" in their website than those who included "homeopathy is a scam". That doesn't tell us anything about whether or not that represents the beliefs of the population in general.

In my paragraph above, I hinted at the second way it's wrong. As sophisticated as Google's algorithm might be, it's not magic. It can't (yet) figure out what the words you type into the search box mean, only what the words themselves are.

In other words, when you type in "homeopathy is a scam", Google won't return all the websites that convey the general idea that homeopathy is not legitimate medicine, it'll only return the ones using those specific words, and not even necessarily in that order. It will also often overlook short words, like "is", "a" and "and". That means it'll include sites containing any of the following phrases:

  1. Homeopathy is a scam.
  2. Homeopathy is not a scam.
  3. People claiming that homeopathy is a scam are themselves scammers.
  4. People who hate homeopathy are perpetrating a scam.

And so on.

Even if you get clever by using Google properly, using quotation marks to ensure it looks at the exact phrase, your search would still include #3 above, which is probably not what you're looking for.

Google is an essential tool in the utility belt of every Internet user. But if you're making an argument by counting the number of results Google comes up with, you're doing it wrong.

Sorry, but there are no shortcuts here. If you want to find out what people on the Internet actually say about things, you're going to have to go beyond the Google page, and into their actual sites. Google is a directory, not a source.