How much traffic is your website really getting?

• Tweet
• Buffer

• Tweet
• Buffer

Google Analytics may be undercounting your visitors, while some sites overcount

Target audience: Website operators, Web publishers, analytics specialists, businesses, nonprofits, educators, blog and website platform providers.

Recently I gave up on HostGator, our unreliable hosting service, and switched over to WPEngine for my two consultancies, Socialmedia.biz and Socialbrite.

Then a funny thing happened. I noticed a disparity between the traffic being reported by Google Analytics and the traffic reported by my new provider.

Not just a little noise in the numbers, but a huge, jaw-dropping disparity of more than 300 percent. Take a look:

WPEngine’s estimate of visits to Socialbrite.org.

Whoa! What gives?

First, I turned to WPEngine to see if “visits” means the same in both WPEngine and Google Analytics.

Chrishaun Keller, Customer Happiness Specialist and ProDoc Goddess for WPEngine, told me: “Our system records all unique IPs that hit your site’s front end. This may include partial loads and bots used for SEO. Google doesn’t log partial loads and some (including their own) bots.”

Google Analytics can undercount site visits by 50% or more

Sky Schuyler: “If someone adjusts their browser to noscript, then their visit to your site won’t show up..”

Next, I turned to Sky Schuyler, a friend who runs Red7 Communications and who was the CTO of the Dalai Lama Foundation and tech lead on the Traveling Geeks trip to the UK in 2009 that I organized. Sky also operates the CyberSpark.net service, which monitors free speech NGOs and protects them from hackers.

The answer to my “what gives?” question is that true traffic numbers, not surprisingly, lie somewhere in the middle. While I had heard over the years that Google Analytics undercounts site visits, I didn’t know that the undercount was so dramatic.

But let Sky tell it:

Yes, there is always a discrepancy, and always in this “direction,” but not usually so large.

GA always measures “low.” You can find this information in their help pages, but here’s my personal take on it.

The first factor is that GA only records visits by browsers that have javascript turned on. So you have the issue that many people do not have javascript turned on, and you have the issues that sometimes the javascript doesn’t execute even if it is turned on (some connections might be blocked). If someone has “noscript” on (which I do), for example, then they may visit your site but never show up.

Usually I tell people that Google Analytics (because of javascript issues) counts about half of what their actual visitor number is. So for Socialmedia.biz, if you had just under 25,000 visitors according to GA, then you more likely were approaching 50,000 real live visitors. And SocialBrite’s GA-reported 79,000 visitors probably was nearly double that number.

I should mention that bots do not care about javascript, so they don’t show up in Google Analytics either, fortunately.

Now, on top of that, WPEngine measures every bot as a daily visitor. And every IP address from which a single bot visits you counts as a new bot. Googlebot, for instance, might visit you from a dozen IP addresses in the course of a day. Generally I say that about 25% to 50% of WPEngine “visitors” are actually bots (and this applies to other hosting services too, of course), but I base that on “before and after” figures — meaning that I know what my blogs see in terms of bot visitors, and then after I move them to WPEngine I know what the figures are, so I estimate percentages based on that. I have no way of checking this in real time.

And WPEngine measures every hit to a photo even though no blog page is picked up. So if you have photos that are very popular and other folks are “stealing them” by just using your URL (say for their icon in a chat forum), then you could have many thousands of “visitors” who actually are not even seeing your pages. You could have many thousands a day. I have seen this happen. Bots, and people who are scarfing up your photos without ever reading your pages, could be as many as 50% of your visitors. I have seen numbers way worse than this in cases where someone picked up a photo and used it for their chat icon.

I think WPEngine could and should do a bit more analysis on this issue, and I wish they’d give “bot visits” a lighter weight in their counts, since bots tend to pull only a page and not all of the associated javascript, styles, plug-in files and photos from that page, while human visitors can pull dozens or even hundreds of files just to render a single page.

What’s been your experience with traffic counters?

External traffic measurement firms, of course, do a vastly poorer job than tools like Google Analytics that measure actual visitor numbers, and so I don’t pay much attention to the guesstimates from Alexa, Quantcast and Compete.com, which don’t have direct access to visitor numbers and thus estimate traffic based on sampling and other factors.

What has been your experience? Have you conducted any experiments to determine how accurate Google Analytics’ numbers are? What other tools do you recommend? Please add your wisdom in the comments below!JD Lasica is founder of Socialmedia.biz. We work with large and mid-size businesses and organizations on social media strategies and optimizing your online presence. Contact JD by email, follow him on Twitter and Google Plus or leave a comment below.