Happy 2016!

If you’re like me you spend a lot of time analyzing your site’s stats … gleaning insights and ideas for improving your sites. With that I also spend a measurable amount of time culling out the extraneous traffic … not a fun job … and more seem to popup every day.

And it turns out I’m not alone. In my research quite a few people were experiencing this growth in unwanted traffic (with some pretty scary names like “humanorightswatch”).

And the Clutter Grows

In my low-tech approach, I would setup my Google Analytics reports with manual filters and save them as reports.  Seems a good straight forward approach to the problem … minimal work and nice clean data.  Or so I thought … until new one spam site came along. So my solution wasn’t scaling very well … I needed something a bit more sophisticated. (If you are new to Google Analytics (GA) and need some help getting started here is a good place to start: “Create and manage Custom Reports“. Then mosey on over to the gallery for some free pre-build solutions shard by the GA community : “Crowd Source Google Analytics Insights”

Getting rid of the clutter

Thankfully, Google has a useful tool for addressing this growing problem:

• View Filters

Creating the Filter

I’ve had good success implementing “View Filters” (if you’re new to GA, my next sentence might be a bit scary … but hand in there!) To take advantage of View Filters you’ll have to do a little “regular expression” work.

Regular expressions(regex) is a way of defining a search pattern to Google (and other systems, like Unix for example). While at first blush they look like somebody’s cat had a dance party on the computer keyboard, however once you understand the logic, they do make sense … and are very useful!

Sidebar – Primer on Regular Expressions

So before I jump into how I made strides in addressing my issues with View Filters, you might want to brush up on regular expressions.

• Regular Expressions Quick Start: a nice concise overview, without being overwhelming
• Regular Expression Language – Quick Reference: a handy guide as you get going
• And if for some reason you really want to dig into this, here is a deep piece on the topic from Princeton.edu: Regular Expressions – The complete Tutorial

Setting up the View Filter

1. The process itself is pretty easy:
2. Login to your GA account and select the property you’d like to add your filter to
3. Select the View
4. In the menu select Filters (see below)
   
5. Then Create new Filter (assuming you currently don’t have any filters)
6. Select Include and in the filter type “Hostname”.

Why “Hostname”?  Most all of the clutter coming through has a clear domain name, so by focusing on that we can keep the regex from becoming too unwieldy.

The Filter List

I want to give credit, where credit is due … my first attempts at this were meager and it took a lot of work to make sure I had he the expression right. Then I came across a very well written article that provided a complete regex script … wonderful!

I will provide that link at the end of this post, because the author is continuing to update the list (a very generous deed!) and you can simply copy it and use it in your own filters.

But for ease of use, here is his current list to use in your very first filter:

.*((darodar|priceg|buttons\-for(\-your)?\-website|makemoneyonline|blackhatworth|hulfingtonpost|o\-o\-6\-o\-o|(social|(simple|free|floating)\-share)\-buttons)\.com|econom\.co|ilovevitaly(\.co(m)?)|(ilovevitaly(\.ru))|(humanorightswatch|guardlink)\.org).*

Copy and paste the above into the filter pattern box and save your work.

When I first deployed this the results were excellent.  A lot of clutter was gone, poof!

However a note, this does NOT clean up any historical data. And it is always a good practice to have a completely unfiltered view of your data. A good safeguard in case you discover you filtered too much and you need to go back and find the missing data. (here is an easy read on the topic : Best Practices for Views in Google Analytics by Tracy Rabold)

The “Lists”

As promised, below is the article I refered to in my post … one to check on every so often and save yourself the need to write your own regex!

• Removing Referral Spam from Google Analytics : Ben Travis on Viget.com