Analytics

Direct Traffic is Dark Traffic, and That’s OK

“Martin, it’s all psychological. You yell, ‘Barracuda,’ everybody says, ‘huh, what?’ You yell ‘Shark,’… we’ve got a panic on our hands on the Fourth of July.” – Mayor Larry Vaughn, Jaws (1975)

animal 1867075 1920

What is Direct Traffic, Sayf?

Hi random person! Thanks for asking!

We’ve all seen “Direct Traffic” in our Google Analytics reports. (direct) / (none) or just “Direct” traffic. Most people have the wrong idea about this traffic though. To this day people still say that it’s one of the following things:

  • Clicking on a saved bookmark
  • Typing in a URL by hand

It’s so much more though. It’s:

  • Clicking on a link from a secure HTTPS page to a non-secure HTTP page
  • Tapping on a link in many mobile, even social, apps
  • Clicking on a link in an email program
  • Clicking on a link shared in an Instant Messenger, Hangout, Video Chat, etc
  • Clicking on certain image or mobile searches
  • Your traffic being deliberately obfuscated
  • Clicking on an untagged link in a doc like a PDF or other document (gotta tag those links)
  • User clicked through a shortened, untagged, URL
  • Clicking on a link in any sort of installed software

Really what (direct) / (not set) means is that there IS no attribution or referral information.

That’s literally what it means. It’s Google’s version of the shrug emoji.

¯\_(ツ)_/¯

direct traffic 1

In this screenshot, there were 5,541 new users who came in direct. Those people probably didn’t click on a bookmark. Maybe some did? I dunno. They could have come in any number of ways. The point is that we generally don’t know. Sometimes I see this and I start getting ranty. I feel the advice of Mayor Vaughn above, but I’m gonna ignore it and scream “Shark.”

Direct Traffic IS Dark Traffic. We need to quit pretending that we know who that traffic is.  It’s not (direct) / (none) it’s  ( ¯\_(ツ)_/¯ )/( ¯\_(ツ)_/¯ ).

“Direct Traffic” makes you go “huh, what?”

“Dark Traffic” should make you go into a panic on the Fourth of July, but it probably won’t. Ok, actually please don’t go into a panic over Dark Traffic. That’s probably overreacting to marketing data. Maybe.

It’s time though we did something about this monster plaguing our fair ocean city before it kills again.

“That Direct Traffic is only 38% of the data, Sayf. We’re good! You’re overreacting!”

Maybe though you’re assuaged by the Mayor, and want to test the water?

Take a look at the data above. See the bigger problem? Here let me highlight it.

direct 2

“Sayf, my data is clean and perfect!”

Great, good for you, but you’re the exception. So many people have different problems that tack onto the Dark Traffic bandwagon. Here’s an example.

  • Actual Direct Traffic = 38.29%
  • A self referral from a subdomain = 13.41%
  • A mistagged link = 8.21%
  • Someone using Campaign Tags on the site (special level of hell for these people) = 2.19%
  • Another self referral = 1.95%

Add that up and it’s basically 64.05% Dark Traffic in the top 10 referral sources. 64% of the traffic that the referral is either overwritten or screwed up. This account has 64%+ of its attribution just flat out scrubbed. Anyone using this traffic looking at the organic numbers is just some poor soul about to head out on Amity Beach ready to be dinner.

shark 2017046 1920

“Ok, Sayf, fine, but I have plenty of good attribution. I can look into my reports and see the conversions!”

The conversions in Google Analytics standard reports? Oh, that’s dark too. Lemme explain.

Let’s say that I do a search for “Skeleton Cycling Jersey” and find an organic result that I like the look of and click it. I go to the website, I check out the jersey. Looks good, but I’m not ready to commit, and I drift off.

search1

Later on I’m wasting time on Facebook and see an Ad for the product. Hey, their remarketing team is pretty slick. They got me. I head back, and purchase the jersey.

search2

If I were only looking in my standard reports I’d see a conversion attributed (hopefully, they tagged it right) to my paid social ads on Facebook. My original organic search would get no value from this in the standard reports. In our modern world, Last Click Attribution  is shit. It’s dark traffic that removes value from efforts we’ve made.

And what if we have a Direct/Dark session in there?

search3

Well… Google, rightfully, doesn’t want to take value away from something it knows (in this case an organic visit) from something it has no idea what it is (in this case a direct/dark visit), so it doesn’t actually do Last Click Attribution in the standard reports, but Non-Direct Last Click Attribution. IE “Last Attribution Click we know what it was and ignore all the visits we don’t have a clue about”.

And what happens when you have multiple dark visits? Say, one where someone visited a non-secure site from a secure one, and then clicked on a link to a blog page from an email link?

search4

Well, it’s Dark followed by Dark. Google doesn’t have a clue about your visits, so even though it’s doing Last Non-Direct sometimes… But sometimes if all it has are direct visits…

search5

Ever see that before in your Channel Paths? Direct X 2, Direct X 3, Direct X 4, Direct X 5. And so on. Dark traffic gives you dark attribution.

Sure there are a variety of attribution models you can use, but what’s the point of doing a Time Decay Attribution model when you have 63% dark traffic because all it’s going to show is a ton of black dark traffic.

search7

Direct Traffic is Dark Traffic

Non-Direct Last Click Attribution is Dark Attribution

If that’s what you’re doing, you’re just sitting there watching someone get eaten by a shark.

Is it starting to hit you yet?

“Sayf. Please. I use Multi-Channel Funnels, because I am a Super-Marketer.”

Well, you’re just a special one aren’t you? Clean data, multi-channel funnels. Great! If you are using Multi Channel Funnels it can be great, especially with clean data.

mcf1

You’ve got the same story above, an organic visit, followed by a paid search visit. You track it all and then jump into your MCF Channel Grouping, and there are your Assisted Conversions for Organic Search, and Last Click Conversions for Paid Search.

mcf2

Oh, what a wonderful world.

One quick question though… What if this was your story instead? You have a user on an Android phone who does a voice search. They get a result and go to your site, where they find your conversion form. They’re not super ready though and go to your Facebook page on their Mobile App to check you out, and make sure you seem legitimate.

mcf3

Later on, after following your page, they see you post something, and they go to your site again. This time though they still aren’t ready for the final conversion, but instead sign up for your email newsletter. They’re just MOWING through that Customer Journey!

mcf4

Finally, 4 months later they’re ready to convert. They get an email from your regular newsletter drip, click it, this time from their laptop, and convert, filling out the form.

mcf5

So… Where’s that gonna break? Where’s that go dark?

We have a Non-Branded Search converting on a click to the social pages.

Then we have a social visit from Facebook, with a newsletter signup conversion

Lastly we have an email marketing visit, and a full on lead form conversion.

So where’s it go dark? It goes dark literally everywhere.

The first visit is from a voice search, and it shows up as direct.

The second visit is direct too. The click from the secure app page to a non-secure website got scrubbed.

The third visit? You forgot to properly campaign tag your email links. It’s direct.

mcf6

So now you have 3 direct visits, conveniently grouped together under Direct in the MCF Reports.

search5

Oh, I forgot to mention something. All those three visits are considered different users, too! Each one was on a different browser, without any proper cross device tracking or tagging (I didn’t mention that did I?) so they show up as 3 different users in Google Analytics.

Don’t worry though, only ONE of those visits will show up in your Multi Channel Funnel report. The one that took place last, 4 months after the other visits? Because normally it’s a 30 day limit, which you can extend to 90 (though you might hit sampling). No worries about sampling here though, because that 4 month visit is well out of range anyway, so it’ll only show up as a single conversion with a single direct visit.

Had to have been a bookmark, amirite?

“Ok so… What am I supposed to do, Sayf? Won’t anyone think of the children?”

You think I’d drop this 20 foot shark on you and not have some Sayf Shark Repellent? I gotchu.

Here are my 3 main recommendations

  1. Use Campaign Tags
  2. Create Some New Filters
  3. Create Some User Cohorts

Lemme break those down.

Use Campaign Tagging in Google Analytics

If you’re not already familiar with Campaign Tags, then get familiar. Here’s a link to a way to easily build them: https://ga-dev-tools.appspot.com/campaign-url-builder/

Campaign tags are ways of putting additional information on the end of a link like this:

yourdomain.com/?utm_source=jingle&utm_medium=radio&utm_campaign=radiojingles

Instead of just sharing a link to yourdomain.com, you share something with these additional parameters. There are five standard ones: utm_campaign, utm_source, utm_medium, utm_term, utm_content. This way if you share something to facebook and tag the source as facebook, the medium as social, etc, then when someone clicks on it, if they happen to go through some sort of “darkifying” redirect, the parameters stick around and your system knows they actually came from Facebook.

They’re great for offline stuff like radio ads or billboards, too.

billboard1

Ever see campaign tags on a billboard? I hope not. Who would type all that shit in? They can’t even read it as they’re speeding by at 80 miles an hour. One time I saw something like the following:billboard2

Seriously, a QR code on the highway billboard. Genius. I’m sure that never caused a wreck, mostly because nobody uses QR codes. But you know what works?

billboard3

That’s a plain old custom URL, it’s not the main domain. Think “learnaboutgreengoo.com” or something like that, which then redirects to a campaign tagged URL on the main domain. Then you know the people who entered that domain likely saw one of the billboards, or at least came indirectly from that campaign. Instead of just having the main company URL there, and direct visitors, you’ve got some information.

Same things for radio ads. Share custom URL’s rather than the main company domain, and redirect that URL with campaign tags to the main domain, and suddenly you’ve got attribution from radio in Google Analytics.

radio

Learn more about Campaign tags, and use them everywhere you share anything, but particularly on social, emails and offline where you can.

Create New Filters

Ok now I’m gonna start getting a little controversial.

If “direct / (none)” means that there is a lack of information, why not add/modify those source mediums. You get nothing out of something being “direct (none)”. It means dark traffic. So let’s shed a little light on it.

giphy

I propose filtering your direct traffic into three buckets:

  • Surface / Dark
  • Shallow / Dark
  • Deep / Dark

I’m gonna mix metaphors a little bit so forgive me. So, we want to first change their mediums to Dark. (not set) is sooooo Googly, but doesn’t really scream that it’s dark. So let’s just flat out say “This is dark traffic.”

Second let’s separate the direct traffic into three groups based on their landing pages.

surfacedark

Surface Dark traffic comes to the home page, like seerinteractive.com. This could be direct bookmarked traffic, people typing in the URL, or branded organic. It’s possible it’s social share too, but probably to a lesser degree, and depends on your business.

shallowdark

Shallow Dark traffic would be traffic that comes to category pages, section fronts, etc. This is a mix of longer tail branded traffic like “sayf sharif seer” and non-branded organic. The bookmarks and directly entering the URL in are reduced.

deepdark

Deep Dark traffic are visits to deep pages, like specific blog posts with long URLs. These might be bookmarks for return visitors, but for new visitors particularly these are probably social shares, or referrals, or from an email. Something besides hand entering a 100 character URL.

By modifying your direct traffic in this way you start separating out the “Direct x3”s into some more interesting channel grouping paths in your Multi Channel Funnels.

darkattribution

You still don’t know what that dark traffic is, but someone landing first on surface dark, is different from someone first landing via deep dark. Adding a little more information to that dark traffic helps us shine a light on that behavior.

Create User Cohorts

Google by default has some nice cohort features, but as we discussed the big problem is that we start getting limited at attribution that goes beyond 30 days, and certainly beyond 90. I don’t know about you, but I have clients who have attribution paths and customer journeys that last 9 months, or even a year and a half. 30 days does not cut it.

So, I recommend capturing some of that attribution information in a custom dimension along with the user.

Without a good long term cohort stamp, of at least the date, your conversion in a 30 day funnel might look like this.

cohort1

One visit, three users, and conversions. Only their visits in that last 30 days accurately represented. But if we add a cohort custom dimension, where we capture the dates and referral information generally for each user, we can then pull out those conversions and see the following:

cohor2

They had different conversion paths, that the 30/90 day window can’t show you, and you are getting a clearer picture of what their customer journey was.

When you normalize it, outside of Google Analytics, you can start seeing patterns. How was the cohort from April 2016 doing a full year later?

cohort5

When you capture this information, you can pull it out and start looking at your user attribution over long stretches of time, much longer than the user segments, or built in cohort analysis will for you.

In a future post, we’ll share some code to help you do the above if you can’t figure it out on your own.

In Conclusion

  • Direct traffic is Dark Traffic is Shrug Emoji
  • Last Non-Direct Attribution is Poop Emoji
  • Multi Channel Funnels by default are tainted by too much Shrug Emoji and Poop Emojis
  • T.A.T.S. = Tag all that shit. Use Campaign Tags Everywhere
  • Make Dark Traffic Channels to shine light on remaining dark traffic.
  • Create user cohorts with custom dimensions to see attribution past 30/90 days.
  • Live the dream.

atoll 2179281 1920