Insights

AI Visibility is a Vanity Metric, here's how to tell your boss

Post written Jan 29 2026


AI visibility on its own is a vanity metric. Just like rankings were, but it is harder to determine how AI visibility is impacting your bottom line. In the immortal words of Biggie, Things. Done. Changed.

In SEO the only major variable was "can my agency get you ranked in X position to get Y CTR".

In AIO or GEO it's way different:

As Rand Fishkin just showed in his study on the inconsistency of AI answers, AI responses are NOT repeatable.  He didn't even account for the personalization effect. Which will make these metrics even harder to track.

Rankings was the vanity metric of the pre-AI-search-days, but it was one that you could easily "check" by tracking traffic from that keyword (then not provided came), and then how much entry traffic to that page converted, drove newsletter signups, etc.


But this is different than not provided!

You had a control group - The pages you optimized, usually were just for your SEO not others

You had dates - You optimized the page on a date, and you had an indexing date, and it was fast

You had 10 blue links per page - each link was clickable (trackable)

You had URLs and before and after dates, you were good to go!

In AI Search you are subject to:

Training date cutoffs (see below)

No stable "ranking" - We are ranked #5 and I can assume a CTR of X

No stable order of answers - We show up for most searchers in position 5 of 10

No consistent number of results - For the same prompt AI will show 3 results then 17 -  there is no page 1 has 10 links

No link consistency (Google) - Google's AIOs are now returning "links" that run more Google searches (check this out)

No link consistency (Open AI) - I just did a search for a contractor, I got 17 results some had links, some didn't. The ones that didn't required me to copy and paste in which is organic search, NOT AI, the ones I clicked, would be AI search, and for some I just typed in that would be direct, all from 1 answer.

Training Cutoffs Matter!

Gemini 3 | Launch: Nov 18th 2025 | Training cut off date: January 2025

In Gemini for "training data" answers - the GEO work you did in Feb 2025 isn't helping you in Feb 2026! Remind your boss of that!


Me? I focus on winning with humans. Then the signals that show me that is true or falls are what follow. I've had calls where people have referenced our B corp status, our studies, our sleepouts for youth homelessness, posts about losing big clients, and posts saying, I don't know what I am doing with AI search (but I'm experimenting a ton).

Ugh, it's a paradigm shift.


I hate the phrase "paradigm shift", but we have to use it. It is the right word for the right time.

If you spouted wings today and could fly, wouldn't that change how you make 100 other decisions?

The old flywheel

Write content get ranked, flip that ranking into traffic, turn traffic into leads, and leads into sales. You wrote the content Google ranked it and the rest is history. Google had all the muscle. If your content ranked, for the right keywords, you won. Period.

Being seen = winning the algorithm

Being believed = winning the audience


The new flywheel

We launched a free template for you to track this almost a year ago, we saw that trust was going to be a signal, winning humans and algorithms was going to be important, so we built this and you can copy it right now.

 

My own content that I will ignore investing in!

This post has low value, will be disrupted, and will get 0 marketing time.

Any post that when seen & rewarded by algorithms, drives people to my site, but those people got to the content but people don't believe it is worth sharing, bookmarking, etc, cause my direct would be growing if it was.

 


The content that will be getting more time, cause there is a flywheel!

This post has built trust for my brand and will get my marketing time & attention.

A new playbook, once I know people loved this post (direct and organic social), now I have a whole new confidence in optimizing it for the "algorithmic" channels, SEO and AI. A post like this needs care and feeding, how do I get it to more journalists, how do I update it regularly, how do I respond to the comments, find the mentions in reddit, let me pour more fuel.

The real work to be done...sort by "humans loved it" but the bots not so much.

Now you can feel like, if I improve this content for search, when people get to it, they will like it - so now you can try to add keywords strategically to that post for a light touch or write a new post to be more aggressive on SEO and GEO, but either way you know this...


When people get to this content, they like it they share it & sign up for your newsletter! That is what being "believed" looks like. Optimize THAT!


 


"We are not visible in AI search!! You better fix that!" -- Your CEO


The question is coming, are you going to cower or consult? Let's get ready to consult...


1. Ranking real estate is a declining asset: The old rank in position 1 get X% of traffic is over, as it declines in value every month, we did a study on that.


2. Tracking visibility in prompts is subjective AF, with personalization alone. One of our client interviews of their customers showed 44% of people put brands in their prompts, meaning they already believed some brands could solve their problem. You can't AI visibility your way into people believing your brand is worth putting into a prompt.

3. Monthly search volume, in a prompt, isn't available in any platform, not ChatGPT, Claude, Perplexity, or Gemini. Some companies have come up with a method, but I haven't seen them publicly disclose how they come up with those so I'm skeptical AF.

4. People are getting answers to complex questions in AI, without having to visit people's sites until they get way down the funnel.

5. Our ability to analyze data and get whole, clear answers is a farce. And it's now your job to tell them that.

But your boss / C-suite are still relying on their marketing data knowledge of 2020 and in this world that's going to give them (and you) a headache.


No amount of yelling will bring back not provided keywords, ad blockers, or extend the cookie tracking window or make Sam Altman launch ChatGPT webmaster tools.


The faster we can accept that reality, the better we can pivot to the new reality, you gotta have more faith that good marketing (in all facets), not listicles, is the true path to winning, and you also have to accept that tracking the impact of good marketing is harder than ever.


When a leader says:

"But when I searched for… XYZ fact about our industry we didn't show up" or "why is our AI traffic so low?"


You say: Visible for what? - Show them our research that shows in B2B that 44% of the prompts, from real people, included brands. Show them Rand's Research on how much variability there is in answers. Then ask why aren't you asking me to better support us becoming a brand given that research?


You say: - Here let me show you how reasoning, memory, or just adding one word to a prompt can change your answers (Give examples)


You say: What's the impact? How do we know if being visible is correlated to [money making metric] and would you be open to me showing you a new way?


You say: Traffic is low, would you mind sharing your screen and opening up one of your prompts recently, I'll show you all the keywords and sites you would have had to click on if you didn't have ChatGPT.


Our job is to make this change tactile and real.



The key is going to be offsetting metrics:


I got a video here where I discuss that specifically.

 

If my AI visibility in ChatGPT or Gemini goes up, what other metrics might also trend up or down? This is going to be the new way. Note: If you say revenue good luck, by the time a conversion happens, every team in marketing is trying to take credit.

Walk them through actual scenarios. This is why we do live AI tests with our clients customers to see and learn how they use AI to solve problems.

Here's a potential role play with an exec:

Let's first ask our exec, are you comfortable with the assumption that people who used to search are now on AI? Yes.

Great, so if people are getting all these answers in AI (like I just showed you you are) - they are highly likely after doing all that research to search for our brand in some way. Ok I'm with you.


Great so is it OK to assume that when people do research in AI that at the end of their journey they are more likely to search our brand in Google? Which is trackable.


Now you've got some yes momentum, level it up.


AI visibility against direct traffic. Ok, now you gotta educate the leaders on the fact that direct is not just "people from bookmarks and typing in our domain" (old paradigm) it is also facebook, whatsapp, TikTok, etc.(new paradigm).

But you could assume that the more people know about you the more people are coming in direct, or direct on your homepage.


"Hey boss, if my AI visibility is going up, maybe one of the metrics we should be looking at is brand search?"

AI visibility is great and all, but where they land on your page when they do click through is important too.

Take the few visitors you do get to your site from AI, and look at where they go, what % are work pages, homepage (I might want to work with them) vs blog pages (I want to learn from them).


When someone comes from ChatGPT and hits your home page or services / solutions pages, you can reasonably assume they were looking for services (not be careful with sales people running agentic browsers, but like I said, gotta have faith here).


Get them to agree. The whole goal is to bring them IN, not to just shove reports and studies in their face.


 

We love helping marketers like you.

Sign up for our newsletter for forward-thinking digital marketers.