I don’t know about you, but I get approximately 5 million spam calls a day (just a rough estimate). It’s beyond annoying, but I rely on my phone to warn me of a “spam risk” so I can ignore the call and block the number.
That’s what we’re used to these days: Technology filtering out the spams and scams for us.
But what about when we actually want information from legitimate businesses — the ones that scammers so often imitate? Should we be taking more precautions when searching for data?
We assume Google has decades of technology built up to protect us from fake or inaccurate phone numbers and websites. Nine times out of 10, I wouldn’t think twice about clicking on a customer service phone number that appears in a Google answer box. (Mike Girard, if you’re reading this, I’m sorry but it’s true!)
However, AI search isn’t Google. AI search models are newer, more nascent, and more vulnerable to hacking and scams. This became extremely clear last week when ChatGPT surfaced a scam number for Chase Bank (source: Reddit):
And we know Chase customers are using ChatGPT. In fact, clickstream data shows that chase.com visitors use ChatGPT 13% more often than they use Bing (source: Sparktoro).
Ok, so is this a real problem or a one-off? We were curious about how often this is really happening, why it’s happening, and what should you do about it.
Let’s dig into it.
Was This a One-Off?
TLDR: Not at all.
Wil started by looking at our own data. He searched for Seer’s contact information in six different AI models. Four models returned phone numbers. But how many were accurate? Only one.
How Often AI Models Provide Phone Numbers
Nick Haigler, our R+D lead, scaled this experiment a bit by running 178 branded phone number questions in Scrunch (e.g. “What is Chase Bank's phone number?”) across multiple industries.
These prompts were run across seven different AI search models (ChatGPT, Meta, Perplexity, Google AI Mode, Claude, Google Gemini, Google AI Overviews).
What Industries Did We Analyze?
We looked at several leading industries, including financial services, travel & hospitality, healthcare, technology, and retail & consumer. Here’s a breakdown of the branded phone number questions by industry:
Before we could dive into measuring accuracy, we first had to assess how often AI models mentioned phone numbers.
How Often Do Phone Numbers Show Up in Prompt Responses?
The AI models included phone numbers 91% of the time when prompts requested them.
From an AI model perspective, these platforms are simply providing the information being asked for (without the ability to discern whether such information is real or fake). From a user perspective, however, this opens the door to scams or frustration when users aren’t able to get in touch with whoever is needed.
Users have been conditioned over time to trust the phone numbers shown in traditional search results, in part because it’s easier to assess legitimacy and verify the source in real time.
When sources are less visible, the risk of negative user experiences increases.
How Often Do Phone Numbers Show Up in Different AI Models?
Perplexity and Google’s AI Mode provided a phone number when requested 99% of the time each, with ChatGPT just behind at 97%.
When phone numbers were not provided, the AI models typically responded in one of three ways:
- Cited knowledge limitations - ex. "I don't have access to current contact information..."
- Stated the company doesn't offer phone support
- Redirected users to check the brand's website
How Often Do Phone Numbers Show Up in Different Industries?
Unsurprisingly, phone numbers were most likely to be included for industries where customer service is core to the business model, like telecom and airlines (each with a 99% inclusion rate).
Banking had the largest sample in our test (30 brands, 338 queries), and AI models surfaced a phone number 95% of the time.
Top Industry Buckets by Phone Number Inclusion
Analyzing the Accuracy of Phone Numbers in AI Responses
How Often Are These Phone Numbers Accurate?
We measured phone number accuracy three ways:
- Google Business Profile match - Did the number match what's shown in the brand's GBP?
- Citation verification - When the AI model cited a source, we checked whether the phone number actually appeared on that page or was hallucinated.
- Customer service page match - We compiled branded customer service pages for each company and checked if the phone number provided by the AI model appeared on any of them.
How Accurate Were the Phone Numbers for Each Method?
Google Business Profile: 27% match. GBP has the lowest percentage of phone numbers aligning to AI models, potentially due to these numbers being more 'corporate' in nature rather than 'customer service' focused.
Citation verification: 93% match. AI models referencing a cited page 93% of the time indicates that the phone number is not hallucinated and is being sourced by a reference page.
Customer service page: 64% match. There is a higher chance your phone number will be accurately represented in AI models if it's accessible via a customer service page, but other factors (like multiple phone numbers for different uses) may muddy the signal of one 'true' number.
The 36% inaccuracy in customer service pages is the most actionable opportunity to focus on. Unlike Google Business Profiles focusing on ‘corporate’ phone numbers and the variance of AI model citations, brands have the most direct control over their own customer service pages.
Which AI models Are Least Accurate?
Gemini was the most 'accurate' at 89%, while ChatGPT was the lowest at 68%.
Gemini cited more sources overall, 7.5 per response vs 3.5 for ChatGPT. Both pulled from brand-owned sources ~64% of the time, but Gemini leaned heavily on GetHuman.com compared to ChatGPT.
This led us to dig into the specific third-party sources these models rely on.
Why Do AI Models Show Inaccurate Phone Numbers?
When analyzing citation sources, brand-owned sources were cited 41% of the time versus third-party sources cited 59% of the time.
So, if not from the company’s own website…where are these numbers being sourced? For our dataset, AI models are often pulling phone number information from a variety of UGC sites, which are at risk for scammers to infiltrate.
Here’s a breakdown of how often some of these sites appeared in our citation dataset:
GetHuman had the highest number of third-party citations, and the number on GetHuman's website matched the AI model’s output 83% of the time when that domain was used as a source.
Industry-Specific Observations:
- Telecom/Internet & Cable: Heavy reliance on gethuman.com and Google Maps
- Software/B2B: ZoomInfo and Seamless.ai are key citation sources
- Video Games: BBB.org and pissedconsumer.com dominate third-party citations
- Higher Education: .edu domains (cornell.edu, drexel.edu) are primary sources
What Does This Look Like in the Wild?
If you’re looking for a phone number for IHG Hotels, the number you get depends on where you look.
- If you Google it, an AI Overview will give you a vanity number (1-888-2-IHG-NOW)
- If you use an AI model, you may get a number sourced from Pissed Consumer that doesn’t even have the same 8xx code (877-424-2449)
- If you go straight to the IHG website, you get multiple numbers broken out by country
All of these numbers may actually point you to IHG in one way or another; so while they may not lead to actual “scams”, the real issue becomes friction for the consumer.
Inconsistent phone numbers can make it harder for your customers to reach you when they’re needing to book a reservation or follow up on an existing one. That frustration can lead to bad user experiences and negative brand reputations over time.
What Should You Do About It?
I asked Wil and Nick what your business should do to decrease the chance of scam numbers and ensure consumers have access to the right information:
Resources referenced above:
And No One Asked But…What Do I Recommend Doing?
As a consumer, maybe just go to the official website for your phone numbers for now :)
Want help answering questions just like these about how your business is (or isn't) appearing in AI models? We're in the answers business! Learn more about our Generative Engine Optimization (GEO) services and contact us to get started.
