While we can rely on tests and citation data to understand LLMs, the platforms themselves offer little transparency into how information is surfaced in prompts. Bing gives us a glimpse into the black box through grounding queries, a feature within its AI Performance report, currently in beta.
Grounding queries are searches that LLMs generate to retrieve information before constructing a response. These queries do not reflect search behavior 1:1 as they are not the exact prompts users are typing into Copilot or Bing Chat. However, the data can provide insight into a site’s topical authority as it reflects how LLMs are associating relationships between your content and the query.
Our team reviewed the grounding query data of seerinteractive.com to better understand how Copilot and Bing Chat retrieve information. Within Bing Webmaster Tools, we discovered that our brand was being associated with the phrase, "GEO testing methodology." From there, we decided to explore what pages were being cited.
We discovered:
- Our case studies page was dominating citations for that query. GEO services, the page that most directly speaks to Seer’s GEO testing solutions, was being cited far less in comparison.
- A 2014 post about PPC geo-targeting was surfacing at comparable volume to our high-intent, "What is GEO" page, despite being less relevant.
The data suggests that LLMs don’t fully know what GEO means yet
GEO as in generative engine optimization is a new acronym. GEO as in geographic targeting and geolocation has existed in digital marketing for decades.
When the LLM encounters "GEO testing" without enough semantic context to connect it to the generative engine optimization meaning, it defaults to the definition with a wider usage. You can see this example in action within Bing's answer for "Seer interactive GEO testing methodology," which describes our approach as evaluating "geographic and SEO-related factors." That was not our intent, but the LLM didn't have strong enough context to know otherwise.
.png?width=636&height=677&name=seer-interactive-geo-testing-methodology%20(1).png)
This is a problem that is not unique to Seer. Any brand operating in a space where terminology is still being established is vulnerable to this. The broader web hasn't caught up to the newer meaning yet, so the LLM fills the missing piece with what it knows.
How grounding query performance insights can lead to action items
The result of high intent and wrong page is what made this finding so useful. This allowed our team to take a broader look at where our own content was missing information that Bing was trying to fill on our behalf.
When we looked at our own content on the business impact of AI, which covers LLM experimentation and testing in depth, we found that the phrase "GEO test" was missing from the page. Bing was making inferences from related content, but it was not understanding semantic relevance. This surfaced the opportunity to both expand content and improve page equity through internal linking.
How to leverage Bing Webmaster Tool’s grounding query data to understand if your content is showing up in the right contexts
Seeing if the correct information is showing up when LLMs provide answers to users is critical because it can help you see where there are gaps in your own content.
To find this yourself in Bing Webmaster Tools:
- Review the “AI Performance” tab.
- Ensure that “Grounding Queries” are selected under the “List by” section. Start with grounding queries that appear to include ambiguous or emerging terms.
- Compare intent to the pages that Bing is citing. Ask yourself if the purpose of these pages are aligned with the intention behind the query.
Now what? Deciding how to act on misaligned intent from grounding queries
Determine if the best-fit page is missing or underrepresented. See if patterns emerge, including whether legacy content outranks newer terminology. If the incorrect page type is winning citations, that could indicate that the intended page is lacking the phrasing that is used in the grounding queries.
.png?width=636&height=358&name=Does%20the%20page%20intent%20match%20the%20query%20(2).png)
Applications to your brand beyond “GEO targeting”
When we started pulling on this thread, the tactical actions became obvious fairly quickly: updating metadata, strengthening internal links, making sure the pages with the most authority have helpful, useful and quality information that matches user intent. Sometimes the clear answer is to strengthen the existing page that should win. Other times, the right answer should be to acknowledge that the page is missing and therefore, your team should take action to create one that does.
Beyond these fixes, our team walks away with a thorough process for quickly evaluating the relationship between our brand and these critical, yet newly-pioneered industry terms.
In grey areas where we'd otherwise rely on guesswork, Bing Webmaster Tools gives us a rare glimpse into what LLMs are actually inferring about our brand - and where we can start filling in the gaps.
