New research from Citations.io suggests brands with AI-facing instruction files receive more recommendations and more positive sentiment. But there's a significant difference between correlation and causation.
The rapid growth of AI search has created an equally rapid market for solutions promising to improve brand visibility across platforms such as ChatGPT, Gemini, Claude and Perplexity.
Some of these solutions are genuinely useful. Others are repackaged SEO principles, while a growing number are based on assumptions that haven't yet been properly tested.
One of the more interesting developments has been the adoption of llms.txt files.
The premise is relatively straightforward: provide AI systems with a structured, machine-readable document that explains your website, its content and potentially how an AI agent should interact with it.
The assumption that has followed is that publishing one should improve how frequently your brand appears in AI-generated answers.
But does it?
At Citations.io, we've just published research examining this question using 481,654 structured datapoints across two ecommerce studies, alongside an audit of 1,560 brand websites.
The findings are interesting, particularly because they don't necessarily support the conclusions that some in the industry might expect.
First, what exactly is an LLMs.txt file?
An llms.txt file is a proposed convention for providing AI tools with structured information about a website.
Typically published at /llms.txt, it can contain a description of the business, links to important resources and information intended to help compatible AI systems understand the website.
It's worth distinguishing this from robots.txt.
While robots.txt communicates crawling permissions, llms.txt is primarily intended to provide information and context. It doesn't automatically control how an AI engine crawls, indexes or represents your website.
There are also related files designed for specific use cases, including shop-style instructions intended to help AI agents understand ecommerce capabilities.
These distinctions matter because the research wasn't simply a comparison of every website with an llms.txt file against every website without one.
We specifically examined brands with shop-style instruction content in the checked files and compared them against brands where both checked root files were confirmed absent.
That gives us a more useful comparison, but it also places important limits on what we can conclude.
What did the research find?
We analysed two different ecommerce datasets.
The first focused on UK retail and gifting, producing 334,830 structured datapoints from AI-generated answers collected in September 2026.
The second examined US beauty and skincare, producing another 146,824 datapoints from July 2026.
Together, the studies covered 14,860 AI answers, with datapoints measuring individual brand mentions, recommendations, sentiment, ranked positions and cited sources.
We then audited 1,560 verified brand websites in October to identify the presence of relevant AI-facing files.
Three findings stood out.
1. Brands with shop instructions were recommended more frequently
When a brand was mentioned in an AI-generated answer, those with shop-style instruction files were more likely to receive an explicit recommendation.
| Study | No file confirmed | Shop instructions present |
|---|---|---|
| UK retail and gifting | 60.7% | 62.6% |
| US beauty and skincare | 85.2% | 94.4% |
That's an increase of 1.9 percentage points in the UK study and 9.2 percentage points in the US study.
The difference in the US beauty and skincare dataset is particularly interesting.
However, there's an important distinction between appearing in an answer and being recommended once you've appeared.
These results measure the latter.
They don't demonstrate that publishing a file makes your brand more likely to be discovered or mentioned in the first place.
And they certainly don't prove that the file caused the improvement.
Nevertheless, both datasets showed the same directional relationship.
2. Brands with shop instructions were described more positively
The second finding concerned sentiment.
One of the limitations of traditional SEO reporting is that rankings tell us very little about how a brand is perceived.
In AI search, that becomes a much more significant consideration.
A brand might appear frequently in responses, but if those responses are consistently negative, qualified or less favourable than competing brands, visibility alone could be a misleading measure of success.
Our research found that brands with shop-style instruction files had a higher proportion of positive mentions in both studies.
| Study | No file confirmed | Shop instructions present |
|---|---|---|
| UK retail and gifting | 57.1% | 65.9% |
| US beauty and skincare | 83.4% | 94.8% |
That's an 8.8 percentage point difference in the UK dataset and 11.4 points in the US dataset.
Again, that isn't evidence that an instruction file improved sentiment.
A plausible alternative explanation is that brands with these files may also have more mature ecommerce operations, stronger digital foundations or better-established reputations.
Some ecommerce platforms can publish agent-facing documentation automatically, making the existence of a file an imperfect indicator of deliberate AI optimisation.
We haven't isolated those variables.
What we can say is that the groups performed differently, and the difference was consistent across both datasets.
3. Better recommendations didn't translate into better rankings
This is perhaps the most revealing finding.
If publishing shop-style instruction files provided a straightforward AI ranking advantage, we might reasonably expect brands with those files to appear earlier in ranked recommendations.
That wasn't what we observed.
In the UK study, brands without a confirmed file appeared at an average ranked position of 3.01, compared with 3.30 for brands with shop instructions.
The US study produced a similar result: 3.24 against 3.42.
In both cases, the group with shop instructions appeared slightly later in ranked lists.
The differences were relatively modest, but there was certainly no consistent ranking advantage.
This highlights a fundamental problem with treating AI search visibility as a simple extension of traditional rank tracking.
Being mentioned, being recommended, being described positively and appearing first are four different outcomes.
They're related, but they're not interchangeable.
A business that measures only its average position within an AI answer could miss a meaningful change in how the brand is actually being presented.
The ChatGPT results were particularly interesting
When we separated the UK ecommerce findings by AI engine, the differences became considerably more pronounced.
For ChatGPT, brands with shop instructions were recommended in 94.3% of their eligible mentions, compared with 65.3% among brands without confirmed files.
That's a substantial difference of 29 percentage points.
Yet the other three platforms told a different story.
| AI engine | No file confirmed | Shop instructions present |
|---|---|---|
| ChatGPT | 65.3% | 94.3% |
| Claude | 59.3% | 57.4% |
| Gemini | 48.5% | 47.2% |
| Perplexity | 73.4% | 72.8% |
For Claude, Gemini and Perplexity, the brands with shop instructions actually recorded slightly lower recommendation rates.
It's important to emphasise that these results involve different brands, different numbers of mentions and potentially different contexts.
For example, the ChatGPT comparison included 511 mentions in the shop-instruction group against 8,191 in the no-file group.
We're not comparing identical brands receiving identical prompts before and after an implementation.
Nevertheless, the variation raises an important strategic question.
Why would we expect one technical implementation to produce the same results across AI systems that retrieve, evaluate and synthesise information differently?
The reality is that AI search isn't one homogeneous channel.
A brand's representation can differ substantially between platforms, and improvements made for one environment won't necessarily translate to another.
This is one of the reasons I've become increasingly interested in measuring AI visibility at the individual engine level rather than relying entirely on aggregated performance scores.
Correlation isn't causation, particularly in AI search
I think this is where the industry needs to be more disciplined.
It's becoming increasingly common to see a new AI optimisation tactic accompanied by screenshots, a handful of successful prompts or claims of improved visibility.
But demonstrating that two things occur together is not the same as demonstrating that one caused the other.
There are several limitations worth highlighting in our own research.
Firstly, the AI answers were collected in July and September, while the website files were checked in October.
We therefore cannot confirm that the files existed when the original AI answers were generated.
Secondly, brands weren't randomly assigned to groups. Companies publishing AI-facing documentation may differ substantially from those that don't, in ways that also influence their performance in AI search.
Thirdly, we cannot establish whether any of the AI systems actually retrieved or consumed those files when producing the answers.
And finally, the 481,654 datapoints represent structured findings extracted from the underlying AI answers, not 481,654 independent experiments.
These are important distinctions.
The results are useful for identifying patterns worth investigating. They are not sufficient to establish a causal relationship.
A more definitive test would involve tracking comparable prompts before and after adding an instruction file, ideally against an unchanged control group, while verifying whether the relevant AI systems accessed it.
Until that evidence exists, claims of a guaranteed visibility uplift should be treated cautiously.
Google has already clarified its position
There's also an important distinction between the wider AI ecosystem and Google's own search products.
In June 2026, Google updated its AI search optimisation guidance to clarify that llms.txt files aren't required for Google Search.
Google also stated that maintaining these files doesn't positively or negatively influence visibility or rankings in its search results, including its generative AI features.
That doesn't mean the format has no purpose.
It means we shouldn't confuse a potentially useful documentation convention with an established search ranking mechanism.
The real value of an llms.txt file may ultimately be in helping compatible AI agents navigate websites and consume information more efficiently.
That's a different proposition from claiming that publishing one improves brand recommendations across conversational AI platforms.
What should brands actually be doing?
I wouldn't advise brands to avoid implementing llms.txt altogether.
It's relatively straightforward to maintain, and there are legitimate use cases for supplying information to AI agents and other tools that support the format.
But I also wouldn't prioritise its implementation above the more fundamental work required to improve how a brand is represented online.
My approach would be to concentrate on four areas.
1. Establish how your brand currently appears in AI-generated answers.
Look beyond whether your business receives a mention.
Understand the questions that trigger those mentions, whether your brand is recommended, the sentiment of the response, and which competitors are being positioned more favourably.
2. Identify the information and sources influencing those answers.
AI-generated responses can draw on a combination of owned content, third-party publications, reviews, industry resources and other publicly available information.
Understanding which sources are being cited gives you a much clearer starting point for prioritising improvements.
3. Improve the underlying information before optimising its presentation.
For ecommerce brands, product information, pricing, delivery policies, availability, product comparisons and other commercially relevant details need to be accurate, accessible and consistent.
An AI-friendly summary cannot compensate for inaccurate information or missing content across the wider web.
4. Measure changes against comparable prompts over time.
If you implement an llms.txt file, improve product content or launch a digital PR campaign, establish a baseline first.
Then assess changes in visibility, recommendation rates, sentiment, citations and competitors across the same relevant questions.
Without consistent measurement, it's very easy to assign credit to whichever tactic was implemented most recently.
AI search optimisation needs fewer assumptions and better evidence
One of the things I find most interesting about the development of AI search is how quickly the industry is trying to establish rules around systems we still have relatively limited visibility into.
That's understandable.
Brands want to know what they should be investing in, agencies want repeatable frameworks, and technology vendors want measurable solutions.
But the speed at which a tactic becomes popular shouldn't determine how much confidence we place in it.
Our research found a measurable association between shop-style instruction files and higher recommendation and positive sentiment rates.
It didn't establish that those files caused either outcome.
And it didn't find a corresponding improvement in average ranked positions.
Those findings are useful precisely because they challenge a simplistic explanation.
I suspect we'll see a growing distinction between two disciplines over the next few years.
The first is making websites easier for AI systems and agents to understand and interact with.
The second is influencing how brands are discovered, evaluated, compared and recommended within AI-generated answers.
There's clearly overlap between the two, but they're not the same problem.
And solving the first doesn't automatically mean you've solved the second.
For me, that's the broader lesson from this research.
The objective shouldn't be to implement every new AI search optimisation tactic. It should be to understand which activities genuinely influence the outcomes that matter, and build enough evidence to distinguish those improvements from coincidence.
That's where the real value in AI search optimisation will come from.
This article draws on our October 2026 Citations.io research examining 1,560 brand websites and 481,654 structured datapoints across UK retail and gifting and US beauty and skincare. The study was observational and does not establish that publishing an llms.txt file causes changes in AI search performance.
Explore more AI search visibility research at Citations.io.
