Claude, Codex and Hermes Turned llms.txt Into an Enterprise Execution Risk

by Vytautas Gluoksnis, Performance Marketing & Web Engineering

Analysis

The important change is not that llms.txt can contain bad advice. Documentation has always been capable of containing bad advice. The reported issue is that coding agents with permission to run shell commands treated machine-readable web documentation as authoritative enough to install code from package names nobody owned. That turns an emerging GEO-facing publishing surface into an enterprise execution-control problem.

Researchers scanned 6,214 domains belonging to defense contractors, Fortune 500 companies and Big Tech firms, finding 8,265 llms.txt or llms-full.txt files. On 120 distinct sites, the files pointed to unregistered packages or domains; together, those files contained 227 commands to install non-existent packages or visit unclaimed domains, according to arstechnica.com. After the researchers registered selected unclaimed names and hosted beacon code, responses exposed parent-process chains involving Claude, OpenAI's Codex and Nous Research's Hermes.

For people doing GEO work, this changes the operational meaning of “make content legible to agents.” A file designed to help an agent understand a site can sit in the same retrieval path as setup instructions. If the agent can execute, the distinction between publishing information and supplying an actionable instruction is no longer merely semantic. It is a security boundary that needs technical enforcement.

This is package takeover through an agentic trust path

The reported mechanism is straightforward, but the path is unusually consequential. An llms.txt or llms-full.txt file includes a package-install instruction or a link to a domain that no longer has an owner. An attacker can claim that package or domain later. A coding agent that retrieves the documentation and is permitted to use the shell may then install or follow what it reads.

That sequence was not presented as a purely hypothetical scenario. The researchers registered a handful of unclaimed names and received a phone-home response from a Fortune 500 company within an hour, followed by a few dozen more responses from Fortune 500 companies and startups. The beacon data ultimately identified the coding-agent parent processes, arstechnica.com reported.

One case had already progressed beyond beacon code. A legitimate clerk.com LLM file included the command npx clerk-next-fix-auth-protection. Someone had claimed the previously empty package name and used it to host live malware; Clerk resolved the issue. Ars noted that it remains unclear whether this confusion resulted in actual infections, so that should not be overstated as evidence of a completed enterprise compromise. It is, however, evidence that the unowned-name condition can be actively exploited rather than merely observed in a scan.

The npx detail matters because it can fetch a package into npm’s cache and execute its exposed binary without adding the package to the project dependency manifest, according to arstechnica.com. A dependency manifest is therefore not a complete retrospective record of what an autonomous workflow may have run.

GEO assets are now security-relevant inputs

llms.txt and llms-full.txt are described as an emerging convention for machine-readable summaries of a website’s content and high-level structure: an AI-oriented analogue to robots.txt. That makes them attractive for GEO because they can give agents a concise representation of what an organization offers, how its content is organized and where supporting documentation lives. It also makes them an unusually direct supply of context to an agent.

The security consequence is not that GEO teams should abandon agent-readable publishing. The evidence does not establish that all agents automatically execute content from these files, nor does it establish a universal behavior across products and configurations. The vulnerability occurs when a coding agent that has shell-command permission treats the retrieved file as authoritative setup documentation, and some agents then download and run the referenced package, arstechnica.com reported.

But GEO teams should stop treating these documents as copy-only assets. They can contain executable-adjacent strings: installation commands, package names, SDK references, example domains and links. Each can outlive its original owner. A package name that was harmless when written because it was unused becomes dangerous if an unrelated party claims it later. Researchers said many faulty entries predated the AI era in conventional website files, suggesting human authorship in those cases; they suspected others may have been AI-generated hallucinations. The available reporting does not determine the origin of every bad entry.

That uncertainty is itself useful. A security review cannot assume the problem is confined to careless AI-generated docs, a single publishing tool or a new llms.txt rollout. Old examples copied into a new agent-readable summary can carry the same abandoned references forward.

HTTPS and an official domain do not establish package provenance

This incident exposes a trust mismatch. An agent retrieving an llms.txt file sees HTTPS, an official domain, a standardized AI-facing format and content published by a company or trusted partner. Those signals can make the file look legitimate. They do not prove that a referenced PyPI or npm namespace belongs to that company, or that a linked domain has not expired and been re-registered.

As the researchers put it, “An agent doesn’t distinguish between a page and a command.” Everything it reads is input, and each input can become an instruction; they argued that the published corpus wired into agent retrieval has become an execution surface without the integrity guarantees normally applied to code, yro.slashdot.org.

For enterprise GEO, the practical implication is that first-party status must not be collapsed into software provenance. A vendor’s own documentation may be authentic while the external package it names is unowned. A trusted partner’s documentation can create the same problem because the chain is transitive: an agent that trusts the partner can follow its reference to an unclaimed name. The reporting explicitly says the llms.txt file need not be hosted by the affected Fortune 500 company itself for that trust chain to operate, arstechnica.com.

The controls belong at the instruction-to-execution boundary

The report is particularly useful because it identifies why downstream controls may miss the event. To endpoint detection or a proxy, a coding agent installed intentionally by the company may look like a developer using a legitimate package manager to access pypi.org, a domain corporate proxies already allow. The reported failure sits upstream: the system did not ask whether the instruction and package ownership were trustworthy before execution.

That argues for three distinct controls rather than a generic instruction to “be careful with prompt injection.”

  • Constrain permissions. An agent that retrieves public documentation should not automatically inherit authority to install packages, execute binaries or access sensitive internal resources. Shell access is the condition that converts this class of retrieved instruction into a direct execution path.
  • Verify provenance at the package and domain level. The relevant question is not only whether the document came from an official HTTPS site. It is whether the named package, namespace or linked domain is owned and approved for the intended use. This check must happen before execution, not after an endpoint tool observes package-manager traffic.
  • Sandbox agent work. Where an agent needs to test installation or integration steps, use a constrained runtime rather than a corporate endpoint, cloud environment or developer workstation with broader access. The source reporting does not prescribe a particular sandbox design, but its evidence strongly supports minimizing the consequences of a bad retrieved instruction.
  • Review GEO documentation as code-adjacent inventory. Search llms.txt, llms-full.txt and the documentation they summarize for install commands, package references and external domains. Remove stale examples and unowned references, and treat changes to these files as security-relevant publishing changes.

These are not interchangeable. A sandbox without provenance verification still permits untrusted code to run. A provenance check without permission limits leaves more room for errors and bypasses. Removing stale instructions on owned sites does not protect an organization when its agents retrieve a partner’s content. The control model has to account for the full chain: retrieval, interpretation, package resolution and execution.

Prompt injection is the underlying limit, not a side note

The report connects this issue to the broader prompt-injection problem: language models cannot reliably separate direct user instructions from instructions found in untrusted retrieved content unless specific guardrails block them. Here, the dangerous text need not have been malicious when it was published. It can be ordinary, outdated setup guidance that later points to an attacker-controlled package or domain.

That distinction matters for GEO governance. A content team can publish entirely good-faith material and still create a future execution risk if it embeds references whose ownership is not maintained. Equally, an enterprise can run a well-configured agent and still accept a dangerous chain if it gives public retrieval too much authority. As summarized by daily.dev, the observed risk comes from agents treating vendor-provided documentation as trusted ground truth and installing referenced packages or following domains without verifying that they remain safe.

The evidence is thin on some questions: it does not quantify how often agents encounter these files, how many attempted installs were blocked, or whether confirmed infections occurred. It does not show that every Claude, Codex or Hermes deployment behaves identically. Those limits should shape the response. This is not proof that every agent-readable document is malware delivery. It is proof that enterprises cannot safely rely on document authenticity alone when an agent can turn retrieved content into actions.

For GEO practitioners, the new standard is uncomfortable but clear: agent-readable publishing is part of the software supply chain whenever an agent can act on it. The document may be content, but the capability assigned to the reader determines whether it can become code.

Key takeaway

Treat every retrieved GEO document, including llms.txt, as untrusted execution-adjacent input: agents should not install packages or follow external domains without explicit provenance validation and a constrained runtime.

Sources

Share this

More articles

Want this measured on your own site?

Run a free GEO audit on any page — no account needed — or have the monitoring handled for you.