Block Meta-ExternalAgent Before the October 30 Crawl

On October 30, 2026, Meta gives itself the right to crawl your site. Not some distant news publisher’s site: yours, the moment you run a Page or an Ads Manager account. The awkward part is that this crawl leaves no trace in GA4. Meta-ExternalAgent can visit you every single day without ever showing up in your reports. Before you decide to block anything, you first have to measure. Here is the practitioner’s method to see clearly, logs in hand, and to judge each bot before the terms take effect.

What the October 30 clause actually says

The facts, without the drama. Meta’s revised Commercial Terms take effect on October 30, 2026, replacing the March 2024 version. They add a sentence to Section 1 that lets Meta use automated software and test credentials to “access, index, cache, copy and re-crawl” the websites associated with a business’s products. In plain terms: if you push a URL in your ads or on your Page, that URL falls inside the scope.

The acceptance mechanism is the part that stings most. It is passive: continued use of a Meta commercial product (Ads Manager, a Page) counts as agreement. The text provides no partial opt-out, and it does not mention robots.txt.

Then there is the messaging battle, which deserves a cold look. Meta plays it down: Andy Stone (October 16) calls the clause narrower than critics suggest, meant to pull reliable product information, “not about scraping news sites.” Daniel Roberts, in The Verge on October 17, says “the crawling terms we consolidated only applied to our ads and business products.” On the other side, Jason Kint (Digital Content Next) reads it as a right to scrape the open web, and attorney Matt Topic advises publishers to assume the clause is enforceable. One detail matters: the word “train” was in an early version (“research, train and maintain”). Between October 17 and 20, it vanished from the preview page. But only the verb changed, not the rest: the machinery to access, index, cache and re-crawl is still there.

I am not settling the legal question here, and this is not legal advice. What matters to us is the technical fact, and it is simple: from October 30, one or more Meta bots will hit your pages. The only useful question is which ones, how many, and on which URLs. And that can be measured.

The five Meta user agents (and why GA4 is blind)

First mistake to avoid: talking about “Meta” as a single block. Meta’s documentation describes five distinct bots, with five jobs and five different effects when you block them. Confusing them means making a blocking decision blind.

The trap to remember is mixed use. meta-externalagent serves both foundation-model training and content indexing to improve products, under one and the same robots.txt token. You cannot allow one and refuse the other: it is all or nothing. This is the exact same problem as the mixed bots on the Google side, which I cover in AI Crawlers: Which Ones to Block Before September 15, 2026?. Two other bots (facebookexternalhit and meta-externalfetcher) can also ignore robots.txt, because they act for an integrity check or a user request. Keep that in mind for what follows.

Why none of these bots show up in GA4

GA4 is a JavaScript tag that fires in a browser. These bots are not browsers: they fetch the HTML, full stop. They do not run your gtag, do not load the container, generate no events. As a result, your GA4 report will never show a single Meta-ExternalAgent visit. Even server-side, as long as the bot does not touch the collection endpoint, it stays invisible. The only source of truth is your server or CDN logs. If you only measure AI traffic in GA4, you are measuring exactly what these bots do not do. It is the same logic as tracking AI traffic in GA4: the analytics tool sees the human, not the machine.

Measure what Meta already crawls (logs/BigQuery query)

The prerequisite is having your server or CDN logs exported to BigQuery. If that is not done yet, the setup is described step by step in the AI crawlers article cited above, and I will not repeat it here. I assume you have a table with at least a user agent, a URL, a status code, a byte volume and a timestamp.

Here is a query that isolates the five Meta bots, counts hits, distinct pages, the share of error responses and the volume transferred:

SELECT
  REGEXP_EXTRACT(LOWER(user_agent),
    r'(meta-externalagent|meta-webindexer|meta-externalads|facebookexternalhit|meta-externalfetcher)'
  ) AS meta_bot,
  COUNT(*) AS hits,
  COUNT(DISTINCT request_uri) AS distinct_pages,
  ROUND(100 * COUNTIF(status >= 400) / COUNT(*), 1) AS pct_4xx_5xx,
  ROUND(SUM(response_bytes) / 1e9, 2) AS gigabytes
FROM `your_project.logs.requests`
WHERE TIMESTAMP_TRUNC(event_timestamp, DAY)
      >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 30 DAY)
  AND REGEXP_CONTAINS(LOWER(user_agent),
    r'meta-externalagent|meta-webindexer|meta-externalads|facebookexternalhit|meta-externalfetcher')
GROUP BY meta_bot
ORDER BY hits DESC;

Three readings are worth your time. Volume per day first: a sharp spike after October 30 tells you the clause changed something concrete for you. The most crawled pages next: if Meta mostly hits your product pages, the stakes differ from a bot vacuuming your blog. The 4xx/5xx rate last: lots of 403s means either you already block (on purpose or not), or your server is choking. And the gigabyte volume gives you the real cost, to multiply by your egress bandwidth rate to put a number on the bill these bots hand you.

The decision matrix before you block

Once the baseline is set, you decide bot by bot. The logic is not “block the AIs,” it is “judge each bot by what it costs you and what it earns you.”

The right move is to handle meta-externalagent first, because it is the only real judgment call. Blocking it is an ownership decision: you refuse to let your content feed Meta’s models, accepting that you also drop out of direct product indexing. The three “leave on” bots break useful things if you touch them, facebookexternalhit first of all. One clarification that avoids nasty surprises: robots.txt is a directive, not a barrier. Integrity or user-fetch bots can ignore it. For a block you want to actually enforce, you have to go down to the WAF or CDN level.

Blocking Meta-ExternalAgent: robots.txt and Cloudflare rule

The most common case: you want out of training without touching the rest. So you target meta-externalagent alone, and you let the indexer, the previews and the fetcher through.

# robots.txt: opt out of Meta training, keep the rest
User-agent: meta-externalagent
Disallow: /

Two pitfalls. First, Meta says a robots.txt change can take up to 24 hours to register, since the file is cached on the crawler side. If you are aiming for October 30, deploy by the 29th at the latest. Second, as seen above, robots.txt does not constrain the bots that ignore it. For a hard block, a CDN-level rule is the way. On Cloudflare, a WAF expression that blocks training while explicitly preserving link previews looks like this:

(http.user_agent contains "meta-externalagent"
  and not http.user_agent contains "facebookexternalhit")

You attach this expression to a “Block” action (or “Managed Challenge” if you prefer to leave an escape hatch). The second condition is your safety net: it guarantees you never break link previews, even if Meta shifts its user-agent strings.

Your action plan before October 30

Nothing insurmountable, but the window is short. Here is the order I would follow.

Before October 24: export your logs to BigQuery if you have not already, run the query above, and record your baseline (volume, pages, errors, gigabytes per bot). Without that starting point, you will never know what the clause changed.

From October 24 to 27: decide bot by bot with the matrix. For most sites, the only real question is meta-externalagent, and it is a matter of principle, not a technical emergency.

Before October 28: deploy the robots.txt you chose and, if you want a hard block, the CDN rule, keeping facebookexternalhit allowed. On the 28th, Meta switches on product browsing inside existing ads: check right after that you have not degraded your catalog.

On October 30: the clause takes effect. Run the query again 48 hours later and compare to the starting point. The delta is your real answer, far more than any press statement.

FAQ

Does Meta-ExternalAgent show up in GA4? No. It is a server-side crawler that does not run your JavaScript tag. Only your server or CDN logs see it.

Does blocking Meta-ExternalAgent remove me from Meta advertising? No. Ads and catalog run through meta-externalads and product browsing. Blocking meta-externalagent touches training and direct product indexing, not the delivery of your campaigns.

Is robots.txt enough to block Meta? It is a directive that cooperative bots respect, but some integrity or user-fetch bots can ignore it. For a block you can truly enforce, use a WAF or CDN rule.

Should I block facebookexternalhit? No. This bot generates link previews when your pages are shared on Facebook, Instagram and WhatsApp. Blocking it breaks those previews and gains you nothing on the training side.

The takeaway

October 30 is not a legal deadline you have to handle in a panic, it is a reminder: you should already know who crawls your site. Meta’s clause just makes the question impossible to ignore. Measure first, in your logs, then decide bot by bot, starting with the only real judgment call, meta-externalagent. And keep in mind that doing nothing is also a choice: the choice to let the October 30 text decide for you.