If your company manages a Facebook Page, you are about to accept new terms on October 30. You probably did not read them. The interesting part is a clause about crawling your website.
A recent piece on Search Engine Journal tracked how Meta revised that clause after publishers objected. The short version: one word left, the permission stayed.
What changed, and what did not
The original wording, as reported by Nieman Lab, said Meta "may use any content" it collected to "provide, develop, improve, research, train, and maintain the integrity of" its products. That set off objections from the European Publishers Council and the UK's News Media Association.
The revised clause removes "train". Crawling is now tied to developing and maintaining "advertising and business products", plus "the integrity and security of" Meta's products. Meta's spokesperson said the update "does not change how Meta uses web crawlers" and that it is not about scraping news sites.
What stayed is the access. Meta can still use automated software, create test credentials to gather information from websites associated with your products or services, and periodically pull updates. The new language does not mention robots.txt at all.
The legal experts quoted in the piece were blunt. Media attorney Matt Topic said "develop" is broad enough to include "train", and noted that "business product" does not appear to be a defined term. Cornell professor James Grimmelmann called the purposes language "still broad enough to drive a truck through".
There is also an inconsistency worth knowing. As of October 19, the UK English preview of Meta's Self-Serve Ad Terms still contained the old "may use any content ... train" wording, while the US English version did not. When two versions of the same contract disagree, assume the broader one until someone tells you otherwise in writing.
Robots.txt was always a request
Professor Benjamin Sobel made the point that matters most for operators. Robots.txt signals a site's preferences. Following it is voluntary, and it does not give the site an enforceable legal right.
I made the same argument a while ago in disallow was never a privacy setting. What Meta's terms add is a new layer on top. If you accept a contract that grants crawl access, the question of what your robots.txt file says becomes secondary to what you signed.
Meta does publish separate crawlers, one for AI training and one for Meta AI search. So you can allow one and block the other, in theory. It is unclear from the revised terms whether those settings limit what you have already granted by accepting the agreement.
Why "develop" matters more than "train"
The fight over one word hides the real mechanics. Training a model is only one way to extract value from crawled content. Grounding answers, building product catalogues, matching ads to pages and evaluating model quality all use the same data, and none of them needs the word "train".
That is why the legal reaction focused on "develop". In engineering terms, almost everything a platform builds is development. A clause that permits crawling to develop business products covers a very wide surface, whatever the intention behind it.
Most marketing teams think about AI and content in one dimension: did the model learn from my pages or not? The more useful question is where your content ends up working, and whether you get anything back when it does.
Terms of service are the new crawl policy
This is the pattern I expect to repeat. Platforms will not ask you for training data in a separate negotiation. They will fold access into the terms you accept to run ads, keep a business page, or use a free tool.
That shifts where the decision gets made. For most companies, robots.txt is owned by a developer and terms of service are clicked by whoever set up the ad account. Nobody in that chain is thinking about whether the company's product data, pricing pages or original research end up feeding someone else's model.
It also changes the value of your content. I wrote about how training data has lawyers now, and settlements are starting to put a price on it. Granting broad access through a checkbox is giving that asset away at zero.
There is a quiet imbalance here. Meta can change the terms with a preview and a date. You accept them by continuing to run a page you have run for ten years, because leaving costs you reach, customers and ad history.
That is not a reason to leave. It is a reason to treat platform terms like any supplier contract, with someone tracking versions and flagging changes. Most companies do this carefully for a software licence and not at all for the platforms where their brand actually lives.
What I would check before October 30
Open the current Meta Commercial Terms preview for your region and read the crawling clause yourself. Meta has already updated the preview more than once, so check again close to the date. Save a copy with a timestamp.
Then review how your robots.txt treats each Meta crawler separately. Decide on purpose whether you want to appear in Meta AI search and whether you accept AI training, instead of inheriting a default. Write the decision down.
Finally, give terms of service an owner. Someone in marketing or legal should know which platform agreements the company has accepted and what they grant. Ten years ago that was boring compliance. Now it is part of your content strategy.
Meta changed one word. The permission underneath it is the same size it was last week.