The short versionOn October 17, 2026, The Verge reported that Meta removed language from its pending Commercial Terms that suggested it could scrape content from sites to train its AI. The revised clause still lets Meta use crawlers, and create test credentials, on sites associated with businesses that use Meta for ads or Pages. The terms take effect October 30, 2026. Meta says robots.txt still applies, so check yours.
Meta has walked back the most controversial line in its new business terms. The part about crawling your website stays. Here's what changed, and what didn't.
What happened
"After backlash, Meta is updating its terms again to remove language that suggested it could scrape content from sites, including news sites, to train its AI," The Verge reported on October 17. The removed sentence said Meta "may use any content from such domains, web pages, or URLs to provide, develop, improve, research, train, and maintain the integrity of, the Meta Company Products."
Meta spokesperson Daniel Roberts told The Verge: "As always, content owners can use robots.txt to signal their preferences. This is not about scraping news sites, and these updates don't give us new rights that supersede those preferences."
The details
The crawling permission stays. Unite.AI reports the revised clause still authorizes Meta to use automated software and to "create test credentials" to crawl domains associated with a business's products or services, and that "the verb train no longer appears in the provision."
Who it covers. The terms preview says the updates "will go into effect on October 30, 2026," and defines business purposes to include using ads or marketing messages and managing a Page, among others.
robots.txt. Meta's crawler documentation says: "In order to block these crawlers, add a disallow for the relevant crawler to robots.txt." It also says one crawler, Meta-ExternalFetcher, may bypass robots.txt because it performs fetches requested by a user.
Why brands should care
If you run Meta ads or manage a Facebook Page, these terms apply to you from October 30. The AI training sentence is gone, but Meta can still crawl sites tied to your business. Whether that's fine depends on what's on your site.
What to do
Our suggestions, not legal advice:
- Open yoursite.com/robots.txt. Decide what Meta's crawlers, like meta-externalagent, may read, and add rules if needed.
- Put private content behind a real login. Don't rely on robots.txt alone for anything sensitive.
- Read the final terms before October 30. They've already changed once.
This follows our earlier story, Meta's new terms let it scrape your website.
Sources
- The Verge: Meta removes language that may have allowed it to scrape news sites (Oct 17, 2026)
- Unite.AI: Meta rewrites web-crawling clause in pending Commercial Terms (Oct 17, 2026)
- Meta Commercial Terms preview
- Meta for Developers: Meta Web Crawlers
FAQ
Did Meta drop the AI training language from its Commercial Terms?
Yes. The Verge reported on October 17 that the sentence saying Meta may use content from such domains to "train" Meta Company Products no longer appears, and Unite.AI found the verb train no longer appears in the provision.
Can Meta still crawl my business website?
The revised terms still authorize Meta to use crawlers and create test credentials on domains associated with your business if you use Meta for business purposes, like ads or managing a Page.
How do I block Meta's crawlers?
Meta's developer documentation says to add a disallow for the relevant crawler, such as meta-externalagent, to your robots.txt file.
