How Threads Crawls Your Pages

Understanding facebookexternalhit: the shared Meta crawler behind Threads link previews

Same crawler as Facebook

Threads uses the same facebookexternalhit crawler as Facebook, Instagram, and Messenger:

  • facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)

Blocking this user agent breaks link previews on all Meta platforms at once.

No JavaScript execution

The crawler doesn’t run JavaScript. OG tags must be in the server-rendered HTML. If your tags are injected client-side, the crawler won’t see them, same as with Facebook’s crawling behavior. You need SSR, pre-rendering, or static meta tags in the HTML shell.

meta-externalagent is unrelated

You may see meta-externalagent/1.1 in your server logs. That’s a separate Meta bot for AI training and content indexing and has nothing to do with link previews. Blocking it won’t affect Threads. Blocking facebookexternalhit will.

Shared infrastructure

Threads, Facebook, and Instagram share crawling infrastructure. Everything that affects Facebook previews (SSR, robots.txt, CDN config, redirect chains) affects Threads identically.

Testing crawler access

curl -A "facebookexternalhit/1.1" https://example.com/your-page

If your <meta> tags are in the response, Threads will see them. The Facebook Sharing Debugger shows exactly what Meta’s crawler extracts.