How Instagram Crawls Your Pages

Understanding the shared Meta crawler infrastructure behind Instagram link previews

The facebookexternalhit crawler

Instagram has no dedicated crawler. It uses Meta’s shared facebookexternalhit – the same bot Facebook and Threads use. User agent strings:

  • facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
  • meta-externalagent/1.1

If your site works with Facebook previews, it works with Instagram. Block facebookexternalhit and you break both.

No JavaScript execution

The crawler fetches raw HTML and parses it for OG tags. No JavaScript execution. If your SPA renders meta tags client-side, Instagram sees nothing.

The 60 KB parsing limit

The crawler only parses about the first 60 KB of your HTML. Large inline CSS or JavaScript in the <head> can push meta tags past this threshold. Place OG tags early – right after <meta charset>.

Where Instagram shows previews

Only two places:

  • Direct Messages – inline card with image, title, and domain
  • Story link stickers – tappable overlay with image and title

Links in post captions, comments, and bios do not render previews. DMs and Stories are the only surfaces where your OG tags are visible.

Testing if the crawler can reach your page

curl -A "facebookexternalhit/1.1" https://example.com/your-page

If OG tags appear in the first 60 KB of the response, Instagram will see them. The Facebook Sharing Debugger shows exactly what the crawler extracts.