How WhatsApp Crawls Your Pages

Understanding WhatsApp's crawler and Meta's shared crawling infrastructure

The WhatsApp crawler

WhatsApp uses two user agents to fetch link previews:

  • WhatsApp/[version] [platform] (e.g., WhatsApp/2.23.18.78 i for iOS)
  • facebookexternalhit/1.1: shared Meta crawling infrastructure

Previews are generated server-side through Meta’s infrastructure, not on the sender’s device.

Fetching starts before sending

WhatsApp starts fetching URLs as the user types, before the message is sent. Your server will receive crawler requests for URLs that are never actually shared. The upside: the preview is already cached by the time the recipient opens the chat.

No JavaScript execution

WhatsApp’s crawler doesn’t run JavaScript. Your <meta> tags must be in the initial HTML response. If they’re injected client-side, the crawler sees nothing.

HTTPS required

WhatsApp auto-upgrades all links to HTTPS. No valid SSL certificate means no preview. Self-signed certificates also fail.

Response requirements

The crawler has roughly a 10-second timeout and respects robots.txt. Target it with User-agent: WhatsApp or User-agent: facebookexternalhit.

Checking if WhatsApp can reach your page

Test with both user agents:

curl -A "WhatsApp/2.23.18.78 i" https://example.com/your-page
curl -A "facebookexternalhit/1.1" https://example.com/your-page

If your <meta> tags show up and the response arrives within 10 seconds, you’re good.