How Slack Crawls Your Pages

Understanding Slackbot and how it fetches link unfurl previews

Slackbot and link unfurling

When someone pastes a URL in Slack, two crawlers fire: Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots) fetches the page for meta tags, and Slack-ImgProxy 0.19 (+https://api.slack.com/robots) fetches referenced images. Neither runs JavaScript, so your OG tags need to be in the raw HTML.

The 32KB window

Slackbot sends a Range header requesting only the first 32KB of your HTML. If your meta tags fall outside that window, buried below large inline CSS or script blocks, Slackbot never sees them and the link won’t unfurl.

Put your <meta> tags as early in <head> as possible.

Client-side rendering

Slackbot won’t see meta tags injected by JavaScript. If your app renders OG tags client-side, you need SSR or pre-rendering to make them visible to the crawler.

robots.txt: ignored

Slackbot does not honor robots.txt. Slack treats the crawler as acting on behalf of the user who shared the link. Blocking it in robots.txt has no effect.

No static IPs

Slackbot runs on AWS with rotating IPs. There’s no published IP list, so IP-based allowlisting won’t work. Use the user agent string to identify Slackbot requests.

Testing access

Simulate what Slackbot sees with curl, including the Range header:

curl -A "Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)" \
     -H "Range: bytes=0-32768" \
     https://example.com/your-page

If your <meta> tags appear in the output, Slackbot will find them too.