How TikTok Crawls Your Pages

Understanding TikTokSpider and how it fetches link previews

The TikTokSpider crawler

TikTok uses TikTokSpider to fetch pages for link previews in bios, comments, and DMs. It runs server-side and does not execute JavaScript – OG tags must be in the initial HTML.

TikTokSpider vs. Bytespider

ByteDance operates two distinct crawlers, and the difference matters:

  • TikTokSpider – fetches pages for TikTok link previews. Respects robots.txt.
  • Bytespider – general-purpose crawler for AI training and content indexing. Commonly blocked.

These are separate bots with separate user agents. Blocking Bytespider does not affect TikTok link previews. Allowing TikTokSpider does not grant access to Bytespider.

No JavaScript execution

TikTokSpider won’t see meta tags rendered client-side. If your site is a SPA, you need SSR or pre-rendering.

Response requirements

Expects 200 OK with text/html. Redirects are followed. Auth-protected or IP-restricted pages won’t generate previews.

Checking if TikTokSpider can reach your page

curl -A "TikTokSpider" https://example.com/your-page

If your <meta> tags are in the response, TikTokSpider will see them.