Blocked crawler (usually by accident)
TikTokSpider is likely blocked by your robots.txt or firewall. Many site operators block ByteDance crawlers broadly to stop Bytespider (the AI training bot) and inadvertently kill TikTokSpider too.
Diagnosis checklist
- Check
robots.txt– isTikTokSpiderdisallowed? Look for blanket blocks on all bots. - Check for OG tags –
curlyour URL and verifyog:titleandog:imageare in the HTML. - Check server-side rendering – client-side-only meta tags won’t be visible.
- Check HTTP status – must return
200 OK. - Check firewall/WAF rules – some WAFs block ByteDance IP ranges, catching TikTokSpider as collateral.
- Check HTTPS – valid certificate required.
Allowing TikTokSpider while blocking Bytespider
User-agent: TikTokSpider
Allow: /
User-agent: Bytespider
Disallow: /
This keeps link previews working while still blocking the AI training crawler.
Testing your fix
curl -A "TikTokSpider" https://yoursite.com/page | grep -i "og:title"If you see the tag, share the link on TikTok to verify.