Short Cache Causing Frequent Re-Fetching

Why Snapchat's crawler hits your server more often than other platforms

Why Snapchat re-fetches so often

Snapchat’s cache expires after about 30 minutes. Compare that with Twitter (~7 days) or LinkedIn (~7 days). Every share more than 30 minutes after the last fetch sends a fresh request to your server.

For popular links, this adds up to significantly more crawler traffic than other platforms.

When this becomes a problem

Usually a non-issue. It can matter if:

  • You’re rate-limiting aggressively – rate-limiting the Snap URL Preview Service causes some shares to produce no preview
  • Pages are expensive to render – heavy SSR pages may feel the load
  • You’re paying per request – serverless or metered hosting accumulates costs

How to handle it

  1. Whitelist Snap URL Preview Service in your rate limiter
  2. Set Cache-Control headers so your CDN can serve cached responses
  3. Make sure pages render fast
  4. Monitor access logs for the Snap URL Preview Service user agent to gauge volume

Using a CDN as a buffer

Put a CDN in front that caches HTML responses for 5-10 minutes. The CDN absorbs the re-fetch traffic while still providing reasonably fresh OG tags to the crawler.

This is by design

Snapchat prioritizes fresh preview data over reducing your server load. For most sites, the extra requests are negligible.