Well-designed sitemaps and use of something like https://www.indexnow.org/ helps... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

deceptionatd 6 months ago | parent | context | favorite | on: Show HN: I built a tool that watches webpages and ...

Well-designed sitemaps and use of something like https://www.indexnow.org/ helps.

Cloudflare has Crawler Hints which works well IME: https://blog.cloudflare.com/crawler-hints-how-cloudflare-is-...

hinkley 6 months ago [–]

Do you know if cloudfront has anything like this?

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact