Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The assumption is that "Google" is a good player, not a malicious one. Same idea as robots.txt. It's only there to configure the "good guys" behavior not to prevent malicious actors.

So I'm assuming Cloudflare is basically asking Google to split its crawler's UA and distinguish between search and AI overview and respect something akin ot robots.txt



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: