They have to be scanned in every collection and G1 scans them in a stop-the-world pause. Other references may not be scanned at all in most collections (partial), and when they are, G1 scans them concurrently. They're less of a problem with ZGC.