A commercial driver has a physical every year, severe restrictions on how they can drive, hours behind wheel, etc. But if you have 2 cataract surguries, oncoming dementia, and a body full of arthritis that can barely move, you can drive a 50' Class A Motorhome to your winter home in the South.
Sounds utterly ridiculous I agree but there's something to be said for 60+ years of muscle memory, otherwise olds rocketing motorhomes off of cliffs or augering into the side of the local walmart would feature more prominently in the local news. Quit picking on paw paw and keep the fight where it belongs: suburbanite pavement princesses must die.
There are also countries that reduce the time a license is valid starting at a certain age and that require a medical test for renewal at all ages. IIRC Spain does that.
To do that you have to first recognize that current city designs are 100% for car manufacturers, not people. To take away the olds licenses is to restrict them in their homes, like people under already are. It's freedom they're after.
Change the culture around freedom of movement, and I guarantee the olds will give up their keys before they're even old.