As Jtsummers points out, people can and do give corrective upvotes in such cases. I do that all the time, and I hope people understand that it's part of being a good citizen here.
It's actually kinda useful as an indicator of something that could use a corrective upvote.
I could see putting a time delay on the second or third or later upvotes, but the slight graying seems like the perfect indicator for both "maybe skip it if you're not in that mood" and "maybe take a look if you're in a generous vein".
(Minus the accessibility issues, but they're fixable with CSS.)
https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...