It also sounds pretty strange to me to take the number of 5 previous mass extinctions as unshakable fact. I'm not a paleontologist, but in the several billions of years before living organisms evolved shells/skeletons that could be fossilized, there could have been any number of mass extinction events that we no longer have evidence of?
There is evidence of a lot of mass extinctions. The Big Five is just the biggest 5 we know off (all of them feature atleast 75% of all species becoming extinct on a relatively short timeframe, with the quickest being the KT which lasted at most a thousand years, the biggest of them, the PT event, making 95% of all species extinct). For smaller extinction events, Wikipedia lists about 30 of them. But those aren't nearly as interesting because they didn't have that much impact.
I agree with this. In fact - it's reasonable to take the same attitude toward these numbers as "age of the universe, in billions of years" type numbers: the error bars are pretty big! But the OOM is basically correct, and that's Good Enough (tm) for most people's purposes, including academics in far away fields like climatology.