Well "correct" in the case of linear vs non-linear luminance is pretty much exactly about perception (or at the very least, working to prevent perception differences further down the pipeline), even if it is a somewhat minor effect.
"The Importance of Being Linear"[1] which just came up on HN a few days ago, discusses some issues with non-linearity, specifically in the context of 3D rendering.
There are lots of specific examples where incorrectness leads to perceptually significant problems. I'd agree that you don't normally notice it in the wild, which is why so much software has gotten away with it for so long.
It is possible to optimize the linearization steps so that they don't detract too much from the performance, especially if you're using one of the better (i.e. slower) interpolation methods.
