This is why ABX testing is so big in lossy audio circles. People can and do demonstrate their ability to distinguish between lossy and lossless encodings with certain samples in double-blind tests, at all sorts of bitrates. I've done it myself occasionally.

That people have been doing this for many years is one of the big reasons modern encoders are so good - they've needed tonnes of careful tuning to get to this point.