The word “literally” is an intensifier; do you have the same objection to the use of ‘really’, or ‘actually’? ‘Literally’ was first used to mean “from the text” in the late 17th c. By the early 19th c. it was already fully co-opted as an intensifier.
It still doesn't feel like "literally" has been fully co-opted to me, at least not for anyone who isn't a teenager. I'd same the same about "actually", but I don't think I've even heard teenagers use it as a mere intensifier.
My theory of literally is that it's being used to indicate a literal rendition of the speaker's mental state. So it doesn't mean figuratively, it means "this exclamation is the most direct rendering of how I feel about this. " That is, it's literal because nothing was changed from the verbal impulse that immediately sprang to mind.