Dismissing all of this as some sort of right-wing moral panic avoids engaging with legitimate feminist objections.

It's not "insane" to question whether males should be allowed to access female-only spaces, or to resist the erosion of sex categories in law. Many women on the left have done exactly that, and have been vilified and harassed for it.