The trouble is that the default setting, the do-nothing, don't-care follow-the-norm setting, is anti-feminist. It's the "double standard" in dating. It's women as decoration. It's compulsory sexuality. It's misogynist porn. And, as here, it's sexist jokes. This is why feminism says we live in a patriarchy still.
I see where you're coming from, and agree wholly, when I put a feminist hat on. The thing is, when I then put an 'indiffeminist' (thanks zem) hat on, I feel like you're throwing insults, saying "Oh, you're not a feminist activist? Then you're a misogynist pig."
I think it boils down to the technical meaning in feminist theory of 'anti-feminist' (anything that holds back feminism) not really matching up with what the layperson seeing the word for the first time would take it to mean -- 'anti-feminist' intuitively sounds like it only applies to active, concious opposition to the ideals of feminism; misogyny, denial of female autonomy, Rush Limbaugh, that kinda thing. Laypeople may not think to also consider apathy that unwittingly enables the sexist status quo as part of anti-feminism.
I'm definitely not saying you're wrong, I'm just saying you risk alienating your target audience by... well, to be flippant, by using big fancy words they may not understand ;)