My impression - and it's only that - is that many of the objections come from radical feminists (self-styled), who tend to be far more focused on trans women anyway, and on erecting a cordon sanitaire around their own preferred definition of "women".