Clothes don't have gender but certain genders don't need to be wearing certain clothes. For example, guys don't need to be wearing skirts and dresses, unless it's a kilt or traditional cultural wear for men.
If girls, can wear guy clothes and pull if off as "sexy" or "cute" with no reason of traditional cultural wear , what is the difference between men? Some women are into men wearing women's clothes, a guy wearing women's clothing shouldn't take away his masculinity. I do see your point. Thank you for sharing.
Clothes can be used to identify gender - or to distinguish between cis- and transgender. If I saw a bearded person in a dress, I'd assume she was trans or he was on his way to a costume party.
Benedict Arnold 