Maybe it's because men aren't as aesthetically pleasing (just don't tell my husband that)
Joking aside, the males I draw tend to have softer features and sometimes look slightly feminine or boyish. My style just isn't very... rugged. I'm sure you'll get the pictures to look more like you want the more you practice. Have you tried doing a lot of very quick (as in less than 60 seconds) sketches of various males? That might help you get a feel for the different angles and placement of things.