I don't see why people have an issue with Ben. A lot of the comments that I've seen (more so on Facebook than here) are really just lame attempts by douchebags to make themselves feel better by putting him down for his supposed inadequacies. But honestly, who gives a fuck whether he can cook vegetables or not? Who cares if he wears his pajamas all day? These sorts of things do not have any bearing whatsoever as to whether or not I like a person.
It's also very understandable that an introverted gay man would have great difficulty in finding a partner (another supposed failure that people have tried to criticise him for). He has said on the show that he rarely meets gay people because he never goes to gay clubs. And who could blame him? As a straight guy I'm eternally thankful that that I don't have to go to a nightclub and listen to the bloody Scissor Sisters every time I want to meet someone.