To me the difference is that #1 he wasn't included in the joke and #2 he hadn't even initiated it. If he'd said something like "bloody hell BB did you really need to put stairs this year?" and he'd stumbled... fair enough. If someone had said to his face "those stairs must be a bitch for you" considering the jokes he's said, then yeah. But in this instance, there was no joke. The dude was just cruising up the stairs. Laughing at him stumbling about the house or up the stairs when he's just doing his own thing isn't funny to me. He wasn't a part of the joke and wasn't even intending to be humorous. That's the difference for me.
He makes jokes about it, he invites others to make jokes about it - the HM's do on occasions.
The viewers & audience have picked up on this as well (except you) - nothing wrong with it. you're over reacting...