What exactly does Howard say that's racist? He certainly doesn't call Mexicans stupid. Gilbert definitely delivers shockingly racist humor, but some people accept that he's joking, and some don't.
And that's a big point. A good comedian can say humorously racist things, when it's clear that he's joking. A Jew can joke about Jews, and blacks about blacks. Because it's obvious they're joking. But sports talk people aren't professional comedians, so it's not clear that they're joking.
When Howard goes on rants about Middle Eastern people, it's not clear to me that he's joking, so I do find that offensive.