The race issue

Andrew Sullivan has it half-right:

The whole point of the Muslim rumor is to provide racist voters with something non-racist to hang their hat on.

And I say "half-right" because I'm wondering: why is it any less bigoted for someone to vote against a Muslim than it is against a black man? I suppose you could argue in a very narrow sense that being Muslim is not the same as belonging to a racial minority, but rather, a religious one (though I would tend to disagree that most bigots make such fine distinctions). But even of you do make this argument, it is troubling that anti-Muslim bigotry is implicitly viewed as being an acceptable practice in our society.