Is it racist if no one objects?

This was a discussion I found myself in recently: Say a group of like-minded folk are sitting around, and one of them starts to spout off with what in another group would be considered racist, hateful speech (you pick a target for an example; I simply haven’t the heart). (And yes, I realize this is …