Not necessarily racist, no. I would not particularly want to sit through a sermon in a foreign language. I have, though, when I have attended Mass in other countries. My desire to receive the Sacrament, and my respect for my Sunday obligation, easily won out.
I am not disturbed that these people preferred to hear a sermon in their own language. I am disturbed that they walked out in disgust. I am disturbed that they were not sufficiently concerned about their fellow Catholics who could not understand the sermon in English to offer up this very small sacrifice for their good and spiritual instruction.