I think that there's definitely a danger in using the civil rights movement as a catch all for other minorities.
But also more than highlighting the oppressed, pointing to the civil rights movement allows you to point out the people on the wrong side of history who were so fucking sure that integration would be the end of civilization, that interracial marriage was evil, and that science itself proved there was a superior race.
More than saying x has the same problems as black people, I think the reason for the comparison is to say, don't be so fucking sure that the status quo is acceptable, that's what they thought back then too.