My line of thinking on the Confederate Flag is pretty much in line with Townsend's.
I've always maintained that even if the original intent of the Confederate Flag wasn't necessarily to embrace white supremacy and hatred towards minorities, that's pretty much what it's morphed into over the years. Just like with the Swastika, as was pointed out.
To the general public the Confederate Flag represents the ideals of white supremacy, hatred, segregation, etc. And when you see a Confederate Flag on a vehicle or flying in someone's yard.....it's a virtual lock that the owner of said vehicle/property is white. I've yet to come across a single person of color who embraces the confederate flag. Maybe there are some out there but I've yet to see one. And believe it's because the flag is something that is known more for divisiveness and hate because certain groups decided to embrace it and make it one of their calling cards. Again, just like with the Swastika.
As an aside, I expect there will be some talk in the near future about Mississippi redoing their state flag since this is one of the hot button issues of the week.