First, as an aside, I (a white man) HATE the term African-American as referring to black people. My ancestors were from Russia, but I don't consider myself a Russian-American. Black, white, American indian, etc. etc. are races. People of any race who are American citizens are AMERICANS. PERIOD. To me, an African-American is a person of any race who holds dual citizenship.
"What if the white guy was speaking to a mostly white group like the NRA and said something like "You people have the power to get the right people into congress!" I don't think anyone would think that was derogatory to anyone."
Probably true, but still, the term "you people" does carry a condescending, us-them connotation. The speaker would have been bettter off saying just "You have . . ." or "You, the members of the NRA, have . . ."
I agree that these things can be over analyzed, but certain words and phrases do have implications that while not necessarily intended, are in fact heard.