In general, as long as you stay with the same type, (small pistol for brand X & Y & Z) and stay within the safety guidelines, you'll most likely NOT see any differences worth noting.
However, that said, in rifles, there can be a bit more of a noticable difference, in groups sizes SOMETIMES. It all comes down to how much of a difference you expect. if I'm shooting a rifle, with components I've found to be very good, (say 1/2" groups at 100 yds,) and all I switch is the primer and my groups go to 5/8', then I'm only gonna worry if I'm shooting it in competition. But for general field use, I wouldn't care.
In handguns, most folks won't be able to tell any differences.