I grew up shooting. I shot everything as long as it wasn't a handgun, as my dad did not like them. I started at about 6 years old, and continued all the way through high school. We lived out in the country, and I was one of "those guys you hear about" that would sneak guns out of the cabinet after school and go behind the shed for some good plinkin' fun, and would often get up way too early to do the same.
Once I got done with high school, I joined the Army. Their stuff was phenomenally easy to shoot. Loved the M16. Loved the M4. Loved the M9. Loved the 1911. Loved the Grease gun (no accuracy of any sort, but still fun). Loved the M60. Loved the M249. Loved the M240B. Really loved the M2. Maybe not as much as the 240B, but close. Could shoot all of them reasonably well, and still jump on them every chance I get.
Probably 12 years after joining, I picked up my first Blackhawk. It was quite the learning curve. 6 years after that, it still is. To irritate my wife, I shoot my 1911 off of a rest, and kick bottle caps downrange to about 40 yards, where I start missing consistently. With that gun, I can hit every reasonable target within my 50 yard limit consistently. I've found that I can shoot my 1851 very well. Much better than my other single actions. I struggled with the Blackhawk. I can shoot it much better with wood grips on it than I can with the black plastic grips, and slippery plastic grips are even worse.
So.... where I'm going with all this rambling...
Don't worry about the other stuff. For me, I had to learn to start all over. I blamed the gun for a long time, but it was really me. Different grips helped. Consistency in grip, and dry firing helped even more. And range time helped even more. (I'm not trying to say its you here, so don't get me wrong... Just look at yourself and be willing to be humble... I realize it could be the gun.)
If you are sure it's the gun, get on a rest and shoot it - even if the rest is a wadded up sweatshirt or an old blanket (just beware the flames at the front of the cylinder). Do what you have to, to take that part out of the equation. If you can hit everything off of the rest, you know the next steps. If it's still trouble, then I'd start looking at the gun more. Also, bring a critical friend along to watch you shoot. They'll point out mistakes if you're making any.
Then shoot it, shoot it, shoot it. Despite being more difficult for me, my single actions are still my favorites. (Also, I've also discovered that I shoot the mid-frames better, and it's also much easier for me to shoot short barreled guns.)