Ruger SR45 breakdown problem

When I break down my SR45 for cleaning, it comes apart easily enough. You have to push the ejector down with the slide open and pull out the pin before disassembling. So I do that. It sometimes sticks a little bit, but not too badly, I can usually get it apart with 2 tries or so. I know, that's not great.

Then, I clean it, and try to put it back together.
I make sure that the ejector is still down and run the slide back towards the assembled slide-open position.
BUT, the slide stops when the back of the slide reaches the ejector.
It seems like the ejector might be misaligned or something, because the slide hits it sometimes.
It usually takes me 5 or 6 tries to put it back together.

I actually haven't gotten to shoot it yet, so hopefully it's okay to shoot. But I was wondering if anyone else has had this problem and how I might fix it.

Thanks,
Tom

is empty mag in?

Nope

Sure the slide's not hanging up on the upper tab of the trigger bar? Both my SR9 and SR45 will from time to time.