It's "barrel time." The longer the bullet stays in the barrel, as the gun rises in immediate recoil, the higher the gun will shoot.
So...heavier bullets at the same velocity shoot higher than lighter bullets, due to increased barrel rise during recoil, before the bullet has a chance to exit the muzzle.
Slower bullets shoot higher than faster bullets of equal weight for the same reason.
One way I was able to prove this to myself was varying the bullet weight and velocity for many, many load recipes with my 6-inch barreled custom .500 Linebaugh Bisley years ago when I could still see well at 25 yds. I had John narrow my front sight blade width to 0.095" which gave me more light on either side in my sight picture. A narrow front sight blade may not be such a good idea in the field, for rapid acquisition, but it surely helped me to shoot good targets.
I still have my load books containing the specific load menus, velocities, group size, perceived recoil and point of impact relative to my aiming point.
With a big gun like that (large changes in recoil), it was/is easy to predict and see as much as six inches in elevation change when all I did was incrementally change the load velocity from 850 fps up to 1400 fps, or the bullet weight from 350 grains to 465 grains.
This was all bench-rested and documented, and many of my three to five-round groups measured from 0.6" to 0.75". No doubt in my mind at all.
It's only fair to say that not everyone, including some guys I respect a lot, agree with me on this. Of course, they weren't there, and I can still prove it.
Sonnytoo