No, Dan is right, it won't hurt anything involved with the pistol, but it just might be hard on some of the rounds you're hand feeding.
Consider, the feed lips on your magazines are designed and shaped so that they more, or less, guide a round pretty darn straight into the chamber mouth, so just...