Keep in mind the manual safeties on most striker fired pistols like the SR-series, S&W M&P, etc. were added to meet several state requirements that any pistol sold in their state had to have a manual safety. They are a belt-and-suspender approach, not essential, as these guns have several additional safety features. Glocks have been sold for nearly 30 years without one.
That said, I do like the manual safety on my SE-9c as it makes the gun a bit proprietary. If it's picked up by a child or uninformed adult, they can't fire the gun by just pulling the trigger. I agree that flipping the safety off as you draw can become second nature pretty quickly and does not slow down getting the gun into action.