A lot depends on the material the mags are made of…
When armoring Glocks, I pop the magazine base plates off, and disassemble the mags. I wipe everything down to get it dust-free, paying particular attention to the magazine follower. Then they're assembled dry, and they're ready to go.
My Ruger MKII mags don't normally get disassembled, as they're target guns and the mags virtually never end up on the ground. I wipe the feed lip area down with a powder solvent, if necessary. The blued mag bodies are wiped with a some sort of light rust preventative- I've used oil, Hoppe's No. 9, RustePrufe, etc. successfully. I keep any dust-attracting oil or "wetness" to a minimum, to avoid picking up dust/grit.