since switching to the 9mm Glock platform, I don't have the maintenance problems I had with 1911 magazines, likewise switching to the PMag for the AR....I have a couple of the older AR mags loaded in the safe, but for my competition blasters, only 4 PMags and 4 Glock mags.... do you know something I should be aware of ? 
As Michael says, you want plenty of mags for guns that might be discontinued (say a Winchester 77 or Remington 66).
And some competitions may only require 1 or 4 magazines per stage. Glock (GSSF), for example, only uses 3, 3 & 4 for the three stages so, yeah, 4 are enough, but by taking 10 I don't have to fuss with reloading while at the range. I have a mag pouch for each stage ready to go. At home, at my leisure I can check and double check to make sure each is properly loaded. Under the heat, cold or rain at the range it's easy to make a mistake. Same with IDPA and my revolver. I take enough speedloaders that I don't have to worry about re-freshing any of them during a match. It usually works out, for an indoor match to about 10.
Then lastly of course is the idea that having lots of extra mags could come in handy in some situations. I'm not a "zombie apocalypse" person, but I can understand having plenty on hand.