One thing to consider are some 5.11 Covert Khakis (I've picked up a bunch of them since they seem to be on clearance everywhere). They look great (like regular work pants, not the 'tacticool' 5.11 look), and have a really slick double-pocket design...so you have a regular pocket, and inside that an easy-to-access zippered pocket...that has magazine carriers built right in.
The inner pocket is actually large enough to pocket carry an LCP (in holster) with a spare magazine, so they're plenty roomy!