There are some relatively minor differences between XD and XDM. Part of the price difference is in the improved trigger and match grade barrel. Quite frankly....disregard both of those. Ask yourself if the price difference is worth the extra couple rounds and the extra safety feature in the disassembly process.
My personal preference now (after owning an XDm) would be to buy the XD in the compact size and put a Powder River trigger kit in it. You will then have about the same amount in the gun and the trigger will be absolutely perfect. Even if you do buy an XDm, you will still want a trigger kit as the difference in the before and after is tremendous.
As for not having to pull the trigger to take it apart.....well lets just say that if you are following proper safety guidelines then it is not a deal breaker. You should always be checking and triple checking anyway.