It does sound like you have eyes fighting for dominance, Tape over you weak shouldered eye and try it again, don't tape your eye but your eye glasses. Then give us a report, I have a similiar problem with a shotgun, my left eye likes the sight picture better than looking over the shotgun, with my right eye, means I shoot way off in one direction, unless I put scotch tape on my left eye glasses. Then it works much better.
Report back to us.