I do little things here and there...almost all polishing operations. The only real aggressive work I did was on my Rossi Lever rifle. I took some springs, extractor, and some other parts to the grinding wheel and dremel, but I followed the guidance on a cd from Steves Gunz. The guy has worked hundreds of Rossi rifles. The work smoothed it and lightened the action a little, which is what I wanted.
If I ended up with an unreliable trigger or something like that, there's no way I would start gluing things to the gun...time to get new parts and start over. Still, TAB is right, it could happen to any of us so you really have to think about what you want to do.