What do you mean by 'damage'? if you mean "require a jetting change" the answer is that it depends on what you replace the stock mufflers with. If the new mufflers flow differently than the old ones did, then a rejet will be required.
Note, also, that everything in the exhaust system rusts together. Removing the pipes without damaging the H box is quite doable, but usually not easy.