Depending on what sort of driving you do, noise can be a serious issue.
Noise outside the car- If it is louder than legal will make for tiring driving, and you will be pulled up. Inside the car, noise may be even more important. Noise inside the car is bad but droning makes driving a struggle. Some of the better exhaust brands have tested on a range of vehicles.
Droning - a resonance when the car structure amplifies the sound over a range of rpm. The driver will sub-consciously avoid that rpm range so getting poor performance and economy.
grumpyrex's suggestion is good advice - ride in few different cars.