Well i just drove a 335i and a mint condition low km 2006 E46 M3 back to back- if you can deal with lack of warranty the M3 shits on the 335i IMO. The M3 might not actually be any faster but it feels so much better, the noise, the look, its a sense of occasion to drive the thing.
Needless to say i bought the M3
I would say unless you are hell-bent on 2008+ drive an M3 if you can