Your just not using your gears properly and driving it hard enough, keep the revs higher!
the advantage of a larger turbo is that you don't have to drive it on boost all the time, unlike the td04, so you can drive the car in a much smoother manner under 4000rpm.
also your plumbing before and after the turbo will affect spool - get a good air intake + piping + top mount inter cooler +y pipe and large bell mouth dump pipe and your spool up is a lot faster than a front mount setup, standard y pipe... etc.
I managed to get full power at 4200 from a vf22 which is slightly bigger than the vf34. Once you get used to the different driving style the wrx becomes a much better car imo.