Performance issues with TT? Yowza... I am assuming you're using the XS version, helping it cache, have used a profiler to confirm that's really where it is etc. You must be serving a dramatic amount of dynamic traffic!
TT performance really can be a bitch. When I'm profiling Catalyst apps to improve performance I pretty much always assume that the first set of hotspots I'll find will be either Template Toolkit or waiting for database queries. The controller code and the Catalyst stack itself very rarely show up noticeably - and I'd suspect that Catalyst being the heaviest (or most featureful, depending on how you want to lookat it :) of the currently popular frameworks that that's likely to be true of everything else as well.
