I think Apple Mail actually strikes the right balance here.
There are icons, but they have text below them by default. If you want a cleaner interface once you are able to easily identify the feature by the icons, you can right-click and turn off the text.
It's like training wheels for the UI. I used the text labels until I no longer needed them, and now I only use the icons.