Properly encoding an mp3 is just a matter of using a recent version of lame and using one of the -Vn presets. It's dead easy but a lot of people don't seem to know this somehow. Ogg is a little trickier since some of the best tuning tweaks haven't been folded into the mainline code base yet.

There will probably always be "killer samples" that sound bad at any lossy bitrate. If you're really worried about this you're better off just going lossless instead of wasting tons of bits on across-the-board 320 kbit encoding.