I recently bought a server (xeon 5606 + 24GB Ram + 2 TB storage with hardware raid) for $2k and I am hosting it in a colo for $75. I find this ideal for small services and gives you a lot of performance allowing you to go a long way with a couple of machines before you need to scale. Adding more CPU/Ram/Storage will not add more to your monthly payment since you own the machine. Now of course it's not as convenient as AWS and if a machine goes down you are responsible to get it up again.
Scaling in AWS is a piece of cake and gives you a lot of flexibility but when it comes to performance, especially RDBMS, I find AWS to be far behind. Some will say that you can scale the service to provide good RDBMS performance but that will not avoid the per instance disk IO slowness.
If your network card fails or your CPU overheats, how long are you down for?
If you care if your website is up there are other costs when doing a colo. You should have spare for every component which nearly doubles your hardware cost and have someone on call for sys admin and hands-on fixes.
If I was you I'd just rent a server from ovh.co.uk / kimsufi.co.uk / hetzner.de/en and use a CDN. I am basing on your performance requirements that this is not a local gaming service. The advantage you have is you can upgrade to new hardware every year and the hardware is not your problem.
OVH is building one of the most advance networks with over 1tbps. Their 10gbps servers can actually push 10gbps which just gets me really excited.
Scaling in AWS is a piece of cake and gives you a lot of flexibility but when it comes to performance, especially RDBMS, I find AWS to be far behind. Some will say that you can scale the service to provide good RDBMS performance but that will not avoid the per instance disk IO slowness.