I've been looking at Chrome OS for this very reason. Apparently it includes a terminal app. I almost never have a program other than a terminal and a browser running, so it would work out perfectly.

That being said, since the Chrome OS devices don't have a full suite of command line tools, I'd need Internet access to attach to a VPS. Current cellular data pricing makes this a sketchy requirement for me.