>Too bad Bitcoin couldn't be used that way without a smartphone,

I think the minimal secure interface is something like a Trezor, which is pretty similar to what you describe.

Anything without some kind of confirmation UI can't possibly be secure.