"If your key is on a smart card it can not be stolen, even if the machine you're ssh'ing from has been compromised."
If the machine you're ssh'ing from has been compromised by a halfway competent attacker, there's no need for them to get the key from the smart card. They can log in to the remote machine by simply piggybacking on to your ssh session (with a compromised ssh client).
If the machine you're ssh'ing from has been compromised by a halfway competent attacker, there's no need for them to get the key from the smart card. They can log in to the remote machine by simply piggybacking on to your ssh session (with a compromised ssh client).