Maybe a weird ControlMaster/ControlPath config? I have had issues with the ControlPath result being too long with certain hostname/port combinations in the past -- which resulted in ssh to ip working but ssh to hostname not working. As a result, I haven since started using %C instead of %l%h%p%r in my ControlPath config.