it's prone to MITM attacks when setting up (in a way you are very unlikely to detect if done well)

it's prone to MITM attacks when being used (in a way you are very unlikely to detect if done well)

it's MITM attack vectors are not just usable with "on the wire" MITM but can be archived with social engineering making them IMHO pretty bad

it's also prone to certain kinds of brute-force attacks in certain situations and protecting against them without making your login trivially DDOSable is very very hard

from a security POV it's better then SMS but still a pretty bad design