How would this even work if it was stored on the server? Every singkr sound triggers a roundtrip to the server from your phone to check the voice? I can't imagine this working any other way than being stored locally.
I don't really see how this follows, particularly since I don't believe you can sync the voiceprint across phones. If you're saying that it's _possible_ to send voice prints to the server unnecessarily, then sure: but this has nothing to do with always-listening/voice-print storage; it could be the case for any sort of voice command, which means that what we're talking about here isn't any different from any voice command system in that respect.