external (encrypted) usb?

[f1-outsourcing]

"If your machine is compromised by malware, the attacker could steal a copy of the secret keys stored by this authenticator"

But when you use this to authenticate, you can choose some usb pen drive or key file stored on a usb drive to authenticate? So if you our pen drive is not connected no harm/copying can be done.