Debug FIDO SSH Operations Without Running ssh-sk-helper Directly

You will troubleshoot an OpenSSH FIDO security-key operation by turning on debugging at the command that owns the operation, then separating a missing key, a PIN or touch failure, and a helper-process problem. Allow about 15 minutes for a first check if the key is available. The examples are designed to inspect an existing setup and do not create, revoke or overwrite credentials.

1. Check the installed helper and package version

ssh-sk-helper is an internal OpenSSH program. The installed manual says it is used by ssh, ssh-agent and ssh-keygen to access FIDO authenticator keys, and says it is not intended to be invoked directly. On this machine it is supplied by Ubuntu's openssh-client package, version 1:9.6p1-3ubuntu13.19.

$ dpkg-query -W -f='${Package} ${Version}\n' openssh-client
openssh-client 1:9.6p1-3ubuntu13.19
$ command -v ssh
/usr/bin/ssh
$ dpkg -L openssh-client | grep '/ssh-sk-helper$'
/usr/lib/openssh/ssh-sk-helper

Checkpoint: record the package version and helper path before comparing a problem with advice written for another OpenSSH release. The helper is normally not on the shell's PATH, which is another clue that it is a child process rather than a public command-line tool.

2. Do not test the helper as a standalone command

The only documented option is -v. It enables progress diagnostics, and up to three -v options increase verbosity. That does not make the helper a complete FIDO test command. It expects a private message protocol from its OpenSSH parent, so a direct invocation is not a useful test of the authenticator.

$ /usr/lib/openssh/ssh-sk-helper -v
ssh_msg_recv failed
$ printf 'exit status: %s\n' "$?"
exit status: 255

This failure is expected when the helper has no parent process supplying its protocol. Do not add sudo, put the helper in a startup script, or treat exit status 255 from this artificial test as proof that the FIDO key is broken. There is nothing to undo because the command does not change persistent state.

3. Reproduce the real operation with debug output

Run the client or key-management operation that actually fails, and add debug flags there. OpenSSH passes its debug setting to the helper automatically. For an SSH connection, use -vvv; for a key operation, use the corresponding verbose options accepted by your installed ssh-keygen. Start with the exact host, user and key selection you normally use.

$ ssh -vvv -i ~/.ssh/id_ecdsa_sk USER@HOSTNAME

The connection output varies with the server, key type and local configuration. The useful checkpoint is the sequence around the FIDO key: confirm that the client selected the intended public key, look for helper diagnostics, and then check whether the server accepted the resulting signature. Do not copy the whole log into a ticket without reviewing it first. Debug output can contain usernames, host names, file paths and server details.

For a local key inspection that does not contact a server, use a public key file:

$ ssh-keygen -lf ~/.ssh/id_ecdsa_sk.pub
256 SHA256:REPLACE_WITH_YOUR_FINGERPRINT USER@HOST (ECDSA-SK)

Replace the path with the public half of your own key. This confirms which fingerprint and key type you are examining, but it does not prove that the authenticator can sign or that the remote account authorises the key.

4. Classify the failure before changing anything

Use the debug output to classify the first failure, not the last line. If the intended public key is never offered, check the -i path, the effective SSH configuration and file permissions. If the client offers the key but asks for a PIN or touch and then stops, interact with the authenticator and check its PIN or user-presence requirement. A cancelled prompt is different from a missing helper.

If the log shows that the helper starts but cannot communicate, preserve the complete local debug context and check the installed package rather than running the helper by hand. If the helper completes but the server rejects the key, inspect the server's authorised key, account and authentication policy. A successful helper exchange only means that the local side produced a signature; it does not grant access to the remote account.

Keep these boundaries clear:

• The authenticator provides a protected signing operation. The private key material is not a file that you should copy into a shell command.
• The public key file identifies a credential, but it cannot replace the hardware key when a signature is requested.
• ssh-sk-helper debug messages describe helper progress. They do not by themselves prove server acceptance.

5. Increase verbosity only when the first log is insufficient

Use the client debug levels deliberately. One -v is often enough to show configuration and authentication decisions. Two or three levels can expose the helper hand-off and protocol progress, but they also make logs larger and more revealing. The helper itself supports a maximum of three -v options, and the parent passes its own debug level automatically.

$ ssh -vv -i ~/.ssh/id_ecdsa_sk USER@HOSTNAME 2> ssh-fido-debug.log
$ grep -Ei 'sk|fido|security key|helper|publickey|auth' ssh-fido-debug.log
$ rm -- ssh-fido-debug.log

The redirection writes a local diagnostic file and the final command removes it. Review the file before sharing it. If you need to retain it, protect it with normal user-only permissions and delete it after the incident. Do not include a PIN, recovery code or private key in a log, command line or support message.

6. Recover without changing credentials

Stop after collecting evidence if the cause is unclear. Unplug and reconnect the authenticator, retry the same parent command, and compare the two logs. If the key is held by an agent, inspect the agent's normal key listing and restart only that user-level agent when you understand the effect on existing sessions. Do not delete key files or reset the authenticator as a first troubleshooting step: those actions can be irreversible and may destroy access to resident credentials.

If a package repair is required, use your normal system administration process and a maintenance window. A package reinstall changes system files and may require elevated privileges; it is not justified by the expected ssh_msg_recv failed result from a standalone helper invocation. After any repair, repeat the real ssh -vvv operation and compare the package version and helper path again.