Successfully locked out with TOTP

[Chris63]

First post and already a serious issue: How do I recover from TOTP no longer working?

I've configured my two OPNsense machines for TOTP authentication using a Yubikey. Format is <password><totp code> and everything worked great for two years.

Now I've had an unexpected power loss. Both machines booted up back ok, services are running normally. Can't log in anymore though, webinterface, SSH and serial console don't accept my credentials.

First idea was time offset between OPNsense and PC, but it turns out both NTP servers on both machines have the correct time and as a result so does the PC.

Second idea was maybe I remember the passwords wrong. I have paper backups in a secure location for just that. Nope, passwords are correct.

Now I'm out of ideas. Unless the issue magically fixes itself I see no choice but to pull the power and then the SD cards, then hopefully change something to force authentication from the local accounts only. How do I do that?

[franco]

Easiest fix is use install media boot and resert the root password of the installation with it as it will also change local authentication back to plain local database.


Cheers,
Franco

[Chris63]

Thank you for your quick reply, Franco. Unfortunately this doesn't work. Some error occurs but it's so quickly covered up by the installer I have no idea what it says. I'll try the configuration importer next, maybe there is something I can change in the backup xml instead.

[franco]

Sounds strange. What is the version installed on the disk?

The import might fail for the same reason if it's about UFS corruption , but then again the original system won't boot properly.


Cheers,
Franco

[Chris63]

I have 23.1 installed and of course use an USB stick with the 23.1 install image.

The system boots up properly, no error messages on the console, services work including DHCP, NTP, routing, firewall, OpenVPN and Wireguard.

I used the OPNsense importer to import the last known good configuration, sadly it showed the same behavior. Then I relaxed the TOTP grace period (in the XML config) to 5 minutes, 30 minutes and an hour, no change.

Thank you for your help, I'll continue debugging tomorrow. I've just spent 8 hours trying to fix this, my frustration level is increasing when I should be relaxing.

[allan]

I've configured my two OPNsense machines for TOTP authentication using a Yubikey. Format is <password><totp code> and everything worked great for two years.

Have you tried <totp code><password>? That is the format I use and the system might have switched to that. You mentioned NTP working so this is not a time issue.

I recommend setting up ssh public-key authentication to backup TOTP. I use TOTP for web portal logins and pubkey for ssh.

[Chris63]

<totp code><password> was the first thing I tried. I have no idea what the issue was.
I went to the last configuration backup, removed <authmode> from opnsense/system/webgui, removed the <otp_seed> from all <user> in opnsense/system and removed the whole <authserver> from opnsense/system.

Then used the install USB stick, made a config recovery USB stick and went with that. After getting the system back I again created the TOTP server, the seeds etc and now I'm back to where it all works the way it's supposed to.

I have no idea what went wrong, whether I did something wrong or there was a bug that only occurs under very unusual circumstances.

Thank you for trying to help, I very much appreciate your time.