Power loss recovery on Protectli V1410 - USB COM port & system time

Started by OPNenthu, July 30, 2025, 10:28:08 AM

Previous topic - Next topic
I don't know if this issue affects more units than just mine, or if this is maybe on my end (defective PC motherboard?), but I wanted to document the recovery steps.  I did forward my observations to Protectli as well.

Specs:
Vault V1410 (Intel N5105, 8GB) with Protectli coreboot version 0.9.3.  The coreboot detail is important because there's no option to set the RTC that I'm aware of, so will need to be done through the OS.

I don't know if this affects units with the stock AMI UEFI.

The issue:
Following an extended power loss event where the Vault remains disconnected from mains power or battery backup for some time (>1hr in my case), the USB COM port becomes inoperable and fails to list in Windows Device Manager or in the Linux /dev filesystem.  The firewall remains accessible only via GUI and SSH. The issue persists across power cycles.

(EDIT: I haven't tried the vga console as mine runs headless.)

Evidence:
You can observe that the device is not being recognized on your PC that is connected with the USB COM cable.  For example, in Linux 'dmesg' output there will not be any reference to the expected device (/dev/ttyUSB0) that is typically there.  Also there will be no such character device on the /dev on the filesystem:

$ sudo dmesg | grep tty
[    0.152646] printk: legacy console [tty0] enabled
[    0.791432] serial8250: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A

$ ls -l /dev/ttyUSB*
ls: cannot access '/dev/ttyUSB*': No such file or directory

On Windows there will be an error indicator in Device Manager.  The COM port will have become an unrecognized device that fails to initialize.

The fix:
1) Open the device and perform a CMOS reset by shorting two pins.  https://kb.protectli.com/kb/cmos-reset/

This will restore the serial port, but will also wipe out the system time.

2) DNS resolution may be blocked due to the time error and the NTP service in OPNsense will be unable to sync.  Manually set the date to the current wall clock time to within 1-2 minutes.  On the OPNsense console (as root):

$ date yymmddhhMM

For example, enter "2507301300" for "Jul 30 2025 at 1pm"

This will get NTP and other services unstuck, but may take a few minutes to sync.

Validation:

$ sudo dmesg | grep tty
[    0.152646] printk: legacy console [tty0] enabled
[    0.791432] serial8250: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[ 8856.625541] usb 1-1: FTDI USB Serial Device converter now attached to ttyUSB0     <--- this reappears

$ ls -l /dev/ttyUSB*
crw-rw---- 1 root dialout 188, 0 Jul 30 00:37 /dev/ttyUSB0

If on Windows, you should again see the COM0 port in Device Manager.


-----

I did see this issue multiple times already as we have been having more frequent power outages in our region, and also I've seen it on two separate V1410s.  I had gotten a replacement for my original one due to a different issue, but that one exhibited this problem as well. 

On the most recent occurrence I did happen to have a serial terminal session open when the power went out, so it could be a factor.  I don't recall if I was able to close the session cleanly and I don't remember if that was always the case the other times it happened.  If you own this device and depend on the USB COM then maybe consider keeping around a small screwdriver.
"The power of the People is greater than the people in power." - Wael Ghonim

Site 1 | N5105 | 8GB | 256GB | 4x 2.5GbE (I226-V)
Site 2 |  J4125 | 8GB | 256GB | 4x 1GbE (I210)

Quick update from Protectli support:

- Possible dead CMOS battery causing loss of clock time and getting the USB COM port into a bad state when disconnected from power (some BIOS state issue, perhaps).  They offered a replacement battery.

- Unclear why a CMOS reset brings the COM port back up without replacing the battery.  More testing needed.  Protectli trying to reproduce, but have not so far seen this issue.

- I asked why it happened on two separate units which were both new (should have had good batteries).  It's unclear, but there are a couple thoughts on this:

  1) It's unlikely but not impossible that I got two weak batteries in a row.

  2) There could be an unidentified vampiric drain on the battery.

  As a side note, I mentioned to them that I keep the USB COM cable connected to the device 24/7, as it sits on my desk next to my PC.  Not sure if that's contributing.  I'm wondering if that keeps the FTDI chip active and if this has some kind of connection to the CMOS battery (not verified).


"The power of the People is greater than the people in power." - Wael Ghonim

Site 1 | N5105 | 8GB | 256GB | 4x 2.5GbE (I226-V)
Site 2 |  J4125 | 8GB | 256GB | 4x 1GbE (I210)