OPNsense A10 Quad Core SSD - no console menu and no web GUI

[Marcel_75]

Hello,

I've bought an OPNsense A10 Quad Core SSD from Deciso some months ago and it worked very well all the time (liked it a lot). 

SKU: OPN20078B
S/N: 21230

I've always used the web GUI to configure this peace of hardware, never the console.

Some weeks ago I've made an update/upgrade to version 17.7 (don't remember the exact build and version, but I can remember I've changed the source where the update was downloaded from) and after some days I've lost the internet connection and at the end I also could not connect anymore to the web interface of the router.

As a fast troubleshoot I've used my old Apple Airport Extreme as router and it worked fine (with an THOMPSON Cable Modem).

So the OPNsense router was powered off for some weeks ...

Now I would like to get my OPNsense router running again, but there is no chance for me!

- also after a factory reset I don't get any IP on any of the 4 ethernet ports (only self-signed 169.254.44.197)
- if I use a hard-coded IP address like 192.168.1.2 for my Mac and as Gateway 192.168.1.1 I also can't connect (on any of the ethernet ports)

So I decided to connect via USB Console.

Which works, but there is no console menu, all I can see is this:

FreeBSD/amd64 (Amnesiac) (ttyu0)

login: root
Password: opnsense (did not changed this yet)

Warning: require_once(config.inc): failed to open stream: No such file or directory in /usr/local/sbin/opnsense-auth on line 32

Fatal error: require_once(): Failed opening required 'config.inc' (include_path='.:/usr/local/share/pear') in /usr/local/sbin/opnsense-auth on line 32

Last login: Tue Nov  7 10:05:37 on ttyu0
----------------------------------------------
|      Hello, this is OPNsense 17.7          |         @@@@@@@@@@@@@@@
|                                            |        @@@@         @@@@
| Website:      https://opnsense.org/        |         @@@\\\   ///@@@
| Handbook:     https://docs.opnsense.org/   |       ))))))))   ((((((((
| Forums:       https://forum.opnsense.org/  |         @@@///   \\\@@@
| Lists:        https://lists.opnsense.org/  |        @@@@         @@@@
| Code:         https://github.com/opnsense  |         @@@@@@@@@@@@@@@
----------------------------------------------

As far as I remember I've bought this router including some extra support from Deciso (don't remember the exact details) but I got no answer yet ...

Google'ing around I've found not a real solution, also a restart via Console with:

/usr/local/etc/rc.reboot

gave me the same "fatal error" after login.

Looks like I have to reinstall OPNsense completely?

Thanks for ideas and help,
all the best,
Marcel

[Marcel_75]

Ok, now I've created an USB-Stick to re-install OPNsense, but I don't know, how to boot from this stick?

What I've done on my MacBook Pro:

1) Downloaded the OPNsense Image (AMD64 serial) from Hiho.ch

OPNsense-17.7.5-OpenSSL-serial-amd64.img.bz2
7a85ae36b52d6f85239b7a936cefa5c53dddfa272b968e24bc6b61c77f4dfbce

2) Formated an 32 GB USB Stick with Disk Utility as FAT32 MBR stick named T1

3) Proofed the checksum:

openssl sha256 /Users/localadmin/Downloads/OPNsense-17.7.5-OpenSSL-serial-amd64.img.bz2
SHA256(/Users/localadmin/Downloads/OPNsense-17.7.5-OpenSSL-serial-amd64.img.bz2)= 7a85ae36b52d6f85239b7a936cefa5c53dddfa272b968e24bc6b61c77f4dfbce

4) Unpacked the OPNsense-17.7.5-OpenSSL-serial-amd64.img.bz2 to get the OPNsense-17.7.5-OpenSSL-serial-amd64.img

5) Used dd to copy the IMG to my USB-Stick

diskutil list

/dev/disk2 (external, physical):
   #:                       TYPE NAME                    SIZE       IDENTIFIER
   0:     FDisk_partition_scheme                        *31.0 GB    disk2
   1:                 DOS_FAT_32 T1                      31.0 GB    disk2s1

sudo diskutil unmountDisk /dev/disk2

Unmount of all volumes on disk2 was successful

sudo dd if=/Users/localadmin/Downloads/OPNsense-17.7.5-OpenSSL-serial-amd64.img of=/dev/disk2 bs=1m

Interesting: You don't see anything, the unpacked image is nearly 1 GB in size and the dd command will take around 5 minutes to copy everything to the USB stick. But you can use the Activity Monitor.app of macOS and search for a process in the cpu section called dd (it will disappear if dd has finished the job).

Then you will see something like the following on your Terminal.app window:

885+1 records in
885+1 records out
928120832 bytes transferred in 331.571088 secs (2799161 bytes/sec)

So far, so good.

But now, if I connect this stick to the device and reboot from console:

/usr/local/etc/rc.reboot

Sadly it's not booting into the live-installer, all I get is still the normal boot and again these information after login:

FreeBSD/amd64 (Amnesiac) (ttyu0)

Warning: require_once(config.inc): failed to open stream: No such file or directory in /usr/local/sbin/opnsense-auth on line 32

Fatal error: require_once(): Failed opening required 'config.inc' (include_path='.:/usr/local/share/pear') in /usr/local/sbin/opnsense-auth on line 32
Last login: Tue Nov  7 18:40:25 on ttyu0

So how can I restart the device and let it boot from the stick to install OPNsense from scratch? Can I force this in any way via the console?

PS: I'am really lost here and can't understand how and why this router was completely unusable after a normal upgrade process! 

[AdSchellevis]

Hi Marcel,

I miss the boot option selection in your description, you probably missed the bios part.
When starting up your A10 you should see some bios output, use F12 to select the usb stick as boot device.

For more info, see https://www.deciso.com/serial-console-access-boot-options/

If for some reason the usb stick won't boot, you should try to dd the image to /dev/rdisk2 (which is the raw device) as described in the docs https://docs.opnsense.org/manual/install.html

During the installer you can choose to import your config, so your device will use the same config it has now.

Best regards,

Ad

[Marcel_75]

Ok, thanks for the hint.

I couldn't see any boot options in the Terminal.app (Version 2.8 Build 400 of macOS 10.13.1 "High Sierra"), so I've switched to iTerm2 https://www.iterm2.com/index.html now which seems to be more useful especially when using the console mode.

Now I could do this:

cd /dev/
ls -la ./*usb*
screen /dev/tty.usbmodem1421 115200

to see the advice "Press ESC for boot menu."

which I did, but then it was not "F12" but key "2" to select the option to use the USB key:

Press ESC for boot menu.

Select boot device:

1. AHCI/0: TS128GSSD420K ATA-9 Hard-Disk (119 GiBytes)
2. USB MSC Drive Kingston DT microDuo 3.0 PMAP
3. iPXE (PCI 00:00.0)
4. Payload [memtest]

Searching bootorder for: HALT
drive 0x000f3250: PCHS=0/0/0 translation=lba LCHS=1024/255/63 s=60632064
drive 0x000f3280: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=250069680
Space available for UMB: c1000-ee800, f0000-f3250
Returned 253952 bytes of ZoneHigh
e820 map has 7 items:
  0: 0000000000000000 - 000000000009fc00 = 1 RAM
  1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
  2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
  3: 0000000000100000 - 00000000df175000 = 1 RAM
  4: 00000000df175000 - 00000000e0000000 = 2 RESERVED
  5: 00000000f8000000 - 00000000fc000000 = 2 RESERVED
  6: 0000000100000000 - 0000000120000000 = 1 RAM
enter handle_19:
  NULL
Booting from Hard Disk...
Booting from 0000:7c00

Sadly now it hangs at:

Booting from Hard Disk...
Booting from 0000:7c00

since around 5 minutes, should I wait a little longer or better do a dd again with:

sudo dd if=/Users/localadmin/Downloads/OPNsense-17.7.5-OpenSSL-serial-amd64.img of=/dev/disk2s2 bs=1m

instead of

sudo dd if=/Users/localadmin/Downloads/OPNsense-17.7.5-OpenSSL-serial-amd64.img of=/dev/disk2 bs=1m

Ok, sorry - forget this post ... I'am so stupid 

Was using the wrong USB key, now it's booting fine so the normal dd with disk2 instead of disk2s2 was building a working bootable USB key.

[Marcel_75]

But now I'am again a little confused - I could import my configuration from the SSD, but what should I do now as next step?

See what happened:

SeaBIOS (version rel-1.9.0.0-deciso-netboard-a10)
BUILD: gcc: (coreboot toolchain v1.33 November 25th, 2015) 5.2.0 binutils: (GNU Binutils) 2.25
Found mainboard Deciso Netboard A10

Press ESC for boot menu.

Select boot device:

1. AHCI/0: TS128GSSD420K ATA-9 Hard-Disk (119 GiBytes)
2. USB MSC Drive Kingston DT microDuo 3.0 PMAP
3. iPXE (PCI 00:00.0)
4. Payload [memtest]

Searching bootorder for: HALT
drive 0x000f3250: PCHS=0/0/0 translation=lba LCHS=1024/255/63 s=60555264
drive 0x000f3280: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=250069680
Space available for UMB: c1000-ee800, f0000-f3250
Returned 253952 bytes of ZoneHigh
e820 map has 7 items:
  0: 0000000000000000 - 000000000009fc00 = 1 RAM
  1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
  2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
  3: 0000000000100000 - 00000000df175000 = 1 RAM
  4: 00000000df175000 - 00000000e0000000 = 2 RESERVED
  5: 00000000f8000000 - 00000000fc000000 = 2 RESERVED
  6: 0000000100000000 - 0000000120000000 = 1 RAM
enter handle_19:
  NULL
Booting from Hard Disk...
Booting from 0000:7c00
/boot/config: -S115200 -D
Consoles: internal video/keyboard  serial port
BIOS drive C: is disk0
BIOS drive D: is disk1
BIOS 639kB/3654100kB available memory

FreeBSD/x86 bootstrap loader, Revision 1.1
(root@sensey64, Sat Aug 26 10:46:58 CEST 2017)
Loading /boot/defaults/loader.conf

## ... to much info, deleted this ...

Press any key to start the configuration importer: ......

<TS128GSSD420K O0918B>             at scbus0 target 0 lun 0 (pass0,ada0)
<Kingston DT microDuo 3.0 PMAP>    at scbus2 target 0 lun 0 (pass1,da0)

Select device to import from (e.g. ada0) or leave blank to exit: ada0

## So here I've selected ada0 to import the configuration from my internal TS128GSSD

Starting import for partition '/dev/ada0s1a'.

Running fsck...done.
Restoring config.xml...done.
Restoring dhcpleases.tgz...done.
Restoring rrd.tgz...done.
Restoring backup...done.
Restoring sshd...done.
Configuring crash dump device: /dev/null
.ELF ldconfig path: /lib /usr/lib /usr/lib/compat /usr/local/lib /usr/local/lib/ipsec /usr/local/lib/perl5/5.24/mach/CORE
32-bit compatibility ldconfig path:
done.
>>> Invoking early script 'update'
>>> Invoking early script 'configd'
Starting configd.
>>> Invoking early script 'php'
Configuring PHP: OK
>>> Invoking early script 'backup'
Launching the init system...done.
Initializing............done.
Starting device manager...done.
Configuring login behaviour...done.
Configuring loopback interface...done.
Configuring kernel modules...done.
Setting up extended sysctls...done.
Setting timezone...done.
Writing firmware setting...done.
Setting hostname: OPNsense17MacOn.localdomain
Generating /etc/hosts...done.
Starting syslog...done.
Creating OpenVPN instances...done.
Configuring loopback interface...done.
Creating wireless clone interfaces...done.
Configuring WAN interface...done.
Configuring LAN interface...done.
Syncing OpenVPN settings...done.
Generating /etc/resolv.conf...done.
Configuring firewall......done.
Setting up gateway monitors...done.
Configuring OpenSSH...done.
Starting web GUI...done.
Configuring CRON...done.
Setting up routes...done.
Starting Unbound DNS...done.
Starting DHCP service...done.
Generating /etc/hosts...done.
Configuring firewall......done.
Configuring dynamic DNS clients...done.
Starting NTP service...deferred.
Generating RRD graphs...done.
Starting syslog...done.
>>> Invoking start script 'freebsd'
Starting suricata.
8/11/2017 -- 12:17:18 - <Info> - Including configuration file installed_rules.yaml.
>>> Invoking start script 'carp'
>>> Invoking start script 'cron'
Starting Cron: OK
>>> Invoking start script 'beep'
Root file system: /dev/ufs/OPNsense_Install

*** OPNsense17MacOn.localdomain: OPNsense 17.7.5 (amd64/OpenSSL) ***

 LAN (em0)       -> v4: 192.168.1.1/24
 WAN (em1)       ->
 WIFIPORT (em2)  ->

Welcome!  Both `root' and `installer' users are availabe for system
setup or invoking the installer, respectively.  The predefined root
password works for both accounts.  Remote login via SSH is possible.

FreeBSD/amd64 (OPNsense17MacOn.localdomain) (ttyu0)

login: root
Password:
----------------------------------------------
|      Hello, this is OPNsense 17.7          |         @@@@@@@@@@@@@@@
|                                            |        @@@@         @@@@
| Website:      https://opnsense.org/        |         @@@\\\   ///@@@
| Handbook:     https://docs.opnsense.org/   |       ))))))))   ((((((((
| Forums:       https://forum.opnsense.org/  |         @@@///   \\\@@@
| Lists:        https://lists.opnsense.org/  |        @@@@         @@@@
| Code:         https://github.com/opnsense  |         @@@@@@@@@@@@@@@
----------------------------------------------

  0) Logout                              7) Ping host
  1) Assign interfaces                    Shell
  2) Set interface IP address            9) pfTop
  3) Reset the root password            10) Firewall log
  4) Reset to factory defaults          11) Reload all services
  5) Power off system                   12) Upgrade from console
  6) Reboot system                      13) Restore a backup

Looks like it didn't realised yet what the WAN interface is?

Have connected my Thompson Cable Modem to Port 1 and my normal network to port 2 and my Mac to Port 3.

PS: Oh my god, I'am really BLIND - of course I have to connect WAN to Port 2 and the "normal home network" to Port 1!

Like it sayd:

 LAN (em0)       -> v4: 192.168.1.1/24
 WAN (em1)       ->
 WIFIPORT (em2)  ->

[Marcel_75]

ok, two questions now to get a little clearer:

1) If I login to the web GUI now, it says

„You are currently running in LiveCD mode. A reboot will reset the configuration. SSH remote login is enabled for the users „root“ and „installer“ using the same password.“

So the router started now a LIVE system from the USB key and my personal configuration was imported from the internal SSD to this LIVE system, right?

How to move now this working LIVE system including my good old personal configuration to the internal SSD?

Do I have to SSH now into the router and copy the complete data from the USB key to the internal SSD? If so, how exactly?

2) The WAN interface (connected to the cable modem) still getting no internet connection.

I‘ve switched off now the cable modem and will wait around 30 minutes before switching it on again.

As far as I remember the MAC address reservation is often a problem with cable modems (here it‘s an THOMPSON THG570K which works fine if connected to my old Airport Extreme).

But now the problem could be it‘s looking for the MAC address of this Airport Extreme station and it „don‘t accept“ the new MAC address from port 2 of my Deciso router.

So I hope after 30 minutes it will accept the new MAC address and working?

Or is there a better way to fix this behaviour?

Thanks a lot for all the help!

[NilsS]

Dont login as root. Login as installer (same password)

[Ciprian]

When you boot from USB/ Optical the default running mode is "liveCD mode". This is maintained as long as you login and configure/ change settings with root user.

If you want to put everyting on the disk, first thing first login with installer user (same password) - this triggers the install to local storage process/ wizzard - and then reboot without the removable media (USB/ Optical), or without boot order prioritizing removable media.

Also, LiveCD mode is useful when a new version of OPNsense is released, and you want to test if everything is OK/ works well with your config before updating to the new version: copy the new ver. on a removable and boot from that removable in LiveCD Mode, restore from a config backup file the config you want to test in the running LiveCD Mode, see if everything works well, then reboot from internal storage and make the update on the internal storage. Case there are any glitches you cant live with, there is always the easy option to reboot from internal storage to the existing version, since running from removable is LiveCD Mode only, and no changes are saved to the internal storage.

Hope it helps!

[franco]

Sali Marcel,

Warning: require_once(config.inc): failed to open stream: No such file or directory in /usr/local/sbin/opnsense-auth on line 32

Fatal error: require_once(): Failed opening required 'config.inc' (include_path='.:/usr/local/share/pear') in /usr/local/sbin/opnsense-auth on line 32

Looks like file system damage from an unclean shutdown. But we’ve also seen this with larger firmware updates where the next reboot forgets the files being updated/installled. Or damaged storage media. Reinstall was best course of action, can’t fix the file system from our side. All we do is unpack files and expect them to be there after reboot.


Cheers,
Franco

[Marcel_75]

First of all: Thank you very much for your help and support!

It's working now again. 

And that's a good advice - test a major upgrade (like the 17.7 was) with a Live-System. It's a very nice option to import the settings, really great idea (didn't know about this feature yet).

In the future I will do this for sure, because somehow the upgrade to 17.7 "killed" the file system on my router.

At the end I've re-installed now from scratch (booted from the USB key live system, logged in as installer and run the installation setup wizard to install fresh on the internal SSD).

There was the option to import my configuration (and I did this in the beginning) but at the end it was faster for me to setup everything new again (now, as I'am a little more used to the OPNsense web interface it was much faster than some months ago). 

The Unbound DNS is working fine again (to use an ad-blacklist.conf to avoid advertisements) and other features like the OpenVPN server are setup also fast and easy.

One last question to the developers of OPNsense: I can understand you can't control the state of the file system and something bad could happen all the time if there is an "power off" in the wrong moment. But wouldn't it be great to have the option to use a file system which is SAFE against situations like this?

Like ZFS for example? APFS on my Mac is "closed source" so this is no option, but in the future maybe ZFS would be the best choice if we want something like this?

I'am not a network expert (nor an file system expert), but you get the idea?

[Ciprian]

One last question to the developers of OPNsense: I can understand you can't control the state of the file system and something bad could happen all the time if there is an "power off" in the wrong moment. But wouldn't it be great to have the option to use a file system which is SAFE against situations like this?

Like ZFS for example? APFS on my Mac is "closed source" so this is no option, but in the future maybe ZFS would be the best choice if we want something like this?

I'am not a network expert (nor an file system expert), but you get the idea?

If you don't have ECC RAM in your hardware, ZFS is the worst FS you could use: ZFS use CPU and RAM to check in real-time the errors occurred on the storage/ filesystem, but if the info on FS is good, and a faulty RAM causes mismatches between the stored data (storage/ FS) and checked data (RAM), ZFS interprets that your faulty data in RAM is the correct data, and your good data on storage is the bad data!!! (!) Guess what: your good data on storage is replaced with bad data in RAM, since this is how ZFS is designed!

Never use ZFS without having ECC RAM (and supporting motherboard/ BIOS/ system for ECC RAM)!!! (!)