[Work In Progress] OPNsense Ported into ARM Devices

[mipsou]

Hi yrzr,

I found a good documentation on https://github.com/sreinhardt/armada-8040-images

i think with this item you can generate img

[yrzr]

Hi mipsou,

Sorry I cannot. First of all, I don't have a macchiatobin, so whatever I generated, I could not test it. Secondly, the tools I am using are different from the document you provide. Finally, why not buy some Intel-based machines with the price of macchiatobin?

If I were you, I would ask the author of the document for further supports.

Hi yrzr,

I found a good documentation on https://github.com/sreinhardt/armada-8040-images

i think with this item you can generate img

[mipsou]

I already have this material which should run in aarch64.
I have LAN 10GB and my fiber ISP too.

[michaelsage]

Hi,
Firstly thank you to the OPNSense developers and the ARM port developer for all the work so far.

I have successfully installed 21.1.5 from the image provided on to a Pi 3+.

I was wondering if anyone had tried to add a 4G/LTE connection? I guess there would need to be hardware support in the ARM/BSD build as well as OPNSense? I have been looking at the Waveshare hat (https://www.waveshare.com/wiki/SIM7600G-H_4G_HAT). I don't think this will work "out of the box" but I just wanted to see if anyone had tried? If not has anyone tried a 4G USB dongle? Was thinking the OPNSense Pi would make a great "road warrior" router / firewall.
Thanks!

Michael

[wilmardo]

Hi all,

Thanks for the progress on getting this to work on ARM devices @nekoprog and @yrzr. Hopefully the PR will get merged soon :)

For now I resumed my effort to get a working NanoPi R2s image going, but I am stuck at U-Boot problems. I hope someone here is able to help me forward again :)

I have this as my config now:
https://github.com/wilmardo/tools/blob/nanopi-r2s/device/NPIR2S.conf

It contains all kinds of stabs in the dark like the .dtb from Debian and the loadaddr 32K moved over from the address in u-boot.

I currently have a working U-boot but it can't seem to find the kernel (or an UFS partition for that matter). I can mount the UFS partition fine on my PC so the SD card seems to be fine.

This is my current bootlog:

Code Select Expand


DDR version 1.16 20190528
ID:0x805 N
In
DDR4
333MHz
Bus Width=32 Col=10 Bank=4 Bank Group=2 Row=15 CS=1 Die Bus-Width=16 Size=1024MB
ddrconfig:14
OUT
Boot1 Release Time: May 13 2019 17:34:36, version: 2.50
ChipType = 0x11, 248
mmc2:cmd1,20
emmc reinit
mmc2:cmd1,20
emmc reinit
mmc2:cmd1,20
SdmmcInit=2 1
mmc0:cmd5,20
SdmmcInit=0 0
BootCapSize=0
UserCapSize=3781MB
FwPartOffset=2000 , 0
StorageInit ok = 34120
Raw SecureMode = 0
SecureInit read PBA: 0x4
SecureInit read PBA: 0x404
SecureInit read PBA: 0x804
SecureInit read PBA: 0xc04
SecureInit read PBA: 0x1004
SecureInit ret = 0, SecureMode = 0
atags_set_bootdev: ret:(0)
GPT 0x337a9f0 signature is wrong
recovery gpt...
GPT part:  0, name:                 , start:0x800, size:0x762000
recovery gpt success!
no find partition:uboot.
LoadTrust Addr:0x4000
No find bl30.bin
No find bl32.bin
Load uboot, ReadLba = 2000
hdr 000000000337a3b0 + 0x0:0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,0x00,

Load OK, addr=0x200000, size=0xab950
RunBL31 0x40000
INFO:    Preloader serial: 2
NOTICE:  BL31: v1.3(debug):403e0b816
NOTICE:  BL31: Built : 14:13:08, Aug 11 2020
NOTICE:  BL31:Rockchip release version: v1.3
INFO:    ARM GICv2 driver initialized
INFO:    Using opteed sec cpu_context!
INFO:    boot cpu mask: 1
INFO:    plat_rockchip_pmu_init: pd status 0xe
INFO:    BL31: Initializing runtime services
WARNING: No OPTEE provided by BL2 boot loader, Booting device without OPTEE initialization. SMC`s destined for OPTEE will return SMC_UNK
ERROR:   Error initializing runtime service opteed_fast
INFO:    BL31: Preparing for EL3 exit to normal world
INFO:    Entry point address = 0x200000
INFO:    SPSR = 0x3c9


U-Boot 2020.10-armbian (May 06 2021 - 18:04:51 +0000)

Model: FriendlyElec NanoPi R2S
DRAM:  1022 MiB
PMIC:  RK8050 (on=0x40, off=0x01)
MMC:   mmc@ff500000: 1
Loading Environment from MMC... MMC Device 0 not found
*** Warning - No MMC card found, using default environment

In:    serial@ff130000
Out:   serial@ff130000
Err:   serial@ff130000
Model: FriendlyElec NanoPi R2S
Net:   eth0: ethernet@ff540000
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc1 is current device
Scanning mmc 1:1...
Found EFI removable media binary efi/boot/bootaa64.efi
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Scanning disk mmc@ff500000.blk...
** Unrecognized filesystem type **
Found 3 disks
No EFI system partition
BootOrder not defined
EFI boot manager: Cannot load any image
393216 bytes read in 68 ms (5.5 MiB/s)
libfdt fdt_check_header(): FDT_ERR_BADMAGIC
Booting /efi\boot\bootaa64.efi

>> FreeBSD EFI boot block
   Loader path: /boot/loader.efi

   Initializing modules: ZFS UFS
   Load Path: /efi\boot\bootaa64.efi
   Load Device: /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/SD(1)/SD(0)/HD(1,0x01,0,0x800,0x19000)
   Probing 3 block devices...... done
    ZFS found no pools
    UFS found no partitions
Failed to load '/boot/loader.efi'
panic: No bootable partitions found!
## Application failed, r = 1
EFI LOAD FAILED: continuing...
MMC Device 0 not found
no mmc device at slot 0
starting USB...
No working controllers found
USB is stopped. Please issue 'usb start' first.
starting USB...
No working controllers found
ethernet@ff540000 Waiting for PHY auto negotiation to complete......... TIMEOUT !


Could anyone point me into a new direction or something to try? I am really stuck at the moment, I tried everything I could think off.

Again, massive thanks for the porting of tools to ARM!

[spikerguy]

There is a working image gor freebsd and opnsense for NanoPi R2S as since the rtl drivers were recently merged in freebsd.


https://personalbsd.org/images/OPNsense-21.1-OpenSSL-aarch64-NanoPi-R2S-20210612.img.xz.

Please donate to sergey for all his effort at personalbsd.org
Feel free to join telegram chat for opnsense on arm @personalbsd
Thanks.

[aesth]

There is a working image gor freebsd and opnsense for NanoPi R2S as since the rtl drivers were recently merged in freebsd.


https://personalbsd.org/images/OPNsense-21.1-OpenSSL-aarch64-NanoPi-R2S-20210612.img.xz.

Please donate to sergey for all his effort at personalbsd.org
Feel free to join telegram chat for opnsense on arm @personalbsd
Thanks.

Nice! Thanks for the link.

[mentaluproar]

How is this doing performance wise? Compared to atom and celeron-powered boxes, how well can these ARM SBCs hold up? Has anyone tried activating anything heavy on them, like intrusion detection?

[gokulkgm]

There is a working image gor freebsd and opnsense for NanoPi R2S as since the rtl drivers were recently merged in freebsd.


https://personalbsd.org/images/OPNsense-21.1-OpenSSL-aarch64-NanoPi-R2S-20210612.img.xz.

Please donate to sergey for all his effort at personalbsd.org
Feel free to join telegram chat for opnsense on arm @personalbsd
Thanks.

I would like to try the image on R4S, @spikerguy Can you link to the telegram chat? I couldn't find it.

[wilmardo]

I would like to try the image on R4S, @spikerguy Can you link to the telegram chat? I couldn't find it.

https://personalbsd.org/?p=313 < there you go.

I tried booting the R2S image but that didn't work.

[yrzr]

I have recently built a version 21.1 test image for rpi3 (may also work on rpi4).

Feel free to try it, and don't forget to modify config.txt before the first run ;).

------------

update on 2021-02-24

An updated image is now out, which works properly on rpi3b+ with the advice of @testo_cz.

Moreover, after the first boot, add `hw.uart.console=""` to /boot/loader.conf.local or the console will goes nowhere in the next boot.

Finally, you can use https://ftp.yrzr.tk/opnsense/ as the Repo to get almost all the plugins as if on amd64. Edit /usr/local/etc/pkg/repos/OPNsense.conf:

Code Select Expand

OPNsense: {
  fingerprints: "/usr/local/etc/pkg/fingerprints/OPNsense",
  url: "https://ftp.yrzr.tk/opnsense/${ABI}/21.1/latest",
  signature_type: "NONE",
  mirror_type: "NONE",
  priority: 11,
  enabled: yes
}

------------

update on 2021-03-25

Thanks to @nekoprog's code and @testo_cz's advice, the RPI3 images now work fine with the serial console.

I have also made a rough introduction to the images on https://www.yrzr.tk/opnsense-images-for-aarch64.

Here is something interesting. I saw the news that OPNsense 22.1 will base on FreeBSD 13.x instead of HardenedBSD. So the idea jumps into my mind that if the kernel is 13.x then RPI4 works.

And, it works! Although, with flaws as expected.

The kernel version I use is FreeBSD 13.0-RELEASE

Code Select Expand

FreeBSD 13.0-RELEASE #0 releng/13.0-n244733-ea31abc261f: Fri Apr  9 06:06:55 UTC 2021
    root@releng1.nyi.freebsd.org:/usr/obj/usr/src/arm64.aarch64/sys/GENERIC arm64
FreeBSD clang version 11.0.1 (git@github.com:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c3fe)

Moreover, the modules `if_bridge` and `if_enc` stops the kernel from recognizing the sd card. It seems both unrelated, I have no idea why.

Code Select Expand

mmcsd0: Error indicated: 1 Timeout
mmcsd0: Error indicated: 1 Timeout
mmcsd0: Error indicated: 1 Timeout
mmcsd0: Error indicated: 1 Timeout
mmcsd0: Error indicated: 1 Timeout
mountroot: waiting for device /dev/ufs/OPNsense...
Mounting from ufs:/dev/ufs/OPNsense failed with error 19.

Loader variables:
  vfs.root.mountfrom=ufs:/dev/ufs/OPNsense
  vfs.root.mountfrom.options=rw

Manual root filesystem specification:
  <fstype>:<device> [options]
      Mount <device> using filesystem <fstype>
      and with the specified (optional) option list.

    eg. ufs:/dev/da0s1a
        zfs:zroot/ROOT/default
        cd9660:/dev/cd0 ro
          (which is equivalent to: mount -t cd9660 -o ro /dev/cd0 /)

  ?               List valid disk boot devices
  .               Yield 1 second (for background tasks)
  <empty line>    Abort manual input

mountroot>

Luckily, the system boots by disabling the two modules. But I think at least the `if_bridge` module is rather important for OPNsense. Would try nightly build kernels later. But still, it boots.

Finally, you can try the dirty image I made on a RPI4 yourself.

[jclendineng]

Would something like this work on the netgate SG-3100 (ARM not aarch64)?  Having issues with pfsense on it...

[dsbibby]

I have recently built a version 21.1 test image for rpi3 (may also work on rpi4).

Feel free to try it, and don't forget to modify config.txt before the first run ;).

------------

update on 2021-02-24

An updated image is now out, which works properly on rpi3b+ with the advice of @testo_cz.

Moreover, after the first boot, add `hw.uart.console=""` to /boot/loader.conf.local or the console will goes nowhere in the next boot.

Finally, you can use https://ftp.yrzr.tk/opnsense/ as the Repo to get almost all the plugins as if on amd64. Edit /usr/local/etc/pkg/repos/OPNsense.conf:

Code Select Expand

OPNsense: {
  fingerprints: "/usr/local/etc/pkg/fingerprints/OPNsense",
  url: "https://ftp.yrzr.tk/opnsense/${ABI}/21.1/latest",
  signature_type: "NONE",
  mirror_type: "NONE",
  priority: 11,
  enabled: yes
}

------------

update on 2021-03-25

Thanks to @nekoprog's code and @testo_cz's advice, the RPI3 images now work fine with the serial console.

I have also made a rough introduction to the images on https://www.yrzr.tk/opnsense-images-for-aarch64.

Thanks @yrzr for this amazing work and for hosting the arm64 repo! I have opnsense 21.7 up and running happily on my rpi4 under ESXi.

Only thing I was wondering is if there is a reason the "os-vmware" plugin isn't available in your repo? It would be great to get vmware tools going too. I can have a go at compiling and installing vmwtools by hand, but it would be even better if this was possible through opnsense's ui.

Many thanks!

[yrzr]

The plugin depends on open-vm-tools-nox11, currently not supported on aarch64.

However, there are patches out there that you can install on aarch64. See https://vincerants.com/open-vm-tools-on-freebsd-under-vmware-esxi-arm-fling/.

I have recently built a version 21.1 test image for rpi3 (may also work on rpi4).

Feel free to try it, and don't forget to modify config.txt before the first run ;).

------------

update on 2021-02-24

An updated image is now out, which works properly on rpi3b+ with the advice of @testo_cz.

Moreover, after the first boot, add `hw.uart.console=""` to /boot/loader.conf.local or the console will goes nowhere in the next boot.

Finally, you can use https://ftp.yrzr.tk/opnsense/ as the Repo to get almost all the plugins as if on amd64. Edit /usr/local/etc/pkg/repos/OPNsense.conf:

Code Select Expand

OPNsense: {
  fingerprints: "/usr/local/etc/pkg/fingerprints/OPNsense",
  url: "https://ftp.yrzr.tk/opnsense/${ABI}/21.1/latest",
  signature_type: "NONE",
  mirror_type: "NONE",
  priority: 11,
  enabled: yes
}

------------

update on 2021-03-25

Thanks to @nekoprog's code and @testo_cz's advice, the RPI3 images now work fine with the serial console.

I have also made a rough introduction to the images on https://www.yrzr.tk/opnsense-images-for-aarch64.

Thanks @yrzr for this amazing work and for hosting the arm64 repo! I have opnsense 21.7 up and running happily on my rpi4 under ESXi.

Only thing I was wondering is if there is a reason the "os-vmware" plugin isn't available in your repo? It would be great to get vmware tools going too. I can have a go at compiling and installing vmwtools by hand, but it would be even better if this was possible through opnsense's ui.

Many thanks!

[dsbibby]

Amazing - thanks for the link. That's working perfectly.

In case anyone else sees this, I had to temporarily enable the FreeBSD pkg repo in /usr/local/etc/pkg/FreeBSD.conf in order to install some dependancies.

Also, looks like the PR the author of the post you linked to submitted in to the open-vm-tools repo was merged and is released in version 11.3.0 of open-vm-tools. Does this help unlock the official opnsense plugin at all?

Many thanks!

The plugin depends on open-vm-tools-nox11, currently not supported on aarch64.

However, there are patches out there that you can install on aarch64. See https://vincerants.com/open-vm-tools-on-freebsd-under-vmware-esxi-arm-fling/.

I have recently built a version 21.1 test image for rpi3 (may also work on rpi4).

Feel free to try it, and don't forget to modify config.txt before the first run ;).

------------

update on 2021-02-24

An updated image is now out, which works properly on rpi3b+ with the advice of @testo_cz.

Moreover, after the first boot, add `hw.uart.console=""` to /boot/loader.conf.local or the console will goes nowhere in the next boot.

Finally, you can use https://ftp.yrzr.tk/opnsense/ as the Repo to get almost all the plugins as if on amd64. Edit /usr/local/etc/pkg/repos/OPNsense.conf:

Code Select Expand

OPNsense: {
  fingerprints: "/usr/local/etc/pkg/fingerprints/OPNsense",
  url: "https://ftp.yrzr.tk/opnsense/${ABI}/21.1/latest",
  signature_type: "NONE",
  mirror_type: "NONE",
  priority: 11,
  enabled: yes
}

------------

update on 2021-03-25

Thanks to @nekoprog's code and @testo_cz's advice, the RPI3 images now work fine with the serial console.

I have also made a rough introduction to the images on https://www.yrzr.tk/opnsense-images-for-aarch64.

Thanks @yrzr for this amazing work and for hosting the arm64 repo! I have opnsense 21.7 up and running happily on my rpi4 under ESXi.

Only thing I was wondering is if there is a reason the "os-vmware" plugin isn't available in your repo? It would be great to get vmware tools going too. I can have a go at compiling and installing vmwtools by hand, but it would be even better if this was possible through opnsense's ui.

Many thanks!