opnsense 22.1 for VMware ESXi ARM Fling on raspberry pi4B 8gb

[Joonas42]

Sorry for the late reply!
Was busy with other stuff lately, had little to no time and to totally forgot to reply.

Hello @efetropy,

Can you share the patches required to build opnsense 22.1 for a Raspberry Pi 4? I'd like to help with this.

I'm just getting started and have finally understood how to build it but missing the specific configuration for RPI4.

Dear all,

Currently, I have a CM4 (with 4GB RAM and 8GB eMMC and no Wifi) and the DFRobot IoT Router Carrier Board (https://www.dfrobot.com/product-2242.html).

I am interested to compile an OPNsense build for this hardware but I am a little bit lost about it's potential compatibility.

Can someone confirm me that's this will be good or not?

Many thanks in advance!

There is already an image built by @yrzr for the RPi which can be found here:
https://ftp.yrzr.tk/opnsense/FreeBSD%3A13%3Aaarch64/22.1/images/

A generic aarch64 vmdk image is also available (for e.g. for proxmox, esxi fling or whatever you're using).

The RPi 4B should work without problems (if not, just get some different boot files).

CM4 is a different case. It does work with the official I/O board albeit PCIe issues (see bugzilla).
Your carrier board states ETH2: PCI Express 1000BASE-T NIC (based on RTL8111). RTL8111 drivers are in fact available, but
it will be a hit or miss and you might not be able to the use the 2nd ethernet port due to the mentioned PCIe issues.
Furthermore, you have to figure out how to deal with the dtb file. Worst case scenario, you might have to make it compatible with freebsd.

I can confirm that the 22.1 image made by YRZR works with DFrobot carrier board with CM4 with 8GB Ram. Unfortunately the performance is only around 300Mb/s on 1Gb connection. Maybe some tuning needed to get better results. The cpu was only at 40% during downloads. Has anyone got better performance?

[darkain]

Did some research and found out that with open-vm-tools 11.3.0 they added support for arm64 (vmci is not working). You can read more about it here https://github.com/vmware/open-vm-tools/pull/474 and https://vincerants.com/open-vm-tools-on-freebsd-under-vmware-esxi-arm-fling/

Hey, thanks for referencing my work! Hopefully it has helped you all out.

I've been running earlier builds of OPNsense under ESXi ARM Fling for probably at least a year now. I guess it is about time I update to these new builds you are all putting together!

I'll have to switch my 16-core ARM server back over from FreeBSD bare metal to running ESXi ARM Fling instead, and see what kind of bandwidth that thing can push with dual-10gbe NICs.

[BuckRogers25]

Hi all,
Interesting post, I'll give this a go when I have a little time next month.  How's throughput on a pi4 8gb, above 500mb?

Thanks for your efforts.

[daemonix]

Anyone had a similar boot problem?
Thanks!

Code: [Select]

Consoles: EFI console 
    Reading loader env vars from /efi/freebsd/loader.env
Setting currdev to disk0p1:
FreeBSD/arm64 EFI loader, Revision 1.1

   Command line arguments: loader.efi
   Image base: 0xb82ff000
   EFI version: 2.70
   EFI Firmware: EDK II (rev 1.00)
   Console: efi (0x20000000)
\
             ______  _____  _____                         
            /  __  |/ ___ |/ __  |                       
            | |  | | |__/ | |  | |___  ___ _ __  ___  ___
            | |  | |  ___/| |  | / __|/ _ \ '_ \/ __|/ _ \
            | |__| | |    | |  | \__ \  __/ | | \__ \  __/
            |_____/|_|    |_| /__|___/\___|_| |_|___/\___|

 /-----------------------------------------\     @@@@@@@@@@@@@@@@@@@@@@@@@@@@
 |                                         |   @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
 |  1. Boot Multi user [Enter]             |   @@@@@                    @@@@@
 |  2. Boot Single user                    |       @@@@@            @@@@@   
 |  3. Escape to loader prompt             |    @@@@@@@@@@@       @@@@@@@@@@@
 | Autoboot in 0 seconds. [Space] to pause |         \\\\\         /////     
 |  5. Cons: Dual (Video primary)          |   ))))))))))))       (((((((((((
Loading kernel...                          |         /////         \\\\\     
/boot/kernel/kernel text=0x2a8 text=0x924830 text=0x219e64 data=0x1b21c8 data=0x0+0x36a000 syms=[0x8+0x1312e0+0x8+0x156774]
Loading configured modules... (1 of 1)     |       @@@@@            @@@@@   
/boot/kernel/if_enc.ko text=0x15ca text=0x934 data=0x750 syms=[0x8+0xca8+0x8+0xb71]
/boot/kernel/if_bridge.ko text=0x3677 text=0x7168 data=0xd38+0x8 syms=[0x8+0x1b90+0x8+0x15b5]
loading required module 'bridgestp'        |   @@@@@@@@@@@@@@@@@@@@@@@@@@@@ 
/boot/kernel/bridgestp.ko text=0x12e4 text=0x4acc data=0x2c8+0x28 syms=[0x8+0xb88+0x8+0x6e9]
/etc/hostid size=0x25nds. [Space] to pause
/boot/kernel/pf.ko text=0xe344 text=0x3dc7c data=0x59c0+0x39c syms=[0x8+0x5910+0x8+0x46ec]
/boot/kernel/pflog.ko text=0xf78 text=0x898 data=0x430 syms=[0x8+0x9f0+0x8+0x712]
/boot/kernel/if_gre.ko text=0x2736 text=0x4938 data=0x8c8+0x40 syms=[0x8+0x16f8+0x8+0xfaf]
/boot/kernel/if_lagg.ko text=0x3c93 text=0xa41c data=0xd08+0x8 syms=[0x8+0x1c80+0x8+0x15a6]
loading required module 'if_infiniband'
/boot/kernel/if_infiniband.ko text=0x1102 text=0x10fc data=0x300+0x8 syms=[0x8+0x8d0+0x8+0x59a]
/boot/entropy size=0x1000
/boot/kernel/carp.ko text=0x33ec text=0x6e54 data=0xc60+0x48 syms=[0x8+0x1980+0x8+0x1225]
/boot/kernel/pfsync.ko text=0x2f1c text=0x7b24 data=0xb30+0x8 syms=[0x8+0x1800+0x8+0x117d]
No valid device tree blob found!
WARNING! Trying to fire up the kernel, but no device tree blob found!
EFI framebuffer information:
addr, size     0x0, 0x0
dimensions     0 x 0
stride         0
masks          0x00000000, 0x00000000, 0x00000000, 0x00000000


Do you have hw.uart.console="" in loader.conf.local under /boot? If not create the file and add that line.

Whats the best way to mount the image? Im not sure how to edit loader.conf.local

[john_matrix]

I can confirm that the 22.1 image made by YRZR works with DFrobot carrier board with CM4 with 8GB Ram. Unfortunately the performance is only around 300Mb/s on 1Gb connection. Maybe some tuning needed to get better results. The cpu was only at 40% during downloads. Has anyone got better performance?

Thank you very much, I will take a look on this ;-)
Edit: testing cannot continue as I cannot boot on CM4 + DFRobot Mini router board

[sparticle]

Sorry for the late reply!
Was busy with other stuff lately, had little to no time and to totally forgot to reply.

Hello @efetropy,

Can you share the patches required to build opnsense 22.1 for a Raspberry Pi 4? I'd like to help with this.

I'm just getting started and have finally understood how to build it but missing the specific configuration for RPI4.

Dear all,

Currently, I have a CM4 (with 4GB RAM and 8GB eMMC and no Wifi) and the DFRobot IoT Router Carrier Board (https://www.dfrobot.com/product-2242.html).

I am interested to compile an OPNsense build for this hardware but I am a little bit lost about it's potential compatibility.

Can someone confirm me that's this will be good or not?

Many thanks in advance!

There is already an image built by @yrzr for the RPi which can be found here:
https://ftp.yrzr.tk/opnsense/FreeBSD%3A13%3Aaarch64/22.1/images/

A generic aarch64 vmdk image is also available (for e.g. for proxmox, esxi fling or whatever you're using).

The RPi 4B should work without problems (if not, just get some different boot files).

CM4 is a different case. It does work with the official I/O board albeit PCIe issues (see bugzilla).
Your carrier board states ETH2: PCI Express 1000BASE-T NIC (based on RTL8111). RTL8111 drivers are in fact available, but
it will be a hit or miss and you might not be able to the use the 2nd ethernet port due to the mentioned PCIe issues.
Furthermore, you have to figure out how to deal with the dtb file. Worst case scenario, you might have to make it compatible with freebsd.

Trying to get the 22.1.10 build working on my RPI4B 4GB but it does not book.

Can you advise how to get a working build for the RPI4?

Cheers