Messages

Hi

The following is the topology of my network

Site A)
FW: OPNsense 20.7.7_1-amd64
WAN:  192.168.100.34/24
LAN: 192.168.60.1/24
IPSEC: 10.101.0.5/31
GRE: 10.100.0.5/31

Machine on Site A Ubuntu
WAN 192.168.60.20/24 connected to Site A FW LAN

Site B)
FW: OPNsense 20.7.7_1-amd64
WAN:  192.168.100.38/24
LAN: 192.168.59.1/24
IPSEC: 10.101.0.4/31
GRE: 10.100.0.4/31

Machine on Site B Freebsd
WAN 192.168.60.20/24 connected to Site B FW LAN

Both firewall WANs connected to a single gateway and can ping each other

I have setup a routed IPSec Tunnel as given in the link
https://docs.opnsense.org/manual/how-tos/ipsec-s2s-route.html

I have also setup a GRE tunnel on the IPSEC tunnel. On both ends OSPF is running using frr.

All the tunnels come up, the routes information is exchanged properly for OSPF. Both FWs routes show the others LAN subnet

Firewall A routes

Code Select Expand


Code Network Administrative Distance Metric Interface Via Time
O 0.0.0.0/0 110 10 gre0 onlink 10.100.0.4 weight
K > * 0.0.0.0/0 0 0 00:26:37 192.168.100.1
K > * 8.8.8.8/32 0 0 00:24:12 192.168.100.1
O 10.100.0.4/31 110 20 gre0 onlink 10.100.0.4 weight
C > * 10.100.0.4/31 0 1 gre0 00:26:37
C > * 10.101.0.4/32 0 1 ipsec1000 00:26:37
O > * 10.101.0.5/32 110 20 gre0 onlink 10.100.0.4 weight
O > * 192.168.59.0/24 110 20 gre0 onlink 10.100.0.4 weight
C > * 192.168.60.0/24 0 1 em2 00:26:37
O 192.168.100.0/24 110 20 gre0 onlink 10.100.0.4 weight
* 192.168.100.0/24 0 1 em1 00:26:37
C > * 192.168.100.0/24 0 1 em0 00:26:37


Firewall B routes

Code Select Expand


Code Network Administrative Distance Metric Interface Via Time
K > * 0.0.0.0/0 0 0 00:31:21 192.168.100.1
K > * 8.8.8.8/32 0 0 00:28:51 192.168.100.1
O 10.100.0.4/31 110 20 gre0 onlink 10.100.0.5 weight
C > * 10.100.0.4/31 0 1 gre0 00:31:21
O > * 10.101.0.4/32 110 20 gre0 onlink 10.100.0.5 weight
C > * 10.101.0.5/32 0 1 ipsec1000 00:31:21
C > * 192.168.59.0/24 0 1 em2 00:31:21
O > * 192.168.60.0/24 110 20 gre0 onlink 10.100.0.5 weight
O 192.168.100.0/24 110 20 gre0 onlink 10.100.0.5 weight
* 192.168.100.0/24 0 1 em1 00:31:21
C > * 192.168.100.0/24 0 1 em0 00:31:21



The problem is that Machine A and Machine B are unable to ping or communicate with each others. Using TCP dumps the icmp packet hop is like this

Code Select Expand


FWA LAN--> 192.168.60.20 > 192.168.59.20: ICMP echo request, id 3994
FWA GRE--> OPNsense.localdomain > 192.168.59.20: ICMP echo request, id 18358, seq 1
FWB GRE--> 10.100.0.5 > 192.168.59.20: ICMP echo request, id 18358, seq 1
FWB LAN--> 10.100.0.5 > 192.168.59.20: ICMP echo request, id 18358, seq 1

FWB LAN--> 192.168.59.20 > 10.100.0.5: ICMP echo reply, id 18358, seq 1
FWB GRE--> 192.168.59.20 > 10.100.0.5: ICMP echo reply, id 18358, seq 1
FWA GRE--> 192.168.59.20 > OPNsense.localdomain: ICMP echo reply, id 18358, seq 1
FWA LAN-->  ***********


However FW A can ping Machine B and likewise FW B can ping Machine A

The attached images shows the NAT outbound rules on both sides.

As an experiment I changed the interface in GRE tunnel from the IPSEC tunnel to the WAN interfaces on both sides and the Machines on the LAN subnets were able to ping each other.

Is there any special NAT rules that I have to add so that both the networks can communicate with each other?. Also why does the communication between Machines A and B work when the GRE tunnel is unencrypted but not with IPSEC?. Has anyone else faced such an issue?

Hi Franco

Regarding the issue that is being discussed in this thread, unfortunately we are in a same predicament. We dont have the option of an HDD, or USB, so we have to find a way to install OPNSense on the eMMC. Is there any patch available which we can apply to build a custom image which will be able to detect the eMMC and install on it.

As per this bug report, we too are using a Denverton (C3558) to be specific
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=228340

Thanks and Regards

I am also looking at the possibility of booting an OPNSense image using PXE. Specifically without using NFS. Does anybody have any experience in this for OPNSense

Thanks

Thanks again for the help. I tried what you suggested and now the DPDK library is built into the image

Thanks again for the guidance

I am running into one more issues. When I build the virtual image using the command

make vm-vmdk ADDITIONS="dpdk"

I can see the DPDK included in the build logs

Installing dpdk-19.11_2...
`-- Installing openssl-1.1.1d,1...
`-- Extracting openssl-1.1.1d,1: .......... done


When I run the image in VirtualBox, I am able to see the library installed and I was able to run DPDK tests on the image. Now when I tried to build the serial image using the command

make serial ADDITIONS="dpdk"

The DPDK library is not installed or built and when the image is run, the DPDK library is not included in the image.

Is there a special mechanism to add ports to a serial image compared to a VM image.

Thanks and Regards

Thanks Franco.

Your suggestion worked and it builds dpdk now.

Hi All

Our company is testing the performance of OPNSense on our custom board based on an Intel Denverton chip. Since we are using the Denverton chip therefore we wanted to explore how DPDK can be used on OPNSense.

Right now we are having trouble building the DPDK sources using the steps given below.

We followed the build instructions using the following link and were able to build an image.
https://github.com/opnsense/tools.

After building the image we tried to add DPDK into the image as well. We did this by adding the following line to the file /usr/tools/onfig/20.1/ports.conf

net/dpdk

and building the ports and specifically the DPDK port using the command
@freebsd:/usr/tools # make ports-dpdk

However the build fails with the following error.

FAILED: kernel/freebsd/contigmem.ko
make -f ../kernel/freebsd/BSDmakefile.meson KMOD_OBJDIR=kernel/freebsd KMOD_SRC=../kernel/freebsd/contigmem/contigmem.c KMOD=contigmem 'KMOD_CFLAGS=-I/usr/obj/usr/ports/net/dpdk/work/dpdk-19.11/_build -I/usr/obj/usr/ports/net/dpdk/work/dpdk-19.11/config -include rte_config.h' CC=clang
ld: bad -rpath option
*** Error code 1

Has any one ever had any experience integrating the DPDK libraries into an OPNSense build. Did anyone come across this error when integrating DPDK into OPNSense image

Side note, I was able to build the DPDK port freebsd OS on which I was trying to build the  OPNSense image

Thanks