issue with routing ?

[mathie]

I am unable to use opnsense as router.

my setup:

Code: [Select]

root@OPNsense:~ # netstat -4nr
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            192.168.178.1      UGS    ue0_vlan
127.0.0.1          link#2             UH          lo0
192.168.178.0/24   link#6             U      ue0_vlan
192.168.178.2      link#6             UHS         lo0
192.168.188.0/24   link#5             U           ue0
192.168.188.31     link#5             UHS         lo0

Note that I have default gateway set to 192.168.178.1, which is my DSL Modem.

On my Client with IP 192.168.188.48 I am trying to access internet via opnsense router, so I maintain the following routing table:

Code: [Select]

netstat -nr
Kernel IP Routentabelle
Ziel            Router          Genmask         Flags   MSS Fenster irtt Iface
0.0.0.0         192.168.188.31  0.0.0.0         UG        0 0          0 enp2s0
192.168.188.0   0.0.0.0         255.255.255.0   U         0 0          0 enp2s0

Now I am trying to reach internet.
From opnsense it is working (via 192.168.178.1):

Code: [Select]

root@OPNsense:~ # ping freebsd.org
PING freebsd.org (96.47.72.84): 56 data bytes
64 bytes from 96.47.72.84: icmp_seq=0 ttl=53 time=104.885 ms
64 bytes from 96.47.72.84: icmp_seq=1 ttl=53 time=103.891 ms
^C
--- freebsd.org ping statistics ---
2 packets transmitted, 2 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 103.891/104.388/104.885/0.497 ms


From Client it is not:

Code: [Select]

ping freebsd.org
PING freebsd.org (96.47.72.84) 56(84) bytes of data.
^C
--- freebsd.org ping statistics ---
1 Pakete übertragen, 0 empfangen, 100% packet loss, time 0ms

On opnsense I see packats comming in, but seems like they are not routed to default gateway (because I dont get responses)

Code: [Select]

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ue0, link-type EN10MB (Ethernet), capture size 262144 bytes
22:51:06.363473 IP 192.168.188.48 > wfe0.nyi.freebsd.org: ICMP echo request, id 6, seq 1, length 64
^C
1 packet captured
56 packets received by filter
0 packets dropped by kernel


Do I miss firewall rules ? Which rules I have to implement for interfaces ue0 and ue0_vlan ?

[mathie]

What I actually want to achieve is a "router-on-a-stick" setup. So I am using a raspi with only single ethernet port in combination with a managed switch. On the switch I already setup adequate vlans, untagged ports and a trunk port with all my vlans. The latter is connected with my opnsense.
Questions: Is it necessery to setup a bridge and configure vlan interfaces below the bridge ? What is general solution look like ?

[Patrick M. Hausen]

Generally router-on-a-stick with VLANs works just fine. I have that running here. Just configure the VLANs on top of the physical interface and assign them to WAN, LAN, OPT1, ... as fit.

First question: you are running OPNsense on a Raspi? Isn't that supposed to be highly "experimental early alpha etc. etc."? Can you confirm that VLANs work at all on ARM? Just a wild guess, of course. What I can confirm is that this setup works just finde on Intel/AMD.

HTH,
Patrick

[ajm]

Digressing a bit from the OT, but Aarch64 is tier 1 supported since 13.0. Seems reasonable for OPN to leverage this. I have 2x RPi4b 8GB on FreeBSD 13.0 as app servers using jails for failover/redundancy, using dot1q trunk uplinks, all working fine.

[mathie]

Thanks everybody for your replies. Which helped at least mentally  ... because I saw I am somehow on the right track.
In the end I had indeed 2 routing issues  when traffic was routed back into my home-net/client:
1. The fritzbox didnt had the route to reach the nets spanned by opnsense router/gateway.
2. In the first setup I had 2nd fritzbox wich I was using as kind of "poor man" intranet/dmz gateway.

This means no issues with opnsense at all. On the other hand lot of issues while passing back traffic via my 2 fritzboxes.

I think, opnsense is really easy and pretty much forward. From what I saw in the last few days... OK, I have to admit I did pretty much using the defaults, while configuring everything.