OSPF via GRE/IPSec

[mimugmail]

I currently set up a lab cause of wireguard reports with OSPF, I'll try to test this too

[@Vorona]

But could you try to disable IPsec just for debugging purposes and try if OSPF works over GRE alone?

I cannot do that. Only one point has real IP address. Otrer point behind NAT. For this reason IPSec works in tunnel mode. Why do you think, that IPSec can break OSPF, wich works in GRE? Tunnel works, firewall on tunnel at both sides are open.

I currently set up a lab cause of wireguard reports with OSPF, I'll try to test this too

You mean RB IPSec or OSPF relations between Opnsense and MikroTik?

[mimugmail]

Ok, I just finished the lab and I'm able to ping from host-A to host-B via FW-A to FW-B which has a IPsec with type transport and inside a GRE tunnel and inside GRE the OSPF protocol.

root@OPNsense:~ # tcpdump -n -i gre0
15:37:40.506482 IP 10.253.253.2 > 224.0.0.5: OSPFv2, Hello, length 48
15:37:40.518025 IP 10.253.253.1 > 224.0.0.5: OSPFv2, Database Description, length 32
15:37:40.530882 IP 10.253.253.2 > 224.0.0.5: OSPFv2, Database Description, length 32
15:37:40.537441 IP 10.253.253.1 > 224.0.0.5: OSPFv2, Database Description, length 72
15:37:40.541540 IP 10.253.253.2 > 224.0.0.5: OSPFv2, Database Description, length 72
15:37:40.541671 IP 10.253.253.2 > 224.0.0.5: OSPFv2, LS-Request, length 48
15:37:40.542538 IP 10.253.253.1 > 224.0.0.5: OSPFv2, Database Description, length 32
15:37:40.542587 IP 10.253.253.1 > 224.0.0.5: OSPFv2, LS-Request, length 48
15:37:40.542741 IP 10.253.253.1 > 224.0.0.5: OSPFv2, LS-Update, length 88
15:37:40.543885 IP 10.253.253.2 > 224.0.0.5: OSPFv2, LS-Update, length 88
15:37:40.544078 IP 10.253.253.2 > 224.0.0.5: OSPFv2, LS-Update, length 64
15:37:40.545609 IP 10.253.253.1 > 224.0.0.5: OSPFv2, LS-Update, length 64
15:37:40.725297 IP 10.253.253.1 > 224.0.0.5: OSPFv2, LS-Ack, length 64
15:37:41.359517 IP 10.253.253.2 > 224.0.0.5: OSPFv2, LS-Ack, length 64
15:37:45.574028 IP 10.253.253.1 > 224.0.0.5: OSPFv2, LS-Update, length 64
15:37:45.575582 IP 10.253.253.2 > 224.0.0.5: OSPFv2, LS-Update, length 64
15:37:45.595667 IP 10.253.253.1 > 224.0.0.5: OSPFv2, Hello, length 48
15:37:45.893086 IP 10.253.253.1 > 224.0.0.5: OSPFv2, LS-Ack, length 44
15:37:46.587098 IP 10.253.253.2 > 224.0.0.5: OSPFv2, LS-Ack, length 44
15:37:46.673133 IP 192.168.10.2 > 192.168.11.3: ICMP echo request, id 63907, seq 44857, length 64
15:37:46.675096 IP 192.168.11.3 > 192.168.10.2: ICMP echo reply, id 63907, seq 44857, length 64
15:37:46.872543 IP 192.168.11.3 > 192.168.10.2: ICMP echo request, id 34831, seq 44822, length 64
15:37:46.873544 IP 192.168.10.2 > 192.168.11.3: ICMP echo reply, id 34831, seq 44822, length 64
15:37:47.723243 IP 192.168.10.2 > 192.168.11.3: ICMP echo request, id 63907, seq 44858, length 64
15:37:47.723928 IP 192.168.11.3 > 192.168.10.2: ICMP echo reply, id 63907, seq 44858, length 64
15:37:47.942779 IP 192.168.11.3 > 192.168.10.2: ICMP echo request, id 34831, seq 44823, length 64
15:37:47.943702 IP 192.168.10.2 > 192.168.11.3: ICMP echo reply, id 34831, seq 44823, length 64
15:37:48.793598 IP 192.168.10.2 > 192.168.11.3: ICMP echo request, id 63907, seq 44859, length 64
15:37:48.794412 IP 192.168.11.3 > 192.168.10.2: ICMP echo reply, id 63907, seq 44859, length 64
15:37:48.972486 IP 192.168.11.3 > 192.168.10.2: ICMP echo request, id 34831, seq 44824, length 64
15:37:48.973553 IP 192.168.10.2 > 192.168.11.3: ICMP echo reply, id 34831, seq 44824, length 64
15:37:49.823147 IP 192.168.10.2 > 192.168.11.3: ICMP echo request, id 63907, seq 44860, length 64
15:37:49.823893 IP 192.168.11.3 > 192.168.10.2: ICMP echo reply, id 63907, seq 44860, length 64
15:37:50.042721 IP 192.168.11.3 > 192.168.10.2: ICMP echo request, id 34831, seq 44825, length 64

[@Vorona]

Good!
What hardware do you use with opnsense? What version?

In my gre I see OSPF traffic in both ways too, but route wrong.

[mimugmail]

Just VMs with 21.1.4

[Fox_exe]

And same strange things for me.
On mikrotik side, on LSA tab i found this:

Wrong data (OPNsense)

Code Select Expand

flags=
links (type, id, data, metric)
    Point-To-Point 10.74.0.1 0.0.0.7 10
    Stub 10.77.0.0 255.255.255.0 10


Correct data (Mikrotik)

Code Select Expand

flags=
links (type, id, data, metric)
    Point-To-Point 10.74.0.1 172.16.0.2 10
    Stub 172.16.0.0 255.255.255.252 10
    Stub 10.77.0.0 255.255.255.0 10

Note:
* 10.74.0.1/16 - Router ID of Mikrotik #1
* 10.77.0.1/16 - Router ID of OPNsense
172.16.0.0/30 - GRE tunnel network:
172.16.0.1 - mikrotik ip
172.16.0.2 - opnsense ip

What is "0.0.0.7" ?
Any ideas how to fix it (Make it work?)
Look like OSPF won't announce GRE tunnel network to Mikrotik, so mikrotik don't known how to route 10.77 network (And show it as "empty" gateway and interface)

note2:
In OPNSense, on "Network" side i add two networks: LAN and GRE (to area "0.0.0.0"). On "Interfaces" - LAN and GRE interfaces.

[mimugmail]

Screenshots please

[Fox_exe]

What exacly you want to see?
OSPF and GRE settings almost same on Mikrotik and OPNsense.

Router #1: 10.74.0.0/16 (Mikrotik, gre ip: 172.16.74.13)
Router #2: 10.77.0.0/16 (OPNsense, gre ip: 172.16.74.14)
GRE between routers: 172.16.74.12/30
Can ping both gre ip's. Can ping and access resource behind both routers if select interface manually or add static route.
But on mikrotik side - ospf won't work. Route added without gateway and interface.

OSPF config on OPNsense:

Code Select Expand

Current configuration:
!
frr version 7.4
frr defaults traditional
hostname gw.dc.sipcolor.ru.sipcolor.ru
log syslog notifications
!
router ospf
ospf router-id 10.77.0.1
passive-interface bridge0
passive-interface vtnet0
passive-interface vtnet1
network 10.77.0.0/16 area 0.0.0.0
network 172.16.74.12/30 area 0.0.0.0
!
line vty
!
end

[mimugmail]

When you capture on GRE .. do you see OSPF packets in and out?

[Fox_exe]

When you capture on GRE .. do you see OSPF packets in and out?

Yep. Hello packets, LSA...
I see neighbors in OSPF on both routers, i see routes. But on OPNsense routes is correct, but on Mikrotik - with empty gateways/interfaces (see screenshot above).

I test OPNsense and Mikrotik (as 2nd router) and found difference:
OPNsense (fw_v21.1.8_1 / FRR_v7.4_6):

Code Select Expand

Type: Stub     ID: 10.77.0.0       Data: 255.255.0.0     Metric: 10
Type: PTP      ID: 10.74.0.1       Data: 0.0.0.8         Metric: 10

Mikrotik (fw_v6.48.3):

Code Select Expand

Type: Stub     ID: 10.77.0.0       Data: 255.255.0.0     Metric: 10
Type: PTP      ID: 10.74.0.1       Data: 172.16.74.14    Metric: 10
Type: Stub     ID: 172.16.74.12    Data: 255.255.255.252 Metric: 11


As you see - mikrotik send an additional "stub" route, but OPNsense - ony strange "0.0.0.8".
On Wiki i found - thats a port number (MIB-II notation) and Mikrotik won't understand this form of record (Or expect a regular route/ip address).

Any ideas how to fix it?

[mimugmail]

What is 172.16.255? 0.0.0.7 is a wildcard mask.

[Fox_exe]

What is 172.16.255? 0.0.0.7 is a wildcard mask.

172.16.255.74 = 10.74.0.1 (Router ID). Sorry, wrong screenshot (I just try to change router ID)

[lilsense]

create a loopback and place all your devices in the same subnet for the loopback. place the loopback in the area 0. make sure your loopback is a passive interface.

[Fox_exe]

create a loopback and place all your devices in the same subnet for the loopback. place the loopback in the area 0. make sure your loopback is a passive interface.

Ok... I did:
* Interfaces - Other types - Loopback - Add (Description: "172.16.255.77". It is ok? I can't set text here, only ip-like record)
* Interfaces - Assignments - Loopback - Add. Set Enabled, IPv4 = Static, 172.16.255.77/32
* Routing - OSPF - Router ID set to "172.16.255.77", added loopback to passive interfaces
* Routing - OSPF - Networks - Add: 172.16.0.0/16, Area: 0.0.0.0

On mikrotik - add only network to area 0.0.0.0...
Now - i have additional route in ospf routes list on mikrotik... Without gateway/interface...
I think mikrotik just not understand "Wildcard" mask... It is possible to swith it on OPNsense (For compatability)?

On Mikrotik (Now his ID is "172.16.255.74"):

Code Select Expand

flags=
links (type, id, data, metric)
    Stub 10.77.0.0 255.255.0.0 10
    Point-To-Point 172.16.255.74 0.0.0.8 10
    Stub 172.16.255.77 255.255.255.255 0


[Fox_exe]

Hmm.. Seems like its a bug in FRR: https://github.com/FRRouting/frr/issues/3973
And its still not fixed :(