VRF Support Question

Started by seed, February 12, 2022, 08:51:35 PM

Previous topic - Next topic
Hello,

I wanted to ask if it is also possible to create VRFs with OPNsense/Freebsd.
A possible application would be e.g. I build a tunnel to xyz and put the tunnel interface as default gateway in the VRF. Then I create a VLAN interface in the VRF and set it as default gateway for the clients in the VLAN. Then e.g. guests with IPs from a HE.net tunnel could use the internet. Independent of the primary routing table.
i want all services to run with wirespeed and therefore run this dedicated hardware configuration:

AMD Ryzen 7 9700x
ASUS Pro B650M-CT-CSM
64GB DDR5 ECC (2x KSM56E46BD8KM-32HA)
Intel XL710-BM1
Intel i350-T4
2x SSD with ZFS mirror
PiKVM for remote maintenance

private user, no business use


you can look at the post below... you would need FRR for this but not sure how far you can go with it...

https://forum.opnsense.org/index.php?topic=24099.0

Sounds hacky, but keep us updated if it works

February 14, 2022, 09:11:50 AM #4 Last Edit: February 14, 2022, 09:40:10 AM by seed
I did a quick test with my "emergency Interface" this morning. The interface is configured with simple defaults so that im able to access the OPNsense in case i messed somethin up. It is suitable beeing configured in a seperate routing table so that one can rescue the system in case everything is broken.

What i configured:

Tunable Config:

    net.fibs: 2
    net.add_addr_allfibs: 0


GUI: Disable interface that should be in second routing table and restart OPNsense

Commandline:

    ifconfig igb0 fib 1
    setfib 1 netstat -rn


GUI: Enable Interface

Check routing Table:

# The Interface should not be listed in the main Routing Table
setfib 0 netstat -rn

# The Interface should be in the second Routing Table
setfib 1 netstat -rn




What i need to test:

-Configure the Interface IP with an IP thats already configured on another Interface.
-Check how Firewalling behaves
-Check how Applications listenting on the Interface behave




If it works nicely it would be amazing if the next Kernel would be compiled supporting multiple routing tables.
It would be also nice if one can select a different routing table while configuring an interface.
i want all services to run with wirespeed and therefore run this dedicated hardware configuration:

AMD Ryzen 7 9700x
ASUS Pro B650M-CT-CSM
64GB DDR5 ECC (2x KSM56E46BD8KM-32HA)
Intel XL710-BM1
Intel i350-T4
2x SSD with ZFS mirror
PiKVM for remote maintenance

private user, no business use

Quote from: seed on February 14, 2022, 09:11:50 AM
I did a quick test with my "emergency Interface" this morning. The interface is configured with simple defaults so that im able to access the OPNsense in case i messed somethin up. It is suitable beeing configured in a seperate routing table so that one can rescue the system in case everything is broken.

What i configured:

Tunable Config:

    net.fibs: 2
    net.add_addr_allfibs: 0


GUI: Disable interface that should be in second routing table and restart OPNsense

Commandline:

    ifconfig igb0 fib 1
    setfib 1 netstat -rn


GUI: Enable Interface

Check routing Table:

# The Interface should not be listed in the main Routing Table
setfib 0 netstat -rn

# The Interface should be in the second Routing Table
setfib 1 netstat -rn




What i need to test:

-Configure the Interface IP with an IP thats already configured on another Interface.
-Check how Firewalling behaves
-Check how Applications listenting on the Interface behave




If it works nicely it would be amazing if the next Kernel would be compiled supporting multiple routing tables.
It would be also nice if one can select a different routing table while configuring an interface.



Looking at: https://github.com/opnsense/src/blob/stable/13/sys/conf/NOTES

# NETWORKING OPTIONS

#
# Protocol families
#
options INET #Internet communications protocols
options INET6 #IPv6 communications protocols

options RATELIMIT # TX rate limiting support

options [b]ROUTETABLES=2 [/b] # allocated fibs up to 65536. default is 1.
# but that would be a bad idea as they are large.


Is it possible to change the "ROUTETABLES" option to 65536? Does this cause issues?
I guess the Kernel needs to be recompiled in order to make those changes active.

Im curious to find out if its Possible to run multiple "virtual" OPNsense Firewalls on one chassis without the overhead of virtualization.
i want all services to run with wirespeed and therefore run this dedicated hardware configuration:

AMD Ryzen 7 9700x
ASUS Pro B650M-CT-CSM
64GB DDR5 ECC (2x KSM56E46BD8KM-32HA)
Intel XL710-BM1
Intel i350-T4
2x SSD with ZFS mirror
PiKVM for remote maintenance

private user, no business use

Quote from: seed on February 14, 2022, 09:11:50 AM

What i need to test:

-Configure the Interface IP with an IP thats already configured on another Interface.
-Check how Firewalling behaves
-Check how Applications listenting on the Interface behave


I had just tested. Unfortunately, the configuration does not work (yet). The interface in the VRF can be pinged. But the web GUI of the OPNSense cannot be reached. In the meantime I restarted the web GUI, but without success. I suspect that the URPF mentioned in the other forum post is interfering.
i want all services to run with wirespeed and therefore run this dedicated hardware configuration:

AMD Ryzen 7 9700x
ASUS Pro B650M-CT-CSM
64GB DDR5 ECC (2x KSM56E46BD8KM-32HA)
Intel XL710-BM1
Intel i350-T4
2x SSD with ZFS mirror
PiKVM for remote maintenance

private user, no business use

Quote from: seed on February 14, 2022, 05:34:48 PM
I had just tested. Unfortunately, the configuration does not work (yet). The interface in the VRF can be pinged. But the web GUI of the OPNSense cannot be reached. In the meantime I restarted the web GUI, but without success. I suspect that the URPF mentioned in the other forum post is interfering.

By default, all commands run in the context of VRF 0. If a command is expected to run in another context, it must be prepended with setfib(1). So I suspect that changing the VRF of the GUI interface also requires amending rc and configd scripts.
--
Marin BERNARD
System administrator