[SOLVED] Static route removed after reboot

[Seimus]

Hello everyone,

First, let me say great work on the OPNsense as such. I have been using it for a year and am very happy with it!

Now to the problem.

I am running OPN on the latest OS, 23.1.5_4 HW APU2D2. And I am seeing that on this current release, every time I reboot the device, the static route I have set is removed. I have to reapply it in order to have it in the routing table again.

Has anyone else encountered this issue as well?

Regards,
Seimus

[Patrick M. Hausen]

Does the route use a gateway and/or interface that is possibly not up and connected at boot time? Like a PPPoE WAN link?

Sometimes switches take 30 seconds to bring a port into the forwarding state if they run an old fashioned but correct implementation of the spanning tree protocol.

[Seimus]

Thats a fair point.

I have a LAG between the OPN and Zyxel GS1900 managed switch. But I am not running SPT on the switch. The route is set to Follow a GW which is an IP in one of the VLAN and its set to be persistent UP.

For Example I rebooted OPN its UP for 8 min and the static route is still missing.

When I go to Routes > Configuration I can see the route is enabled. I have to hit the apply button for the route to be installed in the Route table.


Bellow Logs from OPN:

#After reboot

Code Select Expand

2023-04-12T20:03:27 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : unbound_configure_do())
2023-04-12T20:03:27 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : dnsmasq_configure_do())
2023-04-12T20:03:27 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns ()
2023-04-12T20:03:27 Error opnsense /usr/local/etc/rc.linkup: The command '/bin/kill -'TERM' '75365'' returned exit code '1', the output was 'kill: 75365: No such process'
2023-04-12T20:03:27 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2023-04-12T20:03:27 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp ()
2023-04-12T20:03:27 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (execute task : ipsec_configure_do(,opt3))
2023-04-12T20:03:27 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (,opt3)
2023-04-12T20:03:27 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: skipping IPv4 default route
2023-04-12T20:03:27 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to wan
2023-04-12T20:03:27 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: entering configure using 'opt3'
2023-04-12T20:03:26 Notice opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for opt3(vlan010)
2023-04-12T20:03:23 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : unbound_configure_do())
2023-04-12T20:03:23 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : dnsmasq_configure_do())
2023-04-12T20:03:23 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns ()
2023-04-12T20:03:23 Error opnsense /usr/local/etc/rc.linkup: The command '/bin/kill -'TERM' '75365'' returned exit code '1', the output was 'kill: 75365: No such process'
2023-04-12T20:03:23 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2023-04-12T20:03:23 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp ()
2023-04-12T20:03:23 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (execute task : ipsec_configure_do(,opt8))
2023-04-12T20:03:23 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (,opt8)
2023-04-12T20:03:23 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: skipping IPv4 default route
2023-04-12T20:03:23 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to wan
2023-04-12T20:03:23 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: entering configure using 'opt8'
2023-04-12T20:03:22 Notice opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for opt8(vlan0999)
2023-04-12T20:03:19 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : unbound_configure_do())
2023-04-12T20:03:19 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : dnsmasq_configure_do())
2023-04-12T20:03:19 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns ()
2023-04-12T20:03:19 Error opnsense /usr/local/etc/rc.linkup: The command '/bin/kill -'TERM' '75365'' returned exit code '1', the output was 'kill: 75365: No such process'
2023-04-12T20:03:19 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2023-04-12T20:03:19 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp ()
2023-04-12T20:03:19 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (execute task : ipsec_configure_do(,opt7))
2023-04-12T20:03:19 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (,opt7)
2023-04-12T20:03:19 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: skipping IPv4 default route
2023-04-12T20:03:19 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to wan
2023-04-12T20:03:19 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: entering configure using 'opt7'
2023-04-12T20:03:19 Notice opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for opt7(vlan0201)
2023-04-12T20:03:15 Notice syslog-ng Configuration reload finished;
2023-04-12T20:03:15 Notice syslog-ng Configuration reload request received, reloading configuration;
2023-04-12T20:03:13 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : unbound_configure_do())
2023-04-12T20:03:13 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : dnsmasq_configure_do())
2023-04-12T20:03:13 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns ()
2023-04-12T20:03:13 Error opnsense /usr/local/etc/rc.linkup: The command '/bin/kill -'TERM' '75365'' returned exit code '1', the output was 'kill: 75365: No such process'
2023-04-12T20:03:13 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2023-04-12T20:03:13 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp ()
2023-04-12T20:03:13 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (execute task : ipsec_configure_do(,opt5))
2023-04-12T20:03:13 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (,opt5)
2023-04-12T20:03:13 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: skipping IPv4 default route
2023-04-12T20:03:13 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to wan
2023-04-12T20:03:13 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: entering configure using 'opt5'
2023-04-12T20:03:13 Notice opnsense /usr/local/etc/rc.routing_configure: plugins_configure monitor (execute task : dpinger_configure_do(1))
2023-04-12T20:03:13 Notice opnsense /usr/local/etc/rc.routing_configure: plugins_configure monitor (1)
2023-04-12T20:03:12 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: keeping current inet default gateway 'IP'
2023-04-12T20:03:12 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: setting IPv4 default route to IP
2023-04-12T20:03:12 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: IPv4 default gateway set to wan
2023-04-12T20:03:12 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: entering configure using defaults
2023-04-12T20:03:12 Notice opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for opt5(vlan099)
2023-04-12T20:03:08 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : unbound_configure_do())
2023-04-12T20:03:08 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : dnsmasq_configure_do())
2023-04-12T20:03:08 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns ()
2023-04-12T20:03:08 Error opnsense /usr/local/etc/rc.linkup: The command '/bin/kill -'TERM' '75365'' returned exit code '1', the output was 'kill: 75365: No such process'
2023-04-12T20:03:08 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2023-04-12T20:03:08 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp ()
2023-04-12T20:03:08 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (execute task : ipsec_configure_do(,opt6))
2023-04-12T20:03:08 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (,opt6)
2023-04-12T20:03:08 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: skipping IPv4 default route
2023-04-12T20:03:08 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to wan
2023-04-12T20:03:08 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: entering configure using 'opt6'
2023-04-12T20:03:07 Notice opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for opt6(vlan0101)
2023-04-12T20:03:07 Notice opnsense /usr/local/etc/rc.routing_configure: plugins_configure monitor (execute task : dpinger_configure_do(1))
2023-04-12T20:03:07 Notice opnsense /usr/local/etc/rc.routing_configure: plugins_configure monitor (1)
2023-04-12T20:03:07 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: keeping current inet default gateway 'IP'
2023-04-12T20:03:07 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: setting IPv4 default route to IP
2023-04-12T20:03:07 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: IPv4 default gateway set to wan
2023-04-12T20:03:07 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: entering configure using defaults
2023-04-12T20:03:06 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure bootup (execute task : unbound_configure_do(1))
2023-04-12T20:03:06 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure bootup (execute task : opendns_configure_do(1))
2023-04-12T20:03:06 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure bootup (execute task : ntpd_configure_do(1))
2023-04-12T20:03:06 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure bootup (1)
2023-04-12T20:03:06 Notice opnsense /usr/local/etc/rc.bootup: Resyncing OpenVPN instances.
2023-04-12T20:03:06 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure vpn (execute task : openvpn_configure_do(1))
2023-04-12T20:03:03 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure vpn (execute task : ipsec_configure_do(1))
2023-04-12T20:03:03 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure vpn (1)
2023-04-12T20:03:02 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure monitor (execute task : dpinger_configure_do(1,null,1))
2023-04-12T20:03:02 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure monitor (1,null,1)
2023-04-12T20:03:02 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dns (execute task : unbound_configure_do(1))
2023-04-12T20:03:02 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dns (execute task : dnsmasq_configure_do(1))
2023-04-12T20:03:02 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dns (1)
2023-04-12T20:03:01 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dhcrelay (execute task : dhcpd_dhcrelay_configure(1))
2023-04-12T20:03:01 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dhcrelay (1)
2023-04-12T20:03:01 Error opnsense /usr/local/etc/rc.bootup: The command '/bin/kill -'TERM' '75365'' returned exit code '1', the output was 'kill: 75365: No such process'
2023-04-12T20:03:01 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure(1))
2023-04-12T20:03:01 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dhcp (1)
2023-04-12T20:03:01 Notice opnsense /usr/local/etc/rc.bootup: ROUTING: setting inet default route to IP
2023-04-12T20:03:01 Notice opnsense /usr/local/etc/rc.bootup: ROUTING: setting IPv4 default route to IP
2023-04-12T20:03:01 Notice opnsense /usr/local/etc/rc.bootup: ROUTING: IPv4 default gateway set to wan
2023-04-12T20:02:59 Notice opnsense /usr/local/etc/rc.bootup: ROUTING: entering configure using defaults
2023-04-12T20:02:59 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure early (execute task : webgui_configure_do(1))
2023-04-12T20:02:59 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure early (execute task : unbound_cache_flush(1))
2023-04-12T20:02:59 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure early (execute task : openssh_configure_do(1))
2023-04-12T20:02:59 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure early (1)
2023-04-12T20:02:58 Error opnsense /usr/local/etc/rc.bootup: ROUTING: not a valid host gateway address: ''
2023-04-12T20:02:57 Error opnsense /usr/local/etc/rc.bootup: Unable to configure nonexistent interface opt1 (wg1)
2023-04-12T20:02:53 Notice syslog-ng syslog-ng starting up; version='3.38.1'
2023-04-12T20:01:36 Notice syslog-ng syslog-ng shutting down; version='3.38.1'

#After re applying the route in configuration

Code Select Expand

2023-04-12T20:10:31 Notice opnsense /usr/local/etc/rc.routing_configure: plugins_configure monitor (execute task : dpinger_configure_do(1))
2023-04-12T20:10:31 Notice opnsense /usr/local/etc/rc.routing_configure: plugins_configure monitor (1)
2023-04-12T20:10:31 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: keeping current inet default gateway 'IP'
2023-04-12T20:10:31 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: setting IPv4 default route to IP
2023-04-12T20:10:31 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: IPv4 default gateway set to wan
2023-04-12T20:10:31 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: entering configure using defaults

On 22.7.6 this issue was not present and I had the same setup. I upgraded from 22.7.6 to 23.1.5_4 and started to see this behavior.

[Seimus]

I upgraded to 23.1.6, and I still see this issue.

If I am the only one having this weird behavior, I am thinking of reprovisioning OPN from scratch on my device, aka a full wipe and clean install.

Regards,
S.

[franco]

Is it this one? Route over dynamic WireGuard interface?

2023-04-12T20:02:58   Error   opnsense   /usr/local/etc/rc.bootup: ROUTING: not a valid host gateway address: ''   
2023-04-12T20:02:57   Error   opnsense   /usr/local/etc/rc.bootup: Unable to configure nonexistent interface opt1 (wg1)   


Cheers,
Franco

[Seimus]

Hello Franco,

No its not that one. I am having Wireguard but no static routes configured for it as WG is configured on the OPN it routes back to the WG Host via directly connected route /28 subnet in my case. WG is working flawlesly.

The static route I have is set to route Traffic for subnet 172.20.1.0/26 over a host 10.0.0.14. These devices on subnet "172.20.1.0/26" are Docker containers behind RPi with IP 10.0.0.14.

What I see During/After a reboot is, that the route is present, but immediately after few seconds its being removed from the route table.

Regards,
S.

[franco]

I'm wondering why this route should disappear after being added (reliably so on each apply).

What additional services (especially plugins) are you running on the firewall?


Cheers,
Franco

[Seimus]

Here is a list of Services and Plugins I am running:

Service

• configd
• cron
• dhcrelay
• dpinger
• ipfw
• login
• ntpd
• pf
• routing
• sysctl
• syslog-ng
• webgui
• wireguard

Plugins

• os-wireguard (installed)

Packages

• os-wireguard
• wireguard-kmod
• wireguard-tools

I think nothing out of ordinary or at least for me nothing that could cause such behaviour.

Regards,
S.

[franco]

The only questionable thing is wireguard, but that would only happen if routes overlap for the 172.20.1.0/26 network being used somewhere in that configuration. WireGuard can be a bit heavy-handed in its approach to (re)configuring a running system.

To rule that out I would try to disable WireGuard or remove plugin temporarily and reboot to see if the route sticks or not.


Cheers,
Franco

[Seimus]

Hello Franco,

I looked within my configuration and found no route or subnet overlap. I did anyway try what you suggested.

• Disabled Wireguard - same issue and behavior seen, route after reboot dissapeared from the route table.

Code Select Expand


2023-05-03T17:46:04 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : unbound_configure_do())
2023-05-03T17:46:04 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : dnsmasq_configure_do())
2023-05-03T17:46:04 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns ()
2023-05-03T17:46:04 Error opnsense /usr/local/etc/rc.linkup: The command '/bin/kill -'TERM' '75365'' returned exit code '1', the output was 'kill: 75365: No such process'
2023-05-03T17:46:03 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2023-05-03T17:46:03 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp ()
2023-05-03T17:46:03 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (execute task : ipsec_configure_do(,opt5))
2023-05-03T17:46:03 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (,opt5)
2023-05-03T17:46:03 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: skipping IPv4 default route
2023-05-03T17:46:03 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to wan
2023-05-03T17:46:03 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: entering configure using 'opt5'
2023-05-03T17:46:02 Notice opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for opt5(vlan099)
2023-05-03T17:46:02 Notice opnsense /usr/local/etc/rc.routing_configure: plugins_configure monitor (execute task : dpinger_configure_do(1))
2023-05-03T17:46:02 Notice opnsense /usr/local/etc/rc.routing_configure: plugins_configure monitor (1)
2023-05-03T17:46:02 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: keeping current inet default gateway 'IP'
2023-05-03T17:46:02 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: setting IPv4 default route to IP
2023-05-03T17:46:02 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: IPv4 default gateway set to wan
2023-05-03T17:46:02 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: entering configure using defaults
2023-05-03T17:45:58 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : unbound_configure_do())
2023-05-03T17:45:58 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : dnsmasq_configure_do())
2023-05-03T17:45:58 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns ()
2023-05-03T17:45:58 Error opnsense /usr/local/etc/rc.linkup: The command '/bin/kill -'TERM' '75365'' returned exit code '1', the output was 'kill: 75365: No such process'
2023-05-03T17:45:58 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2023-05-03T17:45:58 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp ()
2023-05-03T17:45:58 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (execute task : ipsec_configure_do(,opt6))
2023-05-03T17:45:58 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (,opt6)
2023-05-03T17:45:58 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: skipping IPv4 default route
2023-05-03T17:45:58 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to wan
2023-05-03T17:45:58 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: entering configure using 'opt6'
2023-05-03T17:45:57 Notice opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for opt6(vlan0101)
2023-05-03T17:45:57 Notice opnsense /usr/local/etc/rc.routing_configure: plugins_configure monitor (execute task : dpinger_configure_do(1))
2023-05-03T17:45:57 Notice opnsense /usr/local/etc/rc.routing_configure: plugins_configure monitor (1)
2023-05-03T17:45:57 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: keeping current inet default gateway 'IP'
2023-05-03T17:45:57 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: setting IPv4 default route to IP
2023-05-03T17:45:57 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: IPv4 default gateway set to wan
2023-05-03T17:45:57 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: entering configure using defaults
2023-05-03T17:45:55 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure bootup (execute task : unbound_configure_do(1))
2023-05-03T17:45:55 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure bootup (execute task : opendns_configure_do(1))
2023-05-03T17:45:55 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure bootup (execute task : ntpd_configure_do(1))
2023-05-03T17:45:55 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure bootup (1)
2023-05-03T17:45:55 Notice opnsense /usr/local/etc/rc.bootup: Resyncing OpenVPN instances.
2023-05-03T17:45:55 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure vpn (execute task : openvpn_configure_do(1))
2023-05-03T17:45:53 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure vpn (execute task : ipsec_configure_do(1))
2023-05-03T17:45:53 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure vpn (1)
2023-05-03T17:45:52 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure monitor (execute task : dpinger_configure_do(1,null,1))
2023-05-03T17:45:52 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure monitor (1,null,1)
2023-05-03T17:45:52 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dns (execute task : unbound_configure_do(1))
2023-05-03T17:45:52 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dns (execute task : dnsmasq_configure_do(1))
2023-05-03T17:45:52 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dns (1)
2023-05-03T17:45:51 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dhcrelay (execute task : dhcpd_dhcrelay_configure(1))
2023-05-03T17:45:51 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dhcrelay (1)
2023-05-03T17:45:51 Error opnsense /usr/local/etc/rc.bootup: The command '/bin/kill -'TERM' '75365'' returned exit code '1', the output was 'kill: 75365: No such process'
2023-05-03T17:45:51 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure(1))
2023-05-03T17:45:51 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dhcp (1)
2023-05-03T17:45:51 Notice opnsense /usr/local/etc/rc.bootup: ROUTING: setting inet default route to IP
2023-05-03T17:45:51 Notice opnsense /usr/local/etc/rc.bootup: ROUTING: setting IPv4 default route to IP
2023-05-03T17:45:51 Notice opnsense /usr/local/etc/rc.bootup: ROUTING: IPv4 default gateway set to wan
2023-05-03T17:45:50 Notice opnsense /usr/local/etc/rc.bootup: ROUTING: entering configure using defaults
2023-05-03T17:45:49 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure early (execute task : webgui_configure_do(1))
2023-05-03T17:45:49 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure early (execute task : unbound_cache_flush(1))
2023-05-03T17:45:49 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure early (execute task : openssh_configure_do(1))
2023-05-03T17:45:49 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure early (1)
2023-05-03T17:45:47 Error opnsense /usr/local/etc/rc.bootup: Unable to configure nonexistent interface opt1 (wg1)
2023-05-03T17:45:43 Notice syslog-ng syslog-ng starting up; version='3.38.1'
2023-05-03T17:44:21 Notice syslog-ng syslog-ng shutting down; version='3.38.1'

• Removed the plugin from the system - same issue and behavior seen, route after reboot dissapeared from the route table.

Code Select Expand

2023-05-03T17:53:56 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : unbound_configure_do())
2023-05-03T17:53:56 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : dnsmasq_configure_do())
2023-05-03T17:53:56 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns ()
2023-05-03T17:53:56 Error opnsense /usr/local/etc/rc.linkup: The command '/bin/kill -'TERM' '75365'' returned exit code '1', the output was 'kill: 75365: No such process'
2023-05-03T17:53:54 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2023-05-03T17:53:54 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp ()
2023-05-03T17:53:54 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (execute task : ipsec_configure_do(,opt5))
2023-05-03T17:53:54 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (,opt5)
2023-05-03T17:53:54 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: skipping IPv4 default route
2023-05-03T17:53:54 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to wan
2023-05-03T17:53:53 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: entering configure using 'opt5'
2023-05-03T17:53:49 Notice opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for opt5(vlan099)
2023-05-03T17:53:39 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : unbound_configure_do())
2023-05-03T17:53:39 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns (execute task : dnsmasq_configure_do())
2023-05-03T17:53:39 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dns ()
2023-05-03T17:53:39 Error opnsense /usr/local/etc/rc.linkup: The command '/bin/kill -'TERM' '75365'' returned exit code '1', the output was 'kill: 75365: No such process'
2023-05-03T17:53:39 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure())
2023-05-03T17:53:39 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure dhcp ()
2023-05-03T17:53:39 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (execute task : ipsec_configure_do(,opt6))
2023-05-03T17:53:39 Notice opnsense /usr/local/etc/rc.linkup: plugins_configure ipsec (,opt6)
2023-05-03T17:53:39 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: skipping IPv4 default route
2023-05-03T17:53:39 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: IPv4 default gateway set to wan
2023-05-03T17:53:39 Notice opnsense /usr/local/etc/rc.linkup: ROUTING: entering configure using 'opt6'
2023-05-03T17:53:38 Notice opnsense /usr/local/etc/rc.linkup: DEVD: Ethernet attached event for opt6(vlan0101)
2023-05-03T17:53:38 Notice opnsense /usr/local/etc/rc.routing_configure: plugins_configure monitor (execute task : dpinger_configure_do(1))
2023-05-03T17:53:38 Notice opnsense /usr/local/etc/rc.routing_configure: plugins_configure monitor (1)
2023-05-03T17:53:38 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: keeping current inet default gateway 'IP'
2023-05-03T17:53:38 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: setting IPv4 default route to IP
2023-05-03T17:53:38 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: IPv4 default gateway set to wan
2023-05-03T17:53:38 Notice opnsense /usr/local/etc/rc.routing_configure: ROUTING: entering configure using defaults
2023-05-03T17:53:37 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure bootup (execute task : unbound_configure_do(1))
2023-05-03T17:53:37 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure bootup (execute task : opendns_configure_do(1))
2023-05-03T17:53:37 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure bootup (execute task : ntpd_configure_do(1))
2023-05-03T17:53:37 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure bootup (1)
2023-05-03T17:53:37 Notice opnsense /usr/local/etc/rc.bootup: Resyncing OpenVPN instances.
2023-05-03T17:53:37 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure vpn (execute task : openvpn_configure_do(1))
2023-05-03T17:53:34 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure vpn (execute task : ipsec_configure_do(1))
2023-05-03T17:53:34 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure vpn (1)
2023-05-03T17:53:33 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure monitor (execute task : dpinger_configure_do(1,null,1))
2023-05-03T17:53:33 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure monitor (1,null,1)
2023-05-03T17:53:33 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dns (execute task : unbound_configure_do(1))
2023-05-03T17:53:33 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dns (execute task : dnsmasq_configure_do(1))
2023-05-03T17:53:33 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dns (1)
2023-05-03T17:53:32 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dhcrelay (execute task : dhcpd_dhcrelay_configure(1))
2023-05-03T17:53:32 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dhcrelay (1)
2023-05-03T17:53:32 Error opnsense /usr/local/etc/rc.bootup: The command '/bin/kill -'TERM' '75365'' returned exit code '1', the output was 'kill: 75365: No such process'
2023-05-03T17:53:32 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dhcp (execute task : dhcpd_dhcp_configure(1))
2023-05-03T17:53:32 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure dhcp (1)
2023-05-03T17:53:32 Notice opnsense /usr/local/etc/rc.bootup: ROUTING: setting inet default route to IP
2023-05-03T17:53:32 Notice opnsense /usr/local/etc/rc.bootup: ROUTING: setting IPv4 default route to IP
2023-05-03T17:53:32 Notice opnsense /usr/local/etc/rc.bootup: ROUTING: IPv4 default gateway set to wan
2023-05-03T17:53:31 Notice opnsense /usr/local/etc/rc.bootup: ROUTING: entering configure using defaults
2023-05-03T17:53:30 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure early (execute task : webgui_configure_do(1))
2023-05-03T17:53:30 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure early (execute task : unbound_cache_flush(1))
2023-05-03T17:53:30 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure early (execute task : openssh_configure_do(1))
2023-05-03T17:53:30 Notice opnsense /usr/local/etc/rc.bootup: plugins_configure early (1)
2023-05-03T17:53:28 Error opnsense /usr/local/etc/rc.bootup: Unable to configure nonexistent interface opt1 (wg1)
2023-05-03T17:53:24 Notice syslog-ng syslog-ng starting up; version='3.38.1'
2023-05-03T17:52:05 Notice syslog-ng syslog-ng shutting down; version='3.38.1'

I didnt btw remove the Wireguard related config from FW rules or Interface.

As you can see no change with or without Wireguard.

Regards,
S.

[Seimus]

So I updated my OPNsense to latest 23.1.9 and I can still see the issue present.

I had another take to look around and it seems that the route will as well vanish even with port Flap.

I did this:
- Disconnected LAN interface
- Static route was removed
- Reconnected LAN interface
- Static route was not reinstalled


So this is very wierd, I could understand why the route was removed when the Egress interface is taken down.

But why the route is not reinstalled back once the interface comes UP?
How from point of OPNsense the OPNsense is installing the routes into the route table? Is it a cron job?


Either there is some BUG or my APU2 is playing me. If anybody has some idea about the questions above let me know.

In a week or so I should recieve new HW for OPNsense, I will then deploy the FW from scratch from a new install. This should show if the problem is something cooked in my device or its the OS itself (Personaly I have the feel my APU2 is playing me)

Regards,
S.

[Seimus]

So good news my issue was fixed. And it was a configuration issue I made.

I recieved a new H/W A TopTon N5105, did a fresh install and migrated config from APU2 to N5105 and the issue was still present.

However then I went back to the basics and checked how I did configure the route/GW. As it looks like the problem was in the GW configuration for the specific route, back in the past I was using a single LAN interface and after a while changed to LAGG and then to L3 VLAN running over LAGG (where LAGG remained as L2) but I forgot to change the egress interface from LAGG to the specific VLAN where the GW for the route is present.

After changing the GW interface from LAGG to VLAN now the route is installed and now it works as should


Regards,
S.

[franco]

Glad to hear.


Cheers,
Franco