OPNsense Forum
Archive => 22.1 Legacy Series => Topic started by: 5SpeedFun on February 02, 2022, 05:01:01 am
-
This has always been normal at startup. I have to go into interfaces -> Other Types -> Gift -> Edit/Save and it comes up immediately.
However on the 22.1 upgrade, I couldn't get it to come up even then. This is my 6in4 tunnel to HE.net
Anyone know if this is expected or if something else has changed in 22.1 where it may not come up? The only thing I see in the release notes is:
"interfaces: align GIF configuration with base system options".
This doesn't have any practical meaning to me. Am I supposed to be configuring it differently?
-
Well, it sounds like some interface rework in 22.1 could be the cause of this, but I suspect it also means your WAN comes up later than expected leaving the tunnel deactivated when it is first attempted to bring it up. As far as I understand such timing issues could have been in 21.7 as well but not trigger that easily.
Can you share the system log? It should have a message or two relevant to the attempts to bring up GIF and failing.
Thanks,
Franco
-
The gif tunnel never came up in older versions on boot. I've posted about similar issues IIRC on github, and was told to tie services to a loopback interfaces, however I don't think that's possible when assigning a public IP I need on a specific interface.
In any case, after boot I could click "edit" and just click "save" and that would cause it come up, but that is not the case now. In any case I've rolled back to 21.7.8. I won't be able to look at this again for a few days and at that point I'll snapshot the vm this is running on, upgrade, and then I can pull logs.
Edit: Re-read your post and see i can pull this from system logs.
Here is a log (21.7.8) which doesn't come up at boot:
/usr/local/etc/rc.bootup: The command '/sbin/ifconfig 'gif0' tunnel '' '184.105.253.14'' returned exit code '1', the output was 'ifconfig: error in parsing address string: Name does not resolve'
This is odd as the gif config doesn't have any hostnames in it as you can see above.
Thanks! I appreciate all your work on OPNSense. Overall I have no regrets moving to OPNSense from Pfsense & the developers should be proud of all the work that was put into it.
-
The issue seems to be here:
https://github.com/opnsense/core/blob/master/src/etc/inc/interfaces.inc#L795
It's not returning an address, probably because the one it's looking for is set via DHCP later.
Can you share your GIF config?
In theory it should rerun the tunnel when DHCP kicks in. What is the IPv4 mode of the interface the GIF is running on?
Cheers,
Franco
-
I have a static /29, so I'm using an address from that static block. The actual interface is sfxge0_vlan10, -- the ip isn't just on a physical interface, it's on the vlan interface. I wonder if that matters?
Parent interface is my internet connection interface as mentioned above.
Remote ipv4 address (also static as it's he.net)
And then the 2 peer ip addresses for each side of the tunnel, also both static. Nothing complicated.
If the actual addresses matter let me know.
-
VLANs are created before GIFs and if they do have a static address then that is added immediately at least as long as it doesn't have any complications like DHCPv6 tracking or bridging.
Can you provide the output of
# /usr/local/etc/rc.reload_all
which shows us the configuration order. I suspect that GIF is created before sfxge0_vlan10 assigned interface which could cause this to happen.
Also sfxge0_vlan10 is assigned as an interface with a static configuration or do you use a VIP on it? What exactly is selected on GIF parent interface drop down? I don't need an address if one is in there, just the type of string that's in there.
Cheers,
Franco
-
Unfortunately I have both DHCP6-PD tracking on this interface, as well as VIP or two as well.
/usr/local/etc/rc.reload_all
Writing firmware setting...done.
Writing trust files...done.
Configuring login behaviour...done.
Configuring CRON...done.
Setting timezone...done.
Setting hostname: edge01.xxxxxxxx.net
Generating /etc/hosts...done.
Generating /etc/resolv.conf...done.
Configuring loopback interface...done.
Creating wireless clone interfaces...done.
Configuring VLAN interfaces...done.
Configuring Loopback interfaces...Creating OpenVPN instances...done.
Configuring DMZ_Zimbra interface...done.
Configuring DMZ_mail interface...done.
Configuring DMZ_pihole interface...done.
Configuring DMZ_plex interface...done.
Configuring DMZ_www interface...done.
Configuring LAN interface...done.
Configuring LAN_NET_MGMT interface...done.
Configuring Lo1 interface...done.
Configuring Lo2 interface...done.
Configuring Lo3 interface...done.
Configuring TRANSIT interface...done.
Configuring WG_xxxxx interface...done.
Configuring WG_Josh interface...done.
Configuring WG_Parents interface...done.
Configuring GIF interfaces (1)...done.
Configuring GIF interfaces (2)...done.
Configuring HE_Chicago interface...done.
Configuring LAN_Xbox interface...done.
Configuring InternetBiz interface...done.
Creating IPsec VTI instances...done.
Setting up routes...done.
Configuring firewall........done.
Starting DHCPv4 service...done.
Starting router advertisement service...done.
Starting NTP service...done.
Configuring OpenSSH...done.
Starting Unbound DNS...done.
Starting web GUI...done.
Syncing OpenVPN settings...done.
Generating RRD graphs...done.
Stopping named.
Waiting for PIDS: 59873.
Stopping php_fpm.
Waiting for PIDS: 90780.
Stopping zebra.
Waiting for PIDS: 51609.
[#] rm -f /var/run/wireguard/wg0.sock
[#] rm -f /var/run/wireguard/wg1.sock
[#] rm -f /var/run/wireguard/wg2.sock
[#] ifconfig wg create name wg0
[!] Missing WireGuard kernel support (ifconfig: SIOCIFCREATE2: Invalid argument). Falling back to slow userspace implementation.
[#] wireguard-go wg0
┌──────────────────────────────────────────────────────┐
│ │
│ Running wireguard-go is not required because this │
│ kernel has first class support for WireGuard. For │
│ information on installing the kernel module, │
│ please visit: │
│ https://www.wireguard.com/install/ │
│ │
└──────────────────────────────────────────────────────┘
[#] wg setconf wg0 /dev/stdin
[#] ifconfig wg0 inet 192.168.30.1/27 alias
[#] ifconfig wg0 inet6 xxxx:xxxx:xxxx:f1c9::1/64 alias
[#] ifconfig wg0 mtu 1420
[#] ifconfig wg0 up
[#] route -q -n add -inet 192.168.30.7/32 -interface wg0
[#] route -q -n add -inet 192.168.30.6/32 -interface wg0
[#] route -q -n add -inet 192.168.30.5/32 -interface wg0
[#] route -q -n add -inet 192.168.30.4/32 -interface wg0
[#] route -q -n add -inet 192.168.30.3/32 -interface wg0
[#] route -q -n add -inet 192.168.30.2/32 -interface wg0
[+] Backgrounding route monitor
[#] ifconfig wg create name wg1
[!] Missing WireGuard kernel support (ifconfig: SIOCIFCREATE2: Invalid argument). Falling back to slow userspace implementation.
[#] wireguard-go wg1
┌──────────────────────────────────────────────────────┐
│ │
│ Running wireguard-go is not required because this │
│ kernel has first class support for WireGuard. For │
│ information on installing the kernel module, │
│ please visit: │
│ https://www.wireguard.com/install/ │
│ │
└──────────────────────────────────────────────────────┘
[#] wg setconf wg1 /dev/stdin
[#] ifconfig wg1 inet 192.168.30.33/29 alias
[#] ifconfig wg1 inet6 xxxx:xxxx:xxxx:ac09::1/64 alias
[#] ifconfig wg1 mtu 1420
[#] ifconfig wg1 up
[#] route -q -n add -inet6 xxxx:xxxx:xxxx::c01a::35/128 -interface wg1
[#] route -q -n add -inet 192.168.30.35/32 -interface wg1
[#] route -q -n add -inet 192.168.30.34/32 -interface wg1
[+] Backgrounding route monitor
[#] ifconfig wg create name wg2
[!] Missing WireGuard kernel support (ifconfig: SIOCIFCREATE2: Invalid argument). Falling back to slow userspace implementation.
[#] wireguard-go wg2
┌──────────────────────────────────────────────────────┐
│ │
│ Running wireguard-go is not required because this │
│ kernel has first class support for WireGuard. For │
│ information on installing the kernel module, │
│ please visit: │
│ https://www.wireguard.com/install/ │
│ │
└──────────────────────────────────────────────────────┘
[#] wg setconf wg2 /dev/stdin
[#] ifconfig wg2 inet 192.168.30.41/29 alias
[#] ifconfig wg2 inet6 xxxx:xxxx:xxxx:xxxx::1/64 alias
[#] ifconfig wg2 mtu 1420
[#] ifconfig wg2 up
[#] route -q -n add -inet 192.168.30.42/32 -interface wg2
[+] Backgrounding route monitor
Checking zebra.conf
2022/02/03 08:57:33 ZEBRA: [EC 4043309111] Disabling MPLS support (no kernel support)
OK
Starting zebra.
2022/02/03 08:57:33 ZEBRA: [EC 4043309111] Disabling MPLS support (no kernel support)
Performing sanity check on php-fpm configuration:
[03-Feb-2022 08:57:33] NOTICE: configuration file /usr/local/etc/php-fpm.conf test is successful
Starting php_fpm.
setup sfxge0_vlan10 [egress only]
setup sfxge0_vlan100
setup sfxge0_vlan99
setup sfxge0_vlan120
setup lo1
ngctl: send msg: No such file or directory
error lo1: cannot create netflow node for lo1
setup wg0
ngctl: send msg: No such file or directory
error wg0: cannot create netflow node for wg0
Starting named.
-
Ok the good news is I understand the problem. The other good news is I always wanted to clean this up, but then the bad news is this is rather complex and might break things while testing it. Would you be up for it?
Cheers,
Franco
-
I'd love to help!
You guys in the OPNSense project have done so much for the community, this is the least I could do!
This is all in a VM with a pass through nic so it's easy for me to snapshot, try things & roll back if necessary.
Just let me know what you need me to do/try.
Thanks!
Rob
-
Hi Rob,
Great, I added a feature ticket here for reference https://github.com/opnsense/core/issues/5540 and will report back in a bit.
Cheers,
Franco
-
As a HE.net IPv6 tunnelbroker user myself, currently doing a P-o-C on OPNsense 22.1, this thread got my attention.
I've got to say, how impressed I am, that it so quickly led to a code feature request being opened.
Thumbs up !
-
Ok, so..... I've worked through most of the boot sequence once now and here is the plan:
Since the changes are large there is no use throwing opnsense-patch commands around so we are going to ask anyone willing to try the development release bundled with the upcoming 22.1.1 first.
This is to ensure we haven't made broad cleanup errors and is not specifically addressing the boot order problem just yet. The plan with the current code rework is to simplify things for the next step which is reordering according to interface requirement chain. It's going to be a lot easier with all the side effects and failsafe code removed.
I'll follow up again once 22.1.1 is out next week.
Cheers,
Franco
-
Excellent news! When it's ready let me know and I will go from 21.7.8 -> 22.1 -> 22.1.1 and do some testing.
Thanks so much!
Robert
-
Small report. We are switching to plan B. :)
22.1.1 development version should fix the problem class reported by Robert. While the change was conceptually simple the challenge was to "record" which devices (scope is GIF, GRE and bridges for now) need to be configured and only configure them once when that becomes possible. I think prior to this the code would try to start things every time it thought it needed to resolve a problem proactively and the boot sequence differed from the full reconfiguration cycle after boot (rc.reload_all script via console option 11) in that regard which didn't help with boot-bound issues.
GUI-only cleanups related to this change will already be available with the community version of 22.1.1 to reduce code differences and are easier to test than the actual reordering changes in said development release.
All things considered this is good news for other network device types (VLAN, LAGG, OpenVPN, IPsec etc.) which can benefit from this and eventually integration of new network types should be possible via standalone plugins. It will also help with MVC migration of these network devices GUI-wise and move along QinQ support which is currently planned for 22.7.
22.1.1 is scheduled for next week as mentioned earlier.
Cheers,
Franco
-
22.1.1 is out now. No plan C so if you can just try the bundled development version if tunnels come up correctly after boot. Looks promising from our end.
Cheers,
Franco
-
Hey Thanks Franco!
I've upgraded 21.7.8 -> 22.1 -> 22.1.1 but it's still not coming up at boot.
(I waited for uptime of 3 minutes post boot - still not up).
In the past (21.7.x), I waited for system to boot, edit interfaces -> other types -> Gif -> edit -> save (came up)
in 22.1, that method didn't work, so I rolled back & reporterd the issue.
Now it won't come up at boot, but if I start the dpinger via the webui, it does come up. HOwever my ipv6 connectivity is broken. While I have an address it seems it is not routed correctly. Test-ipv6 returns no working ipv6 and ping ipv6.google.com doesn't work either.
-
Hi,
Well just to make sure this wasn't missed: you changed to the development flavour after upgrading to 22.1.1 and checked and installed updates again and then did the reboot to confirm?
Cheers,
Franco
-
WHoops, I missed that. I'll give that a shot tonight. I did not do that part. Apologies for missing that.
-
After switching to development branch & upgrading after 22.1.1, the he.net tunnel did come up on boot! Great job! However in system->firmware->reporter:
An issue was detected.
Would you like to submit this crash report to the developers?
System Information:
User-Agent Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.80 Safari/537.36
FreeBSD 13.0-STABLE stable/22.1-n248057-239b52c9023 SMP amd64
OPNsense 22.7.a_83 789b777d4
Plugins os-bind-1.21 os-frr-1.27 os-realtek-re-1.0 os-upnp-1.4_2 os-wireguard-1.10
Time Fri, 18 Feb 2022 00:42:37 -0600
OpenSSL 1.1.1m 14 Dec 2021
PHP 7.4.27
PHP Errors:
[18-Feb-2022 00:41:29 America/Chicago] PHP Warning: Illegal string offset 'function' in /usr/local/etc/inc/interfaces.inc on line 769
[18-Feb-2022 00:41:29 America/Chicago] PHP Warning: call_user_func_array() expects parameter 1 to be a valid callback, function 'i' not found or invalid function name in /usr/local/etc/inc/interfaces.inc on line 769
[18-Feb-2022 00:41:29 America/Chicago] PHP Warning: Illegal string offset 'name' in /usr/local/etc/inc/interfaces.inc on line 769
dmesg.boot
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 100000000 Hz quality 950
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x608-0x60b on acpi0
pcib0: port 0xcf8-0xcff on acpi0
pci0: on pcib0
isab0: at device 1.0 on pci0
isa0: on isab0
atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xe320-0xe32f at device 1.1 on pci0
ata0: at channel 0 on atapci0
ata1: at channel 1 on atapci0
uhci0: port 0xe300-0xe31f irq 11 at device 1.2 on pci0
usbus0 on uhci0
usbus0: 12Mbps Full Speed USB v1.0
pci0: at device 1.3 (no driver attached)
vgapci0: mem 0xfd000000-0xfdffffff,0xfc510000-0xfc510fff at device 2.0 on pci0
vgapci0: Boot video device
virtio_pci0: port 0xe280-0xe2bf mem 0xfe400000-0xfe403fff irq 11 at device 3.0 on pci0
vtballoon0: on virtio_pci0
virtio_pci1: port 0xe200-0xe27f mem 0xfc511000-0xfc511fff,0xfe404000-0xfe407fff irq 10 at device 10.0 on pci0
vtblk0: on virtio_pci1
vtblk0: 122880MB (251658240 512 byte sectors)
sfxge0: port 0xe000-0xe0ff mem 0xfa000000-0xfaffffff,0xfc4f0000-0xfc4fffff irq 10 at device 17.0 on pci0
sfxge0: Using MSI-X interrupts
sfxge0: Ethernet address: 00:0f:53:0e:20:d0
sfxge0: Solarstorm SFN5162F SFP+ Server Adapter
sfxge1: port 0xe100-0xe1ff mem 0xfb000000-0xfbffffff,0xfc500000-0xfc50ffff irq 10 at device 17.1 on pci0
sfxge1: Using MSI-X interrupts
sfxge1: Ethernet address: 00:0f:53:0e:20:d1
sfxge1: Solarstorm SFN5162F SFP+ Server Adapter
em0: port 0xe2c0-0xe2ff mem 0xfc4c0000-0xfc4dffff irq 10 at device 18.0 on pci0
em0: Using 1024 TX descriptors and 1024 RX descriptors
em0: Ethernet address: e2:34:ac:9e:35:68
em0: netmap queues/slots: TX 1/1024, RX 1/1024
pcib1: mem 0xfc512000-0xfc5120ff irq 10 at device 30.0 on pci0
pci1: on pcib1
pcib2: mem 0xfc513000-0xfc5130ff irq 11 at device 31.0 on pci0
pci2: on pcib2
acpi_syscontainer0: on acpi0
vmgenc0: on acpi0
acpi_syscontainer1: port 0xaf00-0xaf0b on acpi0
acpi_syscontainer2: port 0xafe0-0xafe3 on acpi0
acpi_syscontainer3: port 0xae00-0xae17 on acpi0
atkbdc0: port 0x60,0x64 irq 1 on acpi0
atkbd0: irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
WARNING: Device "psm" is Giant locked and may be deleted before FreeBSD 14.0.
psm0: model IntelliMouse Explorer, device ID 4
fdc0: port 0x3f2-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: does not respond
device_attach: fdc0 attach returned 6
orm0: at iomem 0xea000-0xeffff pnpid ORM0000 on isa0
vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff pnpid PNP0900 on isa0
attimer0: at port 0x40 on isa0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
fdc0: No FDOUT register!
Timecounters tick every 10.000 msec
Trying to mount root from ufs:/dev/ufs/OPNsense [rw]...
ugen0.1: at usbus0
uhub0 on usbus0
uhub0: on usbus0
cd0 at ata1 bus 0 scbus1 target 0 lun 0
cd0: Removable CD-ROM SCSI device
cd0: Serial Number QM00003
cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
uhub0: 2 ports with 2 removable, self powered
ugen0.2: at usbus0
em0: link state changed to UP
debugnet_any_ifnet_update: Bad dn_init result from em0 (ifp 0xfffff80003d52000), ignoring.
Waiting (max 60 seconds) for system process `vnlru' to stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining... 0 0 done
Waiting (max 60 seconds) for system thread `bufdaemon' to stop... done
Waiting (max 60 seconds) for system thread `bufspacedaemon-0' to stop... done
Waiting (max 60 seconds) for system thread `bufspacedaemon-1' to stop... done
Waiting (max 60 seconds) for system thread `bufspacedaemon-2' to stop... done
Waiting (max 60 seconds) for system thread `bufspacedaemon-3' to stop... done
All buffers synced.
Uptime: 36s
uhub0: detached
Rebooting...
cpu_reset: Stopping other CPUs
---<>---
Copyright (c) 1992-2021 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.0-STABLE hyperv-n248055-e91d90c0ac0 SMP amd64
FreeBSD clang version 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
VT(vga): text 80x25
CPU: AMD Ryzen 9 3900X 12-Core Processor (3800.14-MHz K8-class CPU)
Origin="AuthenticAMD" Id=0x870f10 Family=0x17 Model=0x71 Stepping=0
Features=0x1783fbff
Features2=0xfff83203
AMD Features=0x2e500800
AMD Features2=0x8003f7
Structured Extended Features=0x219c01ab
Structured Extended Features2=0x400004
Structured Extended Features3=0xa8000000
XSAVE Features=0xf
IA32_ARCH_CAPS=0x69
AMD Extended Feature Extensions ID EBX=0x3009205
SVM: NP,NRIP,NAsids=16
Hypervisor: Origin = "KVMKVMKVM"
real memory = 8589934592 (8192 MB)
avail memory = 8272318464 (7889 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table:
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
ioapic0 irqs 0-23
Launching APs: 1 2 3
wlan: mac acl policy registered
random: entropy device external interface
kbd1 at kbdmux0
WARNING: Device "spkr" is Giant locked and may be deleted before FreeBSD 14.0.
vtvga0:
kvmclock0:
Timecounter "kvmclock" frequency 1000000000 Hz quality 975
kvmclock0: registered as a time-of-day clock, resolution 0.000001s
aesni0:
acpi0:
acpi0: Power Button (fixed)
cpu0: on acpi0
atrtc0: port 0x70-0x77 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 100000000 Hz quality 950
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x608-0x60b on acpi0
pcib0: port 0xcf8-0xcff on acpi0
pci0: on pcib0
isab0: at device 1.0 on pci0
isa0: on isab0
atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xe320-0xe32f at device 1.1 on pci0
ata0: at channel 0 on atapci0
ata1: at channel 1 on atapci0
uhci0: port 0xe300-0xe31f irq 11 at device 1.2 on pci0
usbus0 on uhci0
usbus0: 12Mbps Full Speed USB v1.0
pci0: at device 1.3 (no driver attached)
vgapci0: mem 0xfd000000-0xfdffffff,0xfc510000-0xfc510fff at device 2.0 on pci0
vgapci0: Boot video device
virtio_pci0: port 0xe280-0xe2bf mem 0xfe400000-0xfe403fff irq 11 at device 3.0 on pci0
vtballoon0: on virtio_pci0
virtio_pci1: port 0xe200-0xe27f mem 0xfc511000-0xfc511fff,0xfe404000-0xfe407fff irq 10 at device 10.0 on pci0
vtblk0: on virtio_pci1
vtblk0: 122880MB (251658240 512 byte sectors)
sfxge0: port 0xe000-0xe0ff mem 0xfa000000-0xfaffffff,0xfc4f0000-0xfc4fffff irq 10 at device 17.0 on pci0
sfxge0: Using MSI-X interrupts
sfxge0: Ethernet address: 00:0f:53:0e:20:d0
sfxge0: Solarstorm SFN5162F SFP+ Server Adapter
sfxge1: port 0xe100-0xe1ff mem 0xfb000000-0xfbffffff,0xfc500000-0xfc50ffff irq 10 at device 17.1 on pci0
sfxge1: Using MSI-X interrupts
sfxge1: Ethernet address: 00:0f:53:0e:20:d1
sfxge1: Solarstorm SFN5162F SFP+ Server Adapter
em0: port 0xe2c0-0xe2ff mem 0xfc4c0000-0xfc4dffff irq 10 at device 18.0 on pci0
em0: Using 1024 TX descriptors and 1024 RX descriptors
em0: Ethernet address: e2:34:ac:9e:35:68
em0: netmap queues/slots: TX 1/1024, RX 1/1024
pcib1: mem 0xfc512000-0xfc5120ff irq 10 at device 30.0 on pci0
pci1: on pcib1
pcib2: mem 0xfc513000-0xfc5130ff irq 11 at device 31.0 on pci0
pci2: on pcib2
acpi_syscontainer0: on acpi0
vmgenc0: on acpi0
acpi_syscontainer1: port 0xaf00-0xaf0b on acpi0
acpi_syscontainer2: port 0xafe0-0xafe3 on acpi0
acpi_syscontainer3: port 0xae00-0xae17 on acpi0
atkbdc0: port 0x60,0x64 irq 1 on acpi0
atkbd0: irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
WARNING: Device "psm" is Giant locked and may be deleted before FreeBSD 14.0.
psm0: model IntelliMouse Explorer, device ID 4
fdc0: port 0x3f2-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: does not respond
device_attach: fdc0 attach returned 6
orm0: at iomem 0xea000-0xeffff pnpid ORM0000 on isa0
vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff pnpid PNP0900 on isa0
attimer0: at port 0x40 on isa0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
fdc0: No FDOUT register!
Timecounters tick every 10.000 msec
Trying to mount root from ufs:/dev/ufs/OPNsense [rw]...
ugen0.1: at usbus0
uhub0 on usbus0
uhub0: on usbus0
cd0 at ata1 bus 0 scbus1 target 0 lun 0
cd0: Removable CD-ROM SCSI device
cd0: Serial Number QM00003
cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
uhub0: 2 ports with 2 removable, self powered
ugen0.2: at usbus0
em0: link state changed to UP
debugnet_any_ifnet_update: Bad dn_init result from em0 (ifp 0xfffff80003cd9000), ignoring.
intsmb0: irq 9 at device 1.3 on pci0
intsmb0: intr IRQ 9 enabled revision 0
smbus0: on intsmb0
uhid0 on uhub0
uhid0: on usbus0
lo0: link state changed to UP
sfxge0: link state changed to UP
vlan0: changing name to 'sfxge0_vlan100'
vlan1: changing name to 'sfxge0_vlan99'
vlan2: changing name to 'sfxge0_vlan90'
vlan3: changing name to 'sfxge0_vlan10'
vlan4: changing name to 'sfxge0_vlan120'
vlan5: changing name to 'sfxge0_vlan11'
vlan6: changing name to 'sfxge0_vlan12'
vlan7: changing name to 'sfxge0_vlan13'
vlan8: changing name to 'sfxge0_vlan14'
vlan9: changing name to 'sfxge0_vlan101'
vlan10: changing name to 'sfxge0_vlan15'
lo1: link state changed to UP
lo2: link state changed to UP
lo3: link state changed to UP
gif0: link state changed to DOWN
WARNING: attempt to domain_add(netgraph) after domainfinalize()
tun0: link state changed to UP
tun0: changing name to 'wg0'
tun1: link state changed to UP
tun1: changing name to 'wg1'
tun2: link state changed to UP
tun2: changing name to 'wg2'
wg0: link state changed to DOWN
wg1: link state changed to DOWN
wg2: link state changed to DOWN
Waiting (max 60 seconds) for system process `vnlru' to stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining... 51 0 0 done
Waiting (max 60 seconds) for system thread `bufdaemon' to stop... done
Waiting (max 60 seconds) for system thread `bufspacedaemon-0' to stop... done
Waiting (max 60 seconds) for system thread `bufspacedaemon-2' to stop... done
Waiting (max 60 seconds) for system thread `bufspacedaemon-1' to stop... done
Waiting (max 60 seconds) for system thread `bufspacedaemon-3' to stop... done
All buffers synced.
Uptime: 3m48s
uhid0: detached
uhub0: detached
Rebooting...
cpu_reset: Stopping other CPUs
---<>---
Copyright (c) 1992-2021 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.0-STABLE stable/22.1-n248057-239b52c9023 SMP amd64
FreeBSD clang version 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
VT(vga): text 80x25
CPU: AMD Ryzen 9 3900X 12-Core Processor (3800.09-MHz K8-class CPU)
Origin="AuthenticAMD" Id=0x870f10 Family=0x17 Model=0x71 Stepping=0
Features=0x1783fbff
Features2=0xfff83203
AMD Features=0x2e500800
AMD Features2=0x8003f7
Structured Extended Features=0x219c01ab
Structured Extended Features2=0x400004
Structured Extended Features3=0xa8000000
XSAVE Features=0xf
IA32_ARCH_CAPS=0x69
AMD Extended Feature Extensions ID EBX=0x3009205
SVM: NP,NRIP,NAsids=16
Hypervisor: Origin = "KVMKVMKVM"
real memory = 8589934592 (8192 MB)
avail memory = 8272314368 (7889 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table:
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
ioapic0 irqs 0-23
Launching APs: 3 1 2
wlan: mac acl policy registered
random: entropy device external interface
kbd1 at kbdmux0
WARNING: Device "spkr" is Giant locked and may be deleted before FreeBSD 14.0.
vtvga0:
kvmclock0:
Timecounter "kvmclock" frequency 1000000000 Hz quality 975
kvmclock0: registered as a time-of-day clock, resolution 0.000001s
aesni0:
acpi0:
acpi0: Power Button (fixed)
cpu0: on acpi0
atrtc0: port 0x70-0x77 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 100000000 Hz quality 950
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x608-0x60b on acpi0
pcib0: port 0xcf8-0xcff on acpi0
pci0: on pcib0
isab0: at device 1.0 on pci0
isa0: on isab0
atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xe320-0xe32f at device 1.1 on pci0
ata0: at channel 0 on atapci0
ata1: at channel 1 on atapci0
uhci0: port 0xe300-0xe31f irq 11 at device 1.2 on pci0
usbus0 on uhci0
usbus0: 12Mbps Full Speed USB v1.0
pci0: at device 1.3 (no driver attached)
vgapci0: mem 0xfd000000-0xfdffffff,0xfc510000-0xfc510fff at device 2.0 on pci0
vgapci0: Boot video device
virtio_pci0: port 0xe280-0xe2bf mem 0xfe400000-0xfe403fff irq 11 at device 3.0 on pci0
vtballoon0: on virtio_pci0
virtio_pci1: port 0xe200-0xe27f mem 0xfc511000-0xfc511fff,0xfe404000-0xfe407fff irq 10 at device 10.0 on pci0
vtblk0: on virtio_pci1
vtblk0: 122880MB (251658240 512 byte sectors)
sfxge0: port 0xe000-0xe0ff mem 0xfa000000-0xfaffffff,0xfc4f0000-0xfc4fffff irq 10 at device 17.0 on pci0
sfxge0: Using MSI-X interrupts
sfxge0: Ethernet address: 00:0f:53:0e:20:d0
sfxge0: Solarstorm SFN5162F SFP+ Server Adapter
sfxge1: port 0xe100-0xe1ff mem 0xfb000000-0xfbffffff,0xfc500000-0xfc50ffff irq 10 at device 17.1 on pci0
sfxge1: Using MSI-X interrupts
sfxge1: Ethernet address: 00:0f:53:0e:20:d1
sfxge1: Solarstorm SFN5162F SFP+ Server Adapter
em0: port 0xe2c0-0xe2ff mem 0xfc4c0000-0xfc4dffff irq 10 at device 18.0 on pci0
em0: Using 1024 TX descriptors and 1024 RX descriptors
em0: Ethernet address: e2:34:ac:9e:35:68
em0: netmap queues/slots: TX 1/1024, RX 1/1024
pcib1: mem 0xfc512000-0xfc5120ff irq 10 at device 30.0 on pci0
pci1: on pcib1
pcib2: mem 0xfc513000-0xfc5130ff irq 11 at device 31.0 on pci0
pci2: on pcib2
acpi_syscontainer0: on acpi0
vmgenc0: on acpi0
acpi_syscontainer1: port 0xaf00-0xaf0b on acpi0
acpi_syscontainer2: port 0xafe0-0xafe3 on acpi0
acpi_syscontainer3: port 0xae00-0xae17 on acpi0
atkbdc0: port 0x60,0x64 irq 1 on acpi0
atkbd0: irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
WARNING: Device "psm" is Giant locked and may be deleted before FreeBSD 14.0.
psm0: model IntelliMouse Explorer, device ID 4
fdc0: port 0x3f2-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: does not respond
device_attach: fdc0 attach returned 6
orm0: at iomem 0xea000-0xeffff pnpid ORM0000 on isa0
vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff pnpid PNP0900 on isa0
attimer0: at port 0x40 on isa0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
fdc0: No FDOUT register!
Timecounters tick every 10.000 msec
Trying to mount root from ufs:/dev/ufs/OPNsense [rw]...
ugen0.1: at usbus0
uhub0 on usbus0
uhub0: on usbus0
cd0 at ata1 bus 0 scbus1 target 0 lun 0
cd0: Removable CD-ROM SCSI device
cd0: Serial Number QM00003
cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
uhub0: 2 ports with 2 removable, self powered
ugen0.2: at usbus0
em0: link state changed to UP
debugnet_any_ifnet_update: Bad dn_init result from em0 (ifp 0xfffff80003cdd000), ignoring.
intsmb0: irq 9 at device 1.3 on pci0
intsmb0: intr IRQ 9 enabled revision 0
smbus0: on intsmb0
uhid0 on uhub0
uhid0: on usbus0
lo0: link state changed to UP
sfxge0: link state changed to UP
vlan0: changing name to 'sfxge0_vlan100'
vlan1: changing name to 'sfxge0_vlan99'
vlan2: changing name to 'sfxge0_vlan90'
vlan3: changing name to 'sfxge0_vlan10'
vlan4: changing name to 'sfxge0_vlan120'
vlan5: changing name to 'sfxge0_vlan11'
vlan6: changing name to 'sfxge0_vlan12'
vlan7: changing name to 'sfxge0_vlan13'
vlan8: changing name to 'sfxge0_vlan14'
vlan9: changing name to 'sfxge0_vlan101'
vlan10: changing name to 'sfxge0_vlan15'
lo1: link state changed to UP
lo2: link state changed to UP
lo3: link state changed to UP
gif0: link state changed to DOWN
pflog0: permanently promiscuous mode enabled
WARNING: attempt to domain_add(netgraph) after domainfinalize()
tun0: link state changed to UP
tun0: changing name to 'wg0'
tun1: link state changed to UP
tun1: changing name to 'wg1'
tun2: link state changed to UP
tun2: changing name to 'wg2'
wg0: link state changed to DOWN
wg1: link state changed to DOWN
wg2: link state changed to DOWN
Waiting (max 60 seconds) for system process `vnlru' to stop... done
Waiting (max 60 seconds) for system process `syncer' to stop...
Syncing disks, vnodes remaining... 6 0 0 done
Waiting (max 60 seconds) for system thread `bufdaemon' to stop... done
Waiting (max 60 seconds) for system thread `bufspacedaemon-0' to stop... done
Waiting (max 60 seconds) for system thread `bufspacedaemon-2' to stop... done
Waiting (max 60 seconds) for system thread `bufspacedaemon-1' to stop... done
Waiting (max 60 seconds) for system thread `bufspacedaemon-3' to stop... done
All buffers synced.
Uptime: 3m10s
uhid0: detached
uhub0: detached
Rebooting...
cpu_reset: Stopping other CPUs
---<>---
Copyright (c) 1992-2021 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 13.0-STABLE stable/22.1-n248057-239b52c9023 SMP amd64
FreeBSD clang version 13.0.0 (git@github.com:llvm/llvm-project.git llvmorg-13.0.0-0-gd7b669b3a303)
VT(vga): text 80x25
CPU: AMD Ryzen 9 3900X 12-Core Processor (3800.09-MHz K8-class CPU)
Origin="AuthenticAMD" Id=0x870f10 Family=0x17 Model=0x71 Stepping=0
Features=0x1783fbff
Features2=0xfff83203
AMD Features=0x2e500800
AMD Features2=0x8003f7
Structured Extended Features=0x219c01ab
Structured Extended Features2=0x400004
Structured Extended Features3=0xa8000000
XSAVE Features=0xf
IA32_ARCH_CAPS=0x69
AMD Extended Feature Extensions ID EBX=0x3009205
SVM: NP,NRIP,NAsids=16
Hypervisor: Origin = "KVMKVMKVM"
real memory = 8589934592 (8192 MB)
avail memory = 8272314368 (7889 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table:
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 4 core(s)
random: registering fast source Intel Secure Key RNG
random: fast provider: "Intel Secure Key RNG"
random: unblocking device.
ioapic0 irqs 0-23
Launching APs: 1 2 3
wlan: mac acl policy registered
random: entropy device external interface
kbd1 at kbdmux0
WARNING: Device "spkr" is Giant locked and may be deleted before FreeBSD 14.0.
vtvga0:
kvmclock0:
Timecounter "kvmclock" frequency 1000000000 Hz quality 975
kvmclock0: registered as a time-of-day clock, resolution 0.000001s
aesni0:
acpi0:
acpi0: Power Button (fixed)
cpu0: on acpi0
atrtc0: port 0x70-0x77 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
hpet0: iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 100000000 Hz quality 950
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x608-0x60b on acpi0
pcib0: port 0xcf8-0xcff on acpi0
pci0: on pcib0
isab0: at device 1.0 on pci0
isa0: on isab0
atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xe320-0xe32f at device 1.1 on pci0
ata0: at channel 0 on atapci0
ata1: at channel 1 on atapci0
uhci0: port 0xe300-0xe31f irq 11 at device 1.2 on pci0
usbus0 on uhci0
usbus0: 12Mbps Full Speed USB v1.0
pci0: at device 1.3 (no driver attached)
vgapci0: mem 0xfd000000-0xfdffffff,0xfc510000-0xfc510fff at device 2.0 on pci0
vgapci0: Boot video device
virtio_pci0: port 0xe280-0xe2bf mem 0xfe400000-0xfe403fff irq 11 at device 3.0 on pci0
vtballoon0: on virtio_pci0
virtio_pci1: port 0xe200-0xe27f mem 0xfc511000-0xfc511fff,0xfe404000-0xfe407fff irq 10 at device 10.0 on pci0
vtblk0: on virtio_pci1
vtblk0: 122880MB (251658240 512 byte sectors)
sfxge0: port 0xe000-0xe0ff mem 0xfa000000-0xfaffffff,0xfc4f0000-0xfc4fffff irq 10 at device 17.0 on pci0
sfxge0: Using MSI-X interrupts
sfxge0: Ethernet address: 00:0f:53:0e:20:d0
sfxge0: Solarstorm SFN5162F SFP+ Server Adapter
sfxge1: port 0xe100-0xe1ff mem 0xfb000000-0xfbffffff,0xfc500000-0xfc50ffff irq 10 at device 17.1 on pci0
sfxge1: Using MSI-X interrupts
sfxge1: Ethernet address: 00:0f:53:0e:20:d1
sfxge1: Solarstorm SFN5162F SFP+ Server Adapter
em0: port 0xe2c0-0xe2ff mem 0xfc4c0000-0xfc4dffff irq 10 at device 18.0 on pci0
em0: Using 1024 TX descriptors and 1024 RX descriptors
em0: Ethernet address: e2:34:ac:9e:35:68
em0: netmap queues/slots: TX 1/1024, RX 1/1024
pcib1: mem 0xfc512000-0xfc5120ff irq 10 at device 30.0 on pci0
pci1: on pcib1
pcib2: mem 0xfc513000-0xfc5130ff irq 11 at device 31.0 on pci0
pci2: on pcib2
acpi_syscontainer0: on acpi0
vmgenc0: on acpi0
acpi_syscontainer1: port 0xaf00-0xaf0b on acpi0
acpi_syscontainer2: port 0xafe0-0xafe3 on acpi0
acpi_syscontainer3: port 0xae00-0xae17 on acpi0
atkbdc0: port 0x60,0x64 irq 1 on acpi0
atkbd0: irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
WARNING: Device "psm" is Giant locked and may be deleted before FreeBSD 14.0.
psm0: model IntelliMouse Explorer, device ID 4
fdc0: port 0x3f2-0x3f5,0x3f7 irq 6 drq 2 on acpi0
fdc0: does not respond
device_attach: fdc0 attach returned 6
orm0: at iomem 0xea000-0xeffff pnpid ORM0000 on isa0
vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff pnpid PNP0900 on isa0
attimer0: at port 0x40 on isa0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
fdc0: No FDOUT register!
Timecounters tick every 10.000 msec
Trying to mount root from ufs:/dev/ufs/OPNsense [rw]...
ugen0.1: at usbus0
uhub0 on usbus0
uhub0: on usbus0
-
I did report this issue via opnsense as well with links to both github ticket# and this thread.
Thanks & let me know what you need me to test next. My version is now listed as : 22.7.a_83
-
Nice catch, this should do the trick https://github.com/opnsense/core/commit/2b21a8838a7
# opnsense-patch 2b21a8838a7
If you could confirm this doesn't happen after reboot again that would be great.
Also I'd like to see the output of the following just to make sure it all goes well under the hood:
# opnsense-log | grep Device
Thanks,
Franco
-
Still seeing the same issue after doing the following upgrade sequence:
21.7.8 -> 22.1 -> 22.1.1 -> devel branch -> 22.7.a_94 -> opnsense-patch 2b21a8838a7 -> reboot
Output as requested:
root@edge01:~ # opnsense-log | grep Device
<11>1 2022-02-18T20:13:21-06:00 edge01.*****net opnsense-devel 537 - [meta sequenceId="11"] /usr/local/etc/rc.bootup: Device gif0 required for opt6, configuring now
<11>1 2022-02-18T20:16:28-06:00 edge01.*****.net opnsense-devel 538 - [meta sequenceId="11"] /usr/local/etc/rc.bootup: Device gif0 required for opt6, configuring now
<11>1 2022-02-18T20:21:13-06:00 edge01.*****.net opnsense-devel 537 - [meta sequenceId="11"] /usr/local/etc/rc.bootup: Device gif0 required for opt6, configuring now
root@edge01:~ # opnsense-patch -l
2b21a8838a7 interfaces: $devices already filtered #5540
Edit: Modified hostname in output log for priacy reasons. Replaced <domain> with ****
I clicked the report error in opnsense webgui & referenched this thread & the github issue #. I hope this helps!
-
Oh so the thing about opnsense-patch is that it will apply or reverse the patch based on the system state. 22.7.a_94 already includes the patch (I updated this for the hotfix on the release yesterday as well) so you do not actually have to run opnsense-patch anymore.
Sorry for the trouble.
Cheers,
Franco
-
So I shut down & destroyed that vm, re-cloned from my 21.7.8
Upgraded as usual 21.7.8 -> 22.1 -> 22.1.1_3 -> devel 22.7.a_94.
No errors @ startup, although first reboot after 22.7.a_94 I had no connectivity at all (no v4 or v6). I had to reboot a 2nd time, then I had ipv4 only.
While the he.net tunnel shows "up" I have no (ipv6) connectivity from behind opnsense.
I tried restarting the RA service (for my "lan" subnet which is what i'm testing behind) & also trying to edit -> resave the tunnel to see if that would kick something dependant on it, but still no go.
I am going to test-ipv6.com for my testing, and I also use ipvfoo in chrome to show if I'm hitting v4/v6.
-
I did some testing from a desktop and rebooted the firewall again.
Upon reboot I had no connectivity (v4 or v6) past the firewall. I was still able to access OPNSense gui via ipv4 and that worked ok. What is weird is connectivity from the OPNsense box itself was ok! I could ping 8.8.8.8 for example, but not "through" the firewall from my "lan" interface (not even on v4).
I started up wireshark & was able to see no RA's coming in on my lan interface whicgh is static (for he.net tunnel). I edit->saved the interface & that kicked something so at least RA's came out & I got an address immediately. Still no connectivity through the firewall.
root@edge01:~ # opnsense-log | grep Device
<11>1 2022-02-19T16:12:44-06:00 edge01.*.net opnsense-devel 537 - [meta sequenceId="11"] /usr/local/etc/rc.bootup: Device gif0 required for opt6, configuring now
<11>1 2022-02-19T16:18:18-06:00 edge01.*.net opnsense-devel 537 - [meta sequenceId="11"] /usr/local/etc/rc.bootup: Device gif0 required for opt6, configuring now
<11>1 2022-02-19T16:26:35-06:00 edge01.*.net opnsense-devel 538 - [meta sequenceId="11"] /usr/local/etc/rc.bootup: Device gif0 required for opt6, configuring now
<11>1 2022-02-19T16:44:18-06:00 edge01.*.net opnsense-devel 538 - [meta sequenceId="11"] /usr/local/etc/rc.bootup: Device gif0 required for opt6, configuring now
<11>1 2022-02-19T16:55:30-06:00 edge01.*.net opnsense-devel 537 - [meta sequenceId="11"] /usr/local/etc/rc.bootup: Device gif0 required for opt6, configuring now
Edit: Added this output as we've used this for troubleshooting earlier in the thread:
root@edge01:~ # /usr/local/etc/rc.reload_all
Writing firmware setting...done.
Writing trust files...done.
Configuring login behaviour...done.
Configuring CRON...done.
Setting timezone...done.
Setting hostname: edge01.*.net
Generating /etc/hosts...done.
Generating /etc/resolv.conf...done.
Configuring loopback interface...done.
Creating wireless clone interfaces...done.
Configuring VLAN interfaces...done.
Configuring Loopback interfaces...Creating OpenVPN instances...done.
Configuring DMZ_Zimbra interface...done.
Configuring DMZ_mail interface...done.
Configuring DMZ_pihole interface...done.
Configuring DMZ_plex interface...done.
Configuring DMZ_www interface...done.
Configuring LAN interface...done.
Configuring LAN_NET_MGMT interface...done.
Configuring Lo1 interface...done.
Configuring Lo2 interface...done.
Configuring Lo3 interface...done.
Configuring TRANSIT interface...done.
Configuring WG_* interface...done.
Configuring WG_Josh interface...done.
Configuring WG_RobsParents interface...done.
Configuring HE_Chicago interface...done.
Configuring LAN_Xbox interface...done.
Configuring InternetBiz interface...done.
Creating IPsec VTI instances...done.
Setting up routes...done.
Configuring firewall.....failed.
Starting DHCPv4 service...done.
Starting router advertisement service...done.
Starting NTP service...done.
Configuring OpenSSH...done.
Starting Unbound DNS...done.
Starting web GUI...done.
Syncing OpenVPN settings...done.
Generating RRD graphs...done.
Stopping named.
Waiting for PIDS: 58875.
Stopping zebra.
Waiting for PIDS: 9690.
[#] rm -f /var/run/wireguard/wg0.sock
[#] rm -f /var/run/wireguard/wg1.sock
[#] rm -f /var/run/wireguard/wg2.sock
Stopping php_fpm.
Waiting for PIDS: 96900.
Performing sanity check on php-fpm configuration:
[19-Feb-2022 17:11:56] NOTICE: configuration file /usr/local/etc/php-fpm.conf test is successful
Starting php_fpm.
setup sfxge0_vlan10 [egress only]
setup sfxge0_vlan100
setup sfxge0_vlan99
setup sfxge0_vlan120
setup lo1
ngctl: send msg: No such file or directory
error lo1: cannot create netflow node for lo1
error : interface wg0 not found
[#] ifconfig wg create name wg0
[!] Missing WireGuard kernel support (ifconfig: SIOCIFCREATE2: Invalid argument). Falling back to slow userspace implementation.
[#] wireguard-go wg0
┌──────────────────────────────────────────────────────┐
│ │
│ Running wireguard-go is not required because this │
│ kernel has first class support for WireGuard. For │
│ information on installing the kernel module, │
│ please visit: │
│ https://www.wireguard.com/install/ │
│ │
└──────────────────────────────────────────────────────┘
[#] wg setconf wg0 /dev/stdin
[#] ifconfig wg0 inet 192.168.30.1/27 alias
[#] ifconfig wg0 inet6 2001:470:c1a3:f1c9::1/64 alias
[#] ifconfig wg0 mtu 1420
[#] ifconfig wg0 up
[#] route -q -n add -inet 192.168.30.7/32 -interface wg0
[#] route -q -n add -inet 192.168.30.6/32 -interface wg0
[#] route -q -n add -inet 192.168.30.5/32 -interface wg0
[#] route -q -n add -inet 192.168.30.4/32 -interface wg0
[#] route -q -n add -inet 192.168.30.3/32 -interface wg0
[#] route -q -n add -inet 192.168.30.2/32 -interface wg0
[+] Backgrounding route monitor
[#] ifconfig wg create name wg1
[!] Missing WireGuard kernel support (ifconfig: SIOCIFCREATE2: Invalid argument). Falling back to slow userspace implementation.
[#] wireguard-go wg1
┌──────────────────────────────────────────────────────┐
│ │
│ Running wireguard-go is not required because this │
│ kernel has first class support for WireGuard. For │
│ information on installing the kernel module, │
│ please visit: │
│ https://www.wireguard.com/install/ │
│ │
└──────────────────────────────────────────────────────┘
[#] wg setconf wg1 /dev/stdin
[#] ifconfig wg1 inet 192.168.30.33/29 alias
[#] ifconfig wg1 inet6 2001:470:c1a3:ac09::1/64 alias
[#] ifconfig wg1 mtu 1420
[#] ifconfig wg1 up
[#] route -q -n add -inet6 2603:300a:1758:c01a::35/128 -interface wg1
[#] route -q -n add -inet 192.168.30.35/32 -interface wg1
[#] route -q -n add -inet 192.168.30.34/32 -interface wg1
[+] Backgrounding route monitor
[#] ifconfig wg create name wg2
[!] Missing WireGuard kernel support (ifconfig: SIOCIFCREATE2: Invalid argument). Falling back to slow userspace implementation.
[#] wireguard-go wg2
┌──────────────────────────────────────────────────────┐
│ │
│ Running wireguard-go is not required because this │
│ kernel has first class support for WireGuard. For │
│ information on installing the kernel module, │
│ please visit: │
│ https://www.wireguard.com/install/ │
│ │
└──────────────────────────────────────────────────────┘
[#] wg setconf wg2 /dev/stdin
[#] ifconfig wg2 inet 192.168.30.41/29 alias
[#] ifconfig wg2 inet6 2001:470:c1a3:ee10::1/64 alias
[#] ifconfig wg2 mtu 1420
[#] ifconfig wg2 up
[#] route -q -n add -inet 192.168.30.42/32 -interface wg2
[+] Backgrounding route monitor
Checking zebra.conf
2022/02/19 17:11:56 ZEBRA: [EC 4043309111] Disabling MPLS support (no kernel support)
OK
Starting zebra.
2022/02/19 17:11:56 ZEBRA: [EC 4043309111] Disabling MPLS support (no kernel support)
Starting named.
I wonder if "Configuring firewall.....failed." is a symptom to why I don't have any connectivity...?
-
Hi all,
After thinking about this a bit, I realized my base sfxge0 interface wasn't assigned anywhere, and I believe this is required for OPNSense > 21.7.8. I assigned the base sfxge0 to an interface I'm not using, and now everything appears to be working. My he.net tunnel is now also working at start without me doing anything else.
This is what my base interface looks like now:
Thank you @Franco!!!!
Now that I'm on 21.7.a_94, will I be able to merge back to stable sometime soon ?
-
Although I did find one more problem.....all my unbound overrides (mostly for things on my lan/domain that are static ip) are all gone in the dev version. They've just disapeared.
-
Hi,
Sorry for not responding earlier... Indeed "Configuring firewall.....failed." would point to an issue. I scanned your previous report but it seemed rather erratic outside of what this patch would/could change.
Nice you figured out the issue about sfxge0 and getting it up and running. I'm somewhat doubting the approach that is currently necessary for assignment but there's no better proposal at the moment.
About Unbound this is an issue of the rework of the overrides into MVC framework. Are they not displaying at all anymore in the new GUI pages for it and/or are they not working? The system will migrate them to a new config.xml location where it can still use them. But there may be 1-2 issues left to iron out. Apologies this got in the way during testing.
Cheers,
Franco
-
Hey Franco,
No problem at all on delayed response. I realize you guys have a ton of tickets to look at and you are volunteers. I think everyone here appreciates all the work you & the developers put into this.
The sfxge should have been obvious. Sorry I missed that. I don't mind assigning the base interface and then not using it. It's really no issue. If that is a FreeBSD limitation, so be it. There are more important things to worry about than something with an easy workaround.
In the (gui) at least the entire overrides is completely blank like they don't exist. I was thinking a good way to troubleshoot would be to actually download the 21.7.8 config & the new (dev) config & compare. Even if they aren't directly comparable due to version changes, I can see at least if the overrides exist which might point to a parser failure or something.
No apologies necessary. I know I'm on a testing devel branch that you added specific interface re-work that I needed & fixed my problem. I still go back & turn off my "production" (for my house) 21.7.8 config & start up the devel config to continue troubleshooting when I have time.
To be honest, this is one of the most responsive open source projects I've ever worked with & am very thankful for that.
Keep up the good work!
-
Thanks, very nice to hear. :)
So yes you can compare the resulting config.xml (just export) where the new overrides should be stored under OPNsense/unboundplus tag. On your 21.7.8 system it is stored under "unbound".
If you don't mind you can also send me the old "unbound" portion to examine via mail franco@opnsense.org so I can see what's going on quickly.
Cheers,
Franco
-
Hey franco,
Just wondering if the new 22.1.2 incorpates the fixes you made for the tunnel. Should I be testing from that or staing on dev channel?
Let me know!
-
Hi 5SpeedFun,
It's still on devel for 22.1.2 but we plan to integrate it into 22.1.3. Unbound overrides code is still problematic so if you could give me more hints for that that would be highly appreciated.
Cheers,
Franco
-
Thanks, very nice to hear. :)
So yes you can compare the resulting config.xml (just export) where the new overrides should be stored under OPNsense/unboundplus tag. On your 21.7.8 system it is stored under "unbound".
If you don't mind you can also send me the old "unbound" portion to examine via mail franco@opnsense.org so I can see what's going on quickly.
Cheers,
Franco
Hey Franco,
I looked into this.
Basically, my <unbound> sections on both 21.7.8 and 22.7.a_94 are identical.
However 22.7.a_94 also has an unboundplus section as follows:
<unboundplus version="1.0.0">
<service_enabled/>
<dnsbl>
<enabled>0</enabled>
<type/>
<lists/>
<whitelists/>
</dnsbl>
<dots/>
<miscellaneous>
<privatedomain/>
</miscellaneous>
</unboundplus>
Maybe the <unbound> section never got migrated for some reason?
I've e-mailed you my 21.7.8 unbound config.
-
Uh, thanks, it means the migration failed which is also why the overrides appear to be gone. They are not gone, but they can't be used either. Should be easy to fix.
Cheers,
Franco
-
Uh, thanks, it means the migration failed which is also why the overrides appear to be gone. They are not gone, but they can't be used either. Should be easy to fix.
Cheers,
Franco
Yeah. I think they are just "gone" in the sense they don't appear in the GUI. I also don' t think they are loaded into unbound since they are in the wrong section. But obviously if you download the opnsense configuration you can still see them there.
Always happy to help :)
5SF
-
Hey @franco, did you get my email with a list of overrides? I haven't heard anything here or by email so not sure if you received it or it went into your junk folder ;D
-
Hi,
I was away with the family for a couple of days. I got your mail to be processed when I have crawled through all open mails and forum posts from the last week. :)
So with 22.1.2(_1) I prepared a backport for 22.1.3 and tested it a few times:
https://github.com/opnsense/core/commit/429b60c26
That should be the gist of your issue taken care of and you can easily try it out after upgrading to the 22.1.2 release (not development) and issuing:
# opnsense-patch 429b60c26
If you think that works we will go ahead and include it in 22.1.3 and the bonus is the unbound changes are not in 22.1.x yet so it keep working until we fix that one in another feature update or maybe 22.7 itself. Not sure yet.
Cheers,
Franco
-
Thanks Franco! I'll give that a shot in the next day or two. I'm due to re-clone my 21.7.8 "production" vm as I've made alias & other small changes.
I am always worried if email from my personal domain goes to spam. While it has been around 10+ years, it's still hosted at home 8)
-
Hi,
I was away with the family for a couple of days. I got your mail to be processed when I have crawled through all open mails and forum posts from the last week. :)
So with 22.1.2(_1) I prepared a backport for 22.1.3 and tested it a few times:
https://github.com/opnsense/core/commit/429b60c26
That should be the gist of your issue taken care of and you can easily try it out after upgrading to the 22.1.2 release (not development) and issuing:
# opnsense-patch 429b60c26
If you think that works we will go ahead and include it in 22.1.3 and the bonus is the unbound changes are not in 22.1.x yet so it keep working until we fix that one in another feature update or maybe 22.7 itself. Not sure yet.
Cheers,
Franco
i started at my 21.7.8 -> 22.1 -> 22.1.2_1 -> patch 429b60c26
Looking relaly good. All my services are started & he.net tunnel works with no finangling and comes up @ startup.
My unbound dns overrides are there as well (I assume this is because the devel stuff I helped you with before, that previously broke, is only in the devel tree and/or after patch 429b60c26.
Overall this looks really good. Excellent work!
Would you like me to help you test the unbound overrides on another thread or github?
Thanks for all your great work!
5SF
ALso: Can I stay on this version until next release and then upgrade cleanly or should I go back to 21.7.8 and wait until this patch is included in a release?
-
Thanks, happy to hear.
The patch you applied will be part of 22.1.3 then. I did more testing this morning and it looks good from here as well. You can stay on what you have now if it works for you. :)
Yes, Unbound is still on devel. Will look at it later today so the devel version with 22.1.3 will hold that code to try out via snapshot. Still unclear on the unbound changes ETA so that would be just for snapshot hopping and helping us out if it looks better then.
Cheers,
Franco
-
Unfortunately I may have to roll back again.
The firewall doesn't nat traffic on startup. I think it may be related to Configuring Firewal...failed.
Doing a packet capture across my "internet" and "lan" interface shows traffic exiting unnatted. Connectivity from the firewall itself is ok, however.
Any suggestions on how to troubleshoot would be appreciated.
5SF
-
It should land in logs... can you see the error?
# opnsense-log | grep "There were error"
Though I'm unsure if this is a transient error on boot that fixes itself later or not. Not loading pf rules at all would likely stop making LAN traffic work...
Cheers,
Franco
-
Sorry for the delay on this.
This is what showed up:
<11>1 2022-03-19T23:59:26-05:00 edge01.mydomain.net opnsense 532 - [meta sequenceId="52"] /usr/local/etc/rc.bootup: There were error(s) loading the rules: /tmp/rules.debug:580: syntax error - The line in question reads [580]: pass in quick on sfxge0_vlan99 route-to ( sfxge0_vlan10 fe80::d094:2eff:fec4:27d2 ) reply-to ( sfxge0_vlan10 fe80::d094:2eff:fec4:27d2 ) inet6 proto ipv6-icmp from {(sfxge0_vlan99:network)} to {any} keep state label "c76413485ed2ae88121270a76c0ee3fa" # : ICMP v6
<11>1 2022-03-19T23:59:28-05:00 edge01.mydomain.net opnsense 65537 - [meta sequenceId="56"] /usr/local/etc/rc.filter_configure: There were error(s) loading the rules: /tmp/rules.debug:580: syntax error - The line in question reads [580]: pass in quick on sfxge0_vlan99 route-to ( sfxge0_vlan10 fe80::d094:2eff:fec4:27d2 ) reply-to ( sfxge0_vlan10 fe80::d094:2eff:fec4:27d2 ) inet6 proto ipv6-icmp from {(sfxge0_vlan99:network)} to {any} keep state label "c76413485ed2ae88121270a76c0ee3fa" # : ICMP v6
<11>1 2022-03-19T23:59:32-05:00 edge01.mydomain.net opnsense 88213 - [meta sequenceId="10"] /usr/local/etc/rc.routing_configure: There were error(s) loading the rules: /tmp/rules.debug:586: syntax error - The line in question reads [586]: pass in quick on sfxge0_vlan99 route-to ( sfxge0_vlan10 fe80::d094:2eff:fec4:27d2 ) reply-to ( sfxge0_vlan10 fe80::d094:2eff:fec4:27d2 ) inet6 proto ipv6-icmp from {(sfxge0_vlan99:network)} to {any} keep state label "c76413485ed2ae88121270a76c0ee3fa" # : ICMP v6
<11>1 2022-03-19T23:59:33-05:00 edge01.mydomain.net opnsense 87533 - [meta sequenceId="15"] /usr/local/etc/rc.newwanipv6: There were error(s) loading the rules: /tmp/rules.debug:586: syntax error - The line in question reads [586]: pass in quick on sfxge0_vlan99 route-to ( sfxge0_vlan10 fe80::d094:2eff:fec4:27d2 ) reply-to ( sfxge0_vlan10 fe80::d094:2eff:fec4:27d2 ) inet6 proto ipv6-icmp from {(sfxge0_vlan99:network)} to {any} keep state label "c76413485ed2ae88121270a76c0ee3fa" # : ICMP v6
<11>1 2022-03-19T23:59:34-05:00 edge01.mydomain.net opnsense 41080 - [meta sequenceId="23"] /usr/local/etc/rc.routing_configure: There were error(s) loading the rules: /tmp/rules.debug:586: syntax error - The line in question reads [586]: pass in quick on sfxge0_vlan99 route-to ( sfxge0_vlan10 fe80::d094:2eff:fec4:27d2 ) reply-to ( sfxge0_vlan10 fe80::d094:2eff:fec4:27d2 ) inet6 proto ipv6-icmp from {(sfxge0_vlan99:network)} to {any} keep state label "c76413485ed2ae88121270a76c0ee3fa" # : ICMP v6
<11>1 2022-03-19T23:59:37-05:00 edge01.mydomain.net opnsense 20 - [meta sequenceId="25"] /usr/local/etc/rc.filter_configure: There were error(s) loading the rules: /tmp/rules.debug:586: syntax error - The line in question reads [586]: pass in quick on sfxge0_vlan99 route-to ( sfxge0_vlan10 fe80::d094:2eff:fec4:27d2 ) reply-to ( sfxge0_vlan10 fe80::d094:2eff:fec4:27d2 ) inet6 proto ipv6-icmp from {(sfxge0_vlan99:network)} to {any} keep state label "c76413485ed2ae88121270a76c0ee3fa" # : ICMP v6
<11>1 2022-03-19T23:59:48-05:00 edge01.mydomain.net opnsense 92965 - [meta sequenceId="33"] /usr/local/etc/rc.filter_configure: There were error(s) loading the rules: /tmp/rules.debug:586: syntax error - The line in question reads [586]: pass in quick on sfxge0_vlan99 route-to ( sfxge0_vlan10 fe80::d094:2eff:fec4:27d2 ) reply-to ( sfxge0_vlan10 fe80::d094:2eff:fec4:27d2 ) inet6 proto ipv6-icmp from {(sfxge0_vlan99:network)} to {any} keep state label "c76413485ed2ae88121270a76c0ee3fa" # : ICMP v6
-
Huh, I think it doesn't support reply-to and route-to at the same time, but I'm a bit confused how it ends up generating it?
Cheers,
Franco
-
I've added additional validation: https://github.com/opnsense/core/commit/77594c8b99a39
You need to remove the reply-to option from this rule as it breaks the generated rule due to rule model trying to be more complex than it is.
Cheers,
Franco
-
Hey Franco,
Thanks for the fix. I disabled this and ping still works ok. I think I had set this way back when trying to figure out how to policy-route my he.net tunnel (for my servers) and my local ipv6 (for my non-static clients like xbox).
After disabling the reply-to in 21.7.8, ping still works, so I will try the upgrade tonight and see how it goes.
-
Super, thanks. I'm unsure we were looking at the right problem.
This might be of more relevance as indicated by another thread: https://github.com/opnsense/core/commit/61500f6790
Cheers,
Franco
-
Hi,
I was away with the family for a couple of days. I got your mail to be processed when I have crawled through all open mails and forum posts from the last week. :)
So with 22.1.2(_1) I prepared a backport for 22.1.3 and tested it a few times:
https://github.com/opnsense/core/commit/429b60c26
That should be the gist of your issue taken care of and you can easily try it out after upgrading to the 22.1.2 release (not development) and issuing:
# opnsense-patch 429b60c26
If you think that works we will go ahead and include it in 22.1.3 and the bonus is the unbound changes are not in 22.1.x yet so it keep working until we fix that one in another feature update or maybe 22.7 itself. Not sure yet.
Cheers,
Franco
Hi Franco,
I re-cloned my 21.7.8 again and started a fresh upgrade -> 22.1 -> 22.1.3. Af far as I understand, 429b60c26 is already included. Everything seems to start up just fine, including my he.net tunnel, however radvd isn't giving out any router advertisements, so I never get an ipv6 address (I use slaac). ipv4/nat seems to work OK, just no ipv6 due to no radvd advertisements. I've tried restarting the service but that didn't help either.
Using my main lan (sfxge0_vlan100) interface, for example, I believe tcpdump -n -i sfxge0_vlan100 icmp6 and 'ip6[40] = 134' should get ras, but it never captures any, even after several minutes or after restarting radvd.
I did find a bunch of these in /var/log/routing/latest.log
<27>1 2022-03-21T22:22:09-05:00 edge01.mydomain.net rtsold 76016 - [meta sequenceId="372"] <call_script> write to child failed: Broken pipe
I also do see radvd running:
root@edge01:/var/log/routing # ps aux | grep radvd
root 7566 0.0 0.0 12724 2332 - Ss 22:13 0:00.03 /usr/local/sbin/radvd -p /var/run/radvd.pid -C /var/etc/radvd.conf -m syslog
Restarting radvd gives the following in /var/log/routing/latest.log
<28>1 2022-03-21T22:27:21-05:00 edge01.mydomain.net radvd 87550 - [meta sequenceId="441"] exiting, 1 sigterm(s) received
<30>1 2022-03-21T22:27:21-05:00 edge01.mydomain.net radvd 87550 - [meta sequenceId="442"] sending stop adverts
<30>1 2022-03-21T22:27:21-05:00 edge01.mydomain.net radvd 87550 - [meta sequenceId="443"] removing /var/run/radvd.pid
<30>1 2022-03-21T22:27:21-05:00 edge01.mydomain.net radvd 87550 - [meta sequenceId="444"] returning from radvd main
<30>1 2022-03-21T22:27:21-05:00 edge01.mydomain.net radvd 28834 - [meta sequenceId="445"] version 2.19 started
<27>1 2022-03-21T22:27:24-05:00 edge01.mydomain.net rtsold 76016 - [meta sequenceId="446"] <call_script> write to child failed: Broken pipe
Let me know where else I should look.
Thanks!
5SF
-
Hi 5SF,
No, 429b60c26 is not included in 22.1.3 but it will be in 22.1.4 as others confirmed this fixes an issue with disappearing address on the tunnel. Maybe that's already the issue here as radvd would need it maybe...
Cheers,
Franco
-
Ok, so then after 22.1.3 upgrade I should do
opnsense-patch 429b60c26 and see if that fixes the radvd issue?
Edit: Tried this, and it said a reverse or previously applied patch was detected. Didn't fix the issue. I'm going to do a little more troubleshooting tonight when I have more time.
Edit 2: I think radvd will still send out ras even if the tunnel is down, as I have the lan interface ip/vip statically set. Even if the tunnel was down, I have ULA on the lan interface & I'm not getting RAs for that either.
-
Argh, sorry I meant 61500f6790 as originally posted, see my comment above. I thought you were referring to this, but you meant the actual feature included in 22.1.3... yes, but 61500f6790 as a fix for it has not been added. Can you try it?
# opnsense-patch 61500f6790
Thanks,
Franco
-
I will try this & report back :)
-
Hey @franco,
Still same issue after & logs after that patch. Not getting an ip address. I still think it's related to radvd. Is there a better place to see more detailed logs from radvd than /var/logs/router/latest ?
-
hmmm, what's the contents of /var/etc/radvd.conf when it works and when it doesn't?
Cheers,
Franco
-
Hey Franco,
So I pulled radvd.conf from my (working) 21.7.8 and compared it to radvd.conf of 22.1.4 and sfxge0_vlan100 (my lan vlan) is entirely missing from radvd.conf. Outside of that it looks idential (i'm comparing using Meld on Linux).
I have no idea why.
Here is the config that is missing:
interface sfxge0_vlan100 {
AdvSendAdvert on;
MinRtrAdvInterval 200;
MaxRtrAdvInterval 600;
AdvLinkMTU 1500;
AdvDefaultPreference medium;
AdvManagedFlag on;
AdvOtherConfigFlag on;
prefix xxxx:yyy:zzzz:a0::/64 {
DeprecatePrefix off;
AdvOnLink on;
AdvAutonomous on;
};
prefix fd4a:ea50:349f:a0::/64 {
DeprecatePrefix off;
AdvOnLink on;
AdvAutonomous on;
};
RDNSS xxxx:yyy:zzzz:a93c::53 {
};
DNSSL mydomain.net {
};
};
# Generated for DHCPv6 server opt2
Note: Replaced some real ips with xxxx:yyy:zzzz and mydomain.net to replace my real domain.
If something doesn't stick out on this, I can see exactly when my config disspears between 21.7.8 and 22.1.4 by pulling the config after upgrading to each version.
Let me know if you want me to do that.
5SF
-
Can you try this diagnostics patch to see which error case we are dealing with?
https://github.com/opnsense/core/commit/8e444c3b32bc
The resulting radvd.conf will indicate why it excluded the interface from the config.
Thanks,
Franco
-
I sure can! Do I apply this against 22.1.4 ?
Is the radvd generated from the interface configs? Edit2: it looks like the code will write a commend in the config if it's excluded..
I'll try this out tonight & post results.
-
Yes, 22.1.4 and, yes, the comment ends up in radvd.conf
Cheers,
Franco
-
# Skipping addressless interface lan
Lan has a static v6 address as well as a VIP assigned to it.
-
Ok, almost there...
# grep rainterface /conf/config.xml
Trying to make sure we can resolve whatever is configured. I don't think it refuses to see the static IPv6 because the "source address" setting may be set instead and is not correct (it can only be a link local CARP or alias address).
Cheers,
Franco
-
Result:
root@edge01:~ # grep rainterface /conf/config.xml
<rainterface>static</rainterface>
-
Aha! "static" no longer exists:
https://github.com/opnsense/changelog/blob/33adbc10d98db0f9c4412696bc58c3614ea6d6fa/community/22.1/22.1#L104
When you clear the "source address" selection (set "automatic") you can get it working again...
As mentioned in the source change https://github.com/opnsense/core/commit/a94c63b2b763ff you can emulate "static" mode by adding two advanced options to router advertisements on the same page:
AdvDeprecatePrefix off
AdvRemoveRoute off
(or adjust as required)
Cheers,
Franco
-
Hey Franco, that seems to have fixed it.
When I looked at source address seelection the only option was automatic, so I just re-saved the RA -> [LAN] interface.
I then added "Off" to both of those options and it promptly broke :)
opnsense-log showed a syntax error at line 12 in radvd.conf, and it appears I needed to put off not Off.
Once I did that, ipv6 is working again!
Also: Does this look as expected?
root@edge01:~ # grep rainterface /conf/config.xml
root@edge01:~ #
Thanks for all your help! I'm going to run on 22.1.4 and see if I find any more issues.
-
Looks good from here... we did it. \o/
Cheers,
Franco
-
Thanks so much Franco!
One question: Can you explain when those 2 Advanced options are needed for an interface? Are they needed any time there is both a static v4 & v6 address? Some other criteria?
-
The legacy static mode would prevent addresses and routes from begin revoked when the router restarts/radvd itself is restarted for reconfiguration. That's done to prevent intermittent connectivity issues on clients if you have a static setup anyway.
After a longer discussion on GitHub it was concluded to break up the static mode into its individual pieces so we removed the setting and added the two configuration flags instead for those who wish to use it still.
For CARP mode these two configuration flags turned to off are still the standard.
Cheers,
Franco