OPNsense Forum
Archive => 18.7 Legacy Series => Topic started by: chemlud on January 11, 2019, 11:41:55 am
-
Hy!
Updated this morning to 18.7.10 amd64 with LibreSSL, some minutes ago I lost one interface, which came up again after plugging out/in the RJ45:
Jan 11 11:36:40 opnsense: /usr/local/etc/rc.newwanip: On (IP address: 10.45.23.1) (interface: iNET[opt1]) (real interface: igb1).
Jan 11 11:36:40 opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'igb1'
Jan 11 11:36:40 opnsense: /usr/local/etc/rc.linkup: Hotplug event detected for iNET(opt1) but ignoring since interface is configured with static IP (10.45.23.1 ::)
Jan 11 11:36:40 kernel: igb1: link state changed to UP
Jan 11 11:36:20 opnsense: /usr/local/etc/rc.linkup: Hotplug event detected for iNET(opt1) but ignoring since interface is configured with static IP (10.45.23.1 ::)
Jan 11 11:36:20 kernel: igb1: link state changed to DOWN
-
Hello there,
Plan to do a snapshot and upgrade to 18.7.10, but I don't seem to understand what's happening there in your post? It was hotplugged?
Regards
-
I didn't change anything, but suddenly I couldn't reach one of my local subnets anymore. Only thing I could do was to pull the RJ45 and stick it in again. Afterwards the subnet was back. Never had this with the opnsense before.
Then I had a look in the logs and found what I posted above...
-
Instead of logs, did you look at 'dmesg' before you unplug the cable, was there a 'link' light on?
-
Here the complete dmesg since the reboot this morning after upgrading to 18.7.10:
Copyright (c) 1992-2017 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 11.1-RELEASE-p18 4b57161bcff(stable/18.7) amd64
FreeBSD clang version 4.0.0 (tags/RELEASE_400/final 297347) (based on LLVM 4.0.0)
VT(vga): resolution 640x480
[HBSD ASLR (compat)] status: opt-out
[HBSD ASLR (compat)] mmap: 14 bit
[HBSD ASLR (compat)] exec base: 14 bit
[HBSD ASLR (compat)] stack: 14 bit
[HBSD ASLR (compat)] vdso: 8 bit
[HBSD LOG] logging to system: enabled
[HBSD LOG] logging to user: disabled
[HBSD HARDENING] procfs hardening: enabled
[HBSD ASLR] status: opt-out
[HBSD ASLR] mmap: 30 bit
[HBSD ASLR] exec base: 30 bit
[HBSD ASLR] stack: 42 bit
[HBSD ASLR] vdso: 28 bit
[HBSD ASLR] map32bit: 18 bit
[HBSD ASLR] disallow MAP_32BIT mode mmap: opt-in
[HBSD SEGVGUARD] status: opt-out
[HBSD SEGVGUARD] expiry: 120 sec
[HBSD SEGVGUARD] suspension: 600 sec
[HBSD SEGVGUARD] maxcrashes: 5
CPU: Intel(R) Core(TM) i3-2100 CPU @ 3.10GHz (3093.04-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x206a7 Family=0x6 Model=0x2a Stepping=7
Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLF>
Features2=0x1d9ae3bf<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE>
AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM>
AMD Features2=0x1<LAHF>
Structured Extended Features3=0xc000000<IBPB,STIBP>
XSAVE Features=0x1<XSAVEOPT>
VT-x: (disabled in BIOS) PAT,HLT,MTF,PAUSE,EPT,UG,VPID
TSC: P-state invariant, performance statistics
real memory = 8589934592 (8192 MB)
avail memory = 8137867264 (7760 MB)
Event timer "LAPIC" quality 600
ACPI APIC Table: <DELL CBX3 >
FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs
FreeBSD/SMP: 1 package(s) x 2 core(s) x 2 hardware threads
random: unblocking device.
ioapic0 <Version 2.0> irqs 0-23 on motherboard
SMP: AP CPU #1 Launched!
SMP: AP CPU #3 Launched!
SMP: AP CPU #2 Launched!
Timecounter "TSC-low" frequency 1546521568 Hz quality 1000
random: entropy device external interface
wlan: mac acl policy registered
netmap: loaded module
module_register_init: MOD_LOAD (vesa, 0xffffffff810b3240, 0) error 19
kbd1 at kbdmux0
nexus0
vtvga0: <VT VGA driver> on motherboard
cryptosoft0: <software crypto> on motherboard
acpi0: <DELL CBX3 > on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
cpu2: <ACPI CPU> on acpi0
cpu3: <ACPI CPU> on acpi0
hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on acpi0
Timecounter "HPET" frequency 14318180 Hz quality 950
Event timer "HPET" frequency 14318180 Hz quality 550
Event timer "HPET1" frequency 14318180 Hz quality 440
Event timer "HPET2" frequency 14318180 Hz quality 440
Event timer "HPET3" frequency 14318180 Hz quality 440
Event timer "HPET4" frequency 14318180 Hz quality 440
atrtc0: <AT realtime clock> port 0x70-0x77 irq 8 on acpi0
atrtc0: Warning: Couldn't map I/O.
Event timer "RTC" frequency 32768 Hz quality 0
attimer0: <AT timer> port 0x40-0x43,0x50-0x53 irq 0 on acpi0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pcib0: _OSC returned error 0x4
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> irq 16 at device 1.0 on pci0
pci1: <ACPI PCI bus> on pcib1
igb0: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0x4020-0x403f mem 0xe34200001
igb0: Using MSIX interrupts with 5 vectors
igb0: Ethernet address: 6c:b3:11:yy:yy:y
igb0: Bound queue 0 to cpu 0
igb0: Bound queue 1 to cpu 1
igb0: Bound queue 2 to cpu 2
igb0: Bound queue 3 to cpu 3
igb0: netmap queues/slots: TX 4/1024, RX 4/1024
igb1: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0x4000-0x401f mem 0xe34000001
igb1: Using MSIX interrupts with 5 vectors
igb1: Ethernet address: 6c:b3:11:yy:yy:yy
igb1: Bound queue 0 to cpu 0
igb1: Bound queue 1 to cpu 1
igb1: Bound queue 2 to cpu 2
igb1: Bound queue 3 to cpu 3
igb1: netmap queues/slots: TX 4/1024, RX 4/1024
vgapci0: <VGA-compatible display> port 0x5000-0x503f mem 0xe0c00000-0xe0ffffff,0xd0000000-0xdffff0
vgapci0: Boot video device
pci0: <simple comms> at device 22.0 (no driver attached)
em0: <Intel(R) PRO/1000 Network Connection 7.6.1-k> port 0x5080-0x509f mem 0xe3f00000-0xe3f1ffff,0
em0: Using an MSI interrupt
em0: Ethernet address: 18:03:73:yy:yy:yy
em0: netmap queues/slots: TX 1/1024, RX 1/1024
ehci0: <Intel Cougar Point USB 2.0 controller> mem 0xe3f70000-0xe3f703ff irq 16 at device 26.0 on0
usbus0: EHCI version 1.0
usbus0 on ehci0
usbus0: 480Mbps High Speed USB v2.0
hdac0: <Intel Cougar Point HDA Controller> mem 0xe3f60000-0xe3f63fff irq 22 at device 27.0 on pci0
pcib2: <ACPI PCI-PCI bridge> irq 16 at device 28.0 on pci0
pci2: <ACPI PCI bus> on pcib2
pcib3: <ACPI PCI-PCI bridge> irq 18 at device 28.2 on pci0
pcib3: [GIANT-LOCKED]
pcib4: <ACPI PCI-PCI bridge> irq 16 at device 28.4 on pci0
pci3: <ACPI PCI bus> on pcib4
igb2: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0x2020-0x203f mem 0xe20200003
igb2: Using MSIX interrupts with 5 vectors
igb2: Ethernet address: 6c:b3:11:yy:yy:yy
igb2: Bound queue 0 to cpu 0
igb2: Bound queue 1 to cpu 1
igb2: Bound queue 2 to cpu 2
igb2: Bound queue 3 to cpu 3
igb2: netmap queues/slots: TX 4/1024, RX 4/1024
igb3: <Intel(R) PRO/1000 Network Connection, Version - 2.5.3-k> port 0x2000-0x201f mem 0xe20000003
igb3: Using MSIX interrupts with 5 vectors
igb3: Ethernet address: 6c:b3:11:yy:yy:yy
igb3: Bound queue 0 to cpu 0
igb3: Bound queue 1 to cpu 1
igb3: Bound queue 2 to cpu 2
igb3: Bound queue 3 to cpu 3
igb3: netmap queues/slots: TX 4/1024, RX 4/1024
ehci1: <Intel Cougar Point USB 2.0 controller> mem 0xe3f50000-0xe3f503ff irq 17 at device 29.0 on0
usbus1: EHCI version 1.0
usbus1 on ehci1
usbus1: 480Mbps High Speed USB v2.0
pcib5: <ACPI PCI-PCI bridge> at device 30.0 on pci0
pci4: <ACPI PCI bus> on pcib5
isab0: <PCI-ISA bridge> at device 31.0 on pci0
isa0: <ISA bus> on isab0
ahci0: <Intel Cougar Point AHCI SATA controller> port 0x50d0-0x50d7,0x50c0-0x50c3,0x50b0-0x50b7,00
ahci0: AHCI v1.30 with 6 6Gbps ports, Port Multiplier not supported
ahcich2: <AHCI channel> at channel 2 on ahci0
ahciem0: <AHCI enclosure management bridge> on ahci0
acpi_button0: <Power Button> on acpi0
acpi_syscontainer0: <System Container> on acpi0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: console (115200,n,8,1)
orm0: <ISA Option ROMs> at iomem 0xce800-0xcf7ff,0xcf800-0xd07ff,0xd0800-0xd17ff,0xd1800-0xd27ff 0
atkbdc0: <Keyboard controller (i8042)> at port 0x60,0x64 on isa0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
ppc0: cannot reserve I/O port range
est0: <Enhanced SpeedStep Frequency Control> on cpu0
est1: <Enhanced SpeedStep Frequency Control> on cpu1
est2: <Enhanced SpeedStep Frequency Control> on cpu2
est3: <Enhanced SpeedStep Frequency Control> on cpu3
Timecounters tick every 1.000 msec
nvme cam probe device init
hdacc0: <Realtek ALC269 HDA CODEC> at cad 0 on hdac0
hdaa0: <Realtek ALC269 Audio Function Group> at nid 1 on hdacc0
pcm0: <Realtek ALC269 (Analog)> at nid 20 and 24 on hdaa0
pcm1: <Realtek ALC269 (Analog 2.0+HP/2.0)> at nid 27,33 and 25 on hdaa0
hdacc1: <Intel Cougar Point HDA CODEC> at cad 3 on hdac0
hdaa1: <Intel Cougar Point Audio Function Group> at nid 1 on hdacc1
pcm2: <Intel Cougar Point (HDMI/DP 8ch)> at nid 6 on hdaa1
ugen1.1: <Intel EHCI root HUB> at usbus1
ugen0.1: <Intel EHCI root HUB> at usbus0
ses0 at ahciem0 bus 0 scbus1 target 0 lun 0
uhub0: ses0: <AHCI SGPIO Enclosure 1.00 0001> SEMB S-E-S 2.00 device
ses0: SEMB SES Device
<Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus1
uhub1: ada0 at ahcich2 bus 0 scbus0 target 0 lun 0
<Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on usbus0
ada0: <TS64GSSD370S P1225CE> ACS-2 ATA SATA 3.x device
ada0: Serial Number E029431460
ada0: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 1024bytes)
ada0: Command Queueing enabled
ada0: 61057MB (125045424 512 byte sectors)
Trying to mount root from ufs:/dev/ufs/OPNsense [rw]...
uhub0: 2 ports with 2 removable, self powered
uhub1: 2 ports with 2 removable, self powered
ugen1.2: <vendor 0x8087 product 0x0024> at usbus1
uhub2 on uhub0
uhub2: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus1
ugen0.2: <vendor 0x8087 product 0x0024> at usbus0
uhub3 on uhub1
uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr 2> on usbus0
uhub3: 6 ports with 6 removable, self powered
uhub2: 8 ports with 8 removable, self powered
igb0: link state changed to UP
igb1: link state changed to UP
em0: link state changed to UP
igb2: link state changed to UP
aesni0: No AESNI support.
coretemp0: <CPU On-Die Thermal Sensors> on cpu0
coretemp1: <CPU On-Die Thermal Sensors> on cpu1
coretemp2: <CPU On-Die Thermal Sensors> on cpu2
coretemp3: <CPU On-Die Thermal Sensors> on cpu3
tun1: changing name to 'ovpns1'
tun5: changing name to 'ovpns5'
tun2: changing name to 'ovpns2'
igb2: link state changed to DOWN
igb0: link state changed to DOWN
em0: link state changed to DOWN
em0: link state changed to UP
igb2: link state changed to UP
igb0: link state changed to UP
igb1: link state changed to DOWN
pflog0: promiscuous mode enabled
igb1: link state changed to UP
ovpns1: link state changed to UP
ovpns5: link state changed to UP
ovpns1: link state changed to DOWN
ovpns1: link state changed to UP
ovpns5: link state changed to DOWN
ovpns5: link state changed to UP
pid 94772 (unbound), uid 59: exited on signal 11
em0: permanently promiscuous mode enabled
em0: link state changed to DOWN
igb0: permanently promiscuous mode enabled
igb0: link state changed to DOWN
igb1: permanently promiscuous mode enabled
igb1: link state changed to DOWN
igb2: permanently promiscuous mode enabled
igb2: link state changed to DOWN
em0: link state changed to UP
igb0: link state changed to UP
igb1: link state changed to UP
igb2: link state changed to UP
ovpns1: link state changed to DOWN
ovpns1: link state changed to UP
ovpns5: link state changed to DOWN
ovpns5: link state changed to UP
pid 36455 (unbound), uid 59: exited on signal 11
[HBSD SEGVGUARD] [unbound (36455)] Suspension expired.
-> pid: 36455 ppid: 1 p_pax: 0x850<SEGVGUARD,ASLR,NODISALLOWMAP32BIT>
pid 36965 (unbound), uid 59: exited on signal 11
igb3: link state changed to UP
igb3: link state changed to DOWN
igb1: link state changed to DOWN
igb1: link state changed to UP
..didn't check the lights on the network card, sorry...
PS: re. the unbound error see
https://forum.opnsense.org/index.php?topic=7811.msg50139#msg50139
I downgraded unbound to 1.8.1, as described in the thread above.
-
Strange about the Unbound issue. There was another thread regarding lost IP, but there were no changes in interface code.
Can you try flipping this commit instead?
https://github.com/opnsense/core/commit/3d8fd0088a
# opnsense-patch 3d8fd0088a
Cheers,
Franco
-
PS: Unbound crashing would imply some sort of manual DoH or TLS implementation?
-
Hi Franco!
See link above, using unbound with TLS (LibreSSL) results in these crashes, that's why I stayed at 1.8.1 until today, until I updated to 18.7.10 this morning. Now I'm back to 1.8.1
RE the patch: I will update to latest unbound and then apply the patch, correct? :-)
-
Patch with 18.7.10, Unbound version shouldn't matter.... 1.8 has been weird so far... :/
-
Hello,
How does userspace service 'breaks' ethernet traffic? I don't understand privileges programs in freebsb, but you saying a bug in unbound will break ethernet?
-
I say this because this seem a very easy dos attack? no
-
OK, patched and unbound restarted. Wait'n see
-
OK, the interface is down again. Nothing in the General log, nothing in dmesg. Lights are on at the physical interface.
However, this time pulling RJ45 and plugging in again doesn't bring up the interface:
Jan 11 17:56:07 sudo: ASDFGH : TTY=ttyu0 ; PWD=/home/ASDFG ; USER=root ; COMMAND=/sbin/dmesg
Jan 11 17:55:59 opnsense: /usr/local/etc/rc.newwanip: On (IP address: 10.45.23.1) (interface: iNET[opt1]) (real interface: igb1).
Jan 11 17:55:59 opnsense: /usr/local/etc/rc.newwanip: IP renewal is starting on 'igb1'
Jan 11 17:55:59 opnsense: /usr/local/etc/rc.linkup: Hotplug event detected for iNET(opt1) but ignoring since interface is configured with static IP (10.45.23.1 ::)
Jan 11 17:55:59 kernel: igb1: link state changed to UP
Jan 11 17:55:55 opnsense: /usr/local/etc/rc.linkup: Hotplug event detected for iNET(opt1) but ignoring since interface is configured with static IP (10.45.23.1 ::)
Jan 11 17:55:55 kernel: igb1: link state changed to DOWN
Jan 11 17:55:18 sudo: ASDFGH : TTY=ttyu0 ; PWD=/home/ASDFGH ; USER=root ; COMMAND=/sbin/dmesg
OK, pulled the RJ45 a second time, this time the interface came back. Does this make sense at all?
-
> How does userspace service 'breaks' ethernet traffic?
I'm merely saying it could meddle with interface reload if e.g. the service hangs while we try to reconfigure an interface.
There is no Ethernet breakage. If you try to go through a door but the door is stuck you're not crashing the door but you don't get anywhere also. ;)
Cheers,
Franco
-
Any VIPs configured? IPS on that interface? I have no idea why the NIC goes down.
Cheers,
Franco
-
No VIP, IPS on this interface: Yes.
The whole interface is only populated by a single client, which basically does web browsing, which is accessed via VNC from a client on a different interface of the same sense install. So not THAT much to do.
Was running stable for weeks, before going down twice today after update. Should I reverse to 18.7.9 and see how that works?
-
18.7.10 added Suricata 4.1... if it runs in IPS mode it could do things to the link...
# opnsense-revert -r 18.7.9 suricata
(restart suricata)
Cheers,
Franco
-
In the suricata log there is nothing. and also no alerts for this interface at that time...
will revert and see....
-
All stable here now :-)
-
Hmmm, I'll push this to the Suricata guys for help... I assume that without IPS mode it's ok on 4.1.
Makes more sense than Unbound having to do with it. ;)
Thanks,
Franco
-
Unbound is simply killed off if I try something after 18.7.7 (but never had a a look at 18.7.8 though...) with DNS over TLS and LibreSSL (was stable with OpenSSL, iirc). This naturally "kills the internet" (as my users complain) completely and on all interfaces.
This one dying interface yesterday started with 18.7.10 (installed yesterday in the morning), however, suricata IPS is running on 3 interfaces on this box, with another interface under much heavier load, but never going down. Can it be something with the VNC traffic to/from the interface which has problems? No idea...
-
Too hard to tell at this point. But Suricata is the only service that can stop packet flow or somehow bring an interface down/up due to the IPS mode which hooks into the network stack (and this already causes a down/up).
I just don't know whether Unbound DoH and TLS is ready for prime yet seeing all these reports of crashes.
Cheers,
Franco
-
Unbound >1.8.1 using DNS over TLS PLUS LibreSSL is the combination that lets unbound crash every 10-20 min.
I have two installs with OpenSSL und the latest Unbound doing just fine with DNS over TLS (same config as on unbound crashing with LibreSSL).
But two installs with LibreSSL don't like Unbound >1.8.1 with DNS over TLS.
DNS over TLS or DNS over HTTPS should be standard imho ;-)
-
Hello,
I'm Christophe from Paris. For your information, i've the same behavior on a WAN IF.
Every 9 mn the WAN GW is unavailable. Just a SAVE and an APPLY on WAN interface parameter panel (or physical disconnect/reconnect) restores the data flow.
(To confirm that issue is under OpnSense, I've tested directly with a laptop connected to the FO PON and i haven't issue.)
All the parameters like LRO, TSO, EEE are correctly set. I've perform a test with a different NIC, and same issue.
I've perform this command 'opnsense-revert -r 18.7.9 suricata' and reboot. Despite this, the bad behavior still remains. The Service Intrusion Detection is not enabled.
Add-on : opnsense-revert -r 18.7.7 unbound. The issue is always present.
I feel, we have the same issue. If you prefer, i can open a specific topic. Please let me know.
I've this contrab task to workaround the issue.
(https://laclairiere91240.fr/Capture.JPG)
Best regards and thank you by advance for your advises.
Christophe
-
I removed the GeoIP blocking rule (see here: https://forum.opnsense.org/index.php?topic=11020.0)
and updated suricata. Reboot.Wait'n see.
(@Christophe: As your problems persist even after downgrading suricata I would assume you have a different problem)
-
Interface went down again. No log entries. Pulled Rj45, waited 10 sec. plugged in again, interface is online again.... :-(
Downgrading again.