Messages

[Swos]

Hey cookiemonster!

They were great. Perhaps you would have as much luck.

So I talked to one of my ex-coworkers and he was kind of a MikroTik Guru (at least in my world) and I asked him to explain things like to a five year old child. So he liked the metaphor with club and bouncers/security guards. So here's what he told me:

TL;DR version - scroll down
Imagine a huge Club named "Swos" with multiple rooms where different music is played. One plays techno, other plays disco, another one metal and another one plays 50s-80s classics.
This club works 24/7 and collaborates with smaller clubs and they agree on working together on reusing the same tags/wristbands so visitors can use the bands together in a network and share the entry ticket profits. There's a small club called "PC" around the corner that doesn't use bands at all if a free-for-all club, there's also club two blocks away called "AP" which also has couple of music rooms but it's not as big as Swos capable of having 4096 rooms playing all together. And there's also one club called "Admin" - a rather small one but also uses wristbands and tags visitors.

This Swos club has bouncers and security guards all over the place and here is who is who:
VLAN Receive
You approach the Club and you notice a big bouncer guy - they call him by this nickname VLAN Receiver. He is in close contact with the Swos owner and lets people in based on the owner's instructions. On a normal, casual day like Tuesday they let in everyone cause people don't go to clubs on weekdays. On these days you can hear see the little badge on VLAN Receiver's jacket with word "any". On this day he will let anyone in. With the bands on their wrist coming from the AP Club or Admin Club and those without the bands coming from the PC Club too.

Usually on the weekend days Swos Club is very popular and everyone would like to come in but there's not enough space for these people so VLAN Receiver has the badge "only tagged" and accepts people from the AP and Admin Club only up until around midnight or so. After midnight there's a little more space in the club so usually he changes the badge back to "any" so those from PC Club can enter without the tickets too.

There was one time when Swos Club owner had a beef with AP Club owner. On that period VLAN Receiver was having a badge "only untagged" and he was accepting only those from PC Club so he wouldn't share profits with the AP Club dude. Unfortunately people from Admin Club were also affected by this decision.
VLAN Mode
This guy was a gate security guy inside the club after VLAN Receive did let you go through. He also was using badges with information.

When VLAN Mode had a badge "disabled" he basically didn't care for the tags/bands. He didn't check if your tag is on the list however he was taking off any tag you had on your wrist and did let you chose the music room you wanted to enter.

But most of the weekdays by default VLAN Mode had a badge called "optional" and you could chose any music room you wanted and he didn't took off your tag.

During weekend days before 6 PM and after midnight the traffic was higher but not critical. So VLAN Mode had a badge "enabled" and he was checking if your tag was on their so-called VLAN list. If your tag was on the list - you can enter any room you want.

But the worst moments were the weekend days between 6 PM and midnight. VLAN Mode had a badge "strict". They did let you through but they also checked if you're permitted to enter the most occupied music rooms. They were the most crowdy and VLAN Mode was checking if you're on their VLAN list and if you're even permitted to enter the room.
Default VLAN ID
One of the VLAN Mode's assistant was Default VLAN. He usually was giving you a new Default tag only when VLAN Mode was having badge "disabled". You could keep the old tag though.
Force VLAN ID
This guy was kind of easy if he was wearing "no" badge. Along with Default VLAN ID he was giving out new tag if Receiver and Mode let you in without a tag and told you to wear it.

But if he had "yes" badge? Oh boy... He took your tag away and gave a new one. No matter if you had a tag or not. No discussion - you had to had a new tag. Period.
TL;DR

• VLAN Receive comes first to analyze if the frame is tagged or not and based on the value
• ANY: Let any traffic in - tagged or not. No drops.
• ONLY TAGGED: Only tagged comes in - untagged traffic is dropped.
• ONLY UNTAGGED: Only untagged comes in - tagged traffic is dropped.
• VLAN Mode comes second to check if the tag is on the VLAN Table
• DISABLED: No checking against VLAN Table but removing all existing VLAN tags. Default VLAN ID becomes powerless as VLAN Mode disabled blocks all VLAN operations.
• OPTIONAL: Both tagged and untagged comes in (usually with Receive==any). Tagged traffic keeps the tag even if it's not on VLAN Table.
• ENABLED: Traffic needs to be on the VLAN Table.
• STRICT: Traffic needs to be on the VLAN Table and checks if this port actually belongs to the VLAN this traffic is tagged with.
• Default VLAN ID is only affecting untagged traffic
• If VLAN Receive==any: Tagged traffic is unaffected, Untagged gets the Default VLAN ID value.
• If VLAN Receive==only tagged: Tagged traffic is unaffected, Untagged doesn't even reach this point - it gets dropped by VLAN Receive
• If VLAN Receive==only untagged: Tagged traffic never gets processed - it gets dropped by VLAN Receive. Any other (untagged) is tagged now by Default VLAN ID value.
• Force VLAN ID works only with Default VLAN ID
• NO: Every tagged traffic keeps own original tag. Untagged is forcefully tagged with our Default VLAN ID but it's not Force VLAN ID's job
• YES: Everyone gets the new tag of Default VLAN ID. Any existing tags (or non existing if Receive==only tagged or Receive==any) are overwritten forcefully.

Yes you disable the parent and rely on the VLANs hanging off it. It is a container yes, but as unassigned, it does not need any atribute like an IP. See how it looks like on mine:

Awesome! Will try this one out. Thank you! <3

It is good to clash and exchange knowledge :)
MikroTik needs more documentation and more tutorials though...

[Port 24]

Hey cookiemonster!

Thank you for the great effort of talking to MikroTik community and explaining the way how MT understands and treats Native VLAN concept.

So in other words let me ask you if I understand this correctly:


Previously I had Cisco switch with following Port 24 config:

Code Select Expand

interface GigabitEthernet1/0/24
  description Link-to-OPN
  switchport trunk encapsulation dot1q
  switchport trunk native vlan 1337
  switchport mode trunk
  switchport trunk allowed-vlan 5,100,1337

And that basically was basically doing it's job. My Mgmt VLAN was "5" and I had:

Code Select Expand

interface GigabitEthernet1/0/22
  description Mgmt
  switchport access vlan 5
interface Vlan5
  ip address dhcp

With this my switch was getting IP from OPN's DHCP pool of vlan0005 (Parent: igb0, Tag 5). Then my PC connected to Gi1/0/22 was always getting next available IP from the 10.0.5.1/24 pool.

Today (thanks to your huge research and communication with MikroTik's community) it seems that SwOS on CSS326 doesn't have an option to set up a Native VLAN.
Instead you configure Port 24 on CSS326 (according to SwOS wiki) to this:

Code Select Expand

VLANs tab:
  VLAN 5, Name: Mgmt, Port Isolation: True, Learning: True, Mirror: False, IGMP Snooping: False, Members: 22, 24
  VLAN 100, Name: WiFi, Port Isolation: True, Learning: True, Mirror: False, IGMP Snooping: False, Members: 1,2,3,4,5,6,7,8,24  # let's say I have eight APs
  VLAN 1337, Name: Native, Port Isolation: True, Learning: True, Mirror: False, IGMP Snooping: False, Members: 22, 24

VLAN tab:
  Port 1-8, VLAN Mode: strict, VLAN Receive: only untagged, Default VLAN ID: 100
  Port 22, VLAN Mode: strict, VLAN Receive: only untagged, Default VLAN ID: 5
  Port 24, VLAN Mode: strict, VLAN Receive: only tagged, Default VLAN ID: 1337

Some additional questions regarding of how MikroTik understands ingress and egress:



For port 22 where my laptop is connected to CSS326 the ingres is red and egress is green, right?
Same thing for port 24 where OPNsense is connected. Ingress is blue, egress is purple, right?

Now is there a concept of ingress/egress distinguishing of traffic inside the switch and between the ports? In that case:
For port 22 the ingress from port 24 would be pink, and egress to port 24 would be orange, and vice versa from the port 24's perspective the ingress would be orange and egress would be pink. Am I understanding this right?

The reason why I ask is the configuration above (VLAN tab) says the following for the Port 22:

• VLAN Mode: I will filter all red and pink traffic and if it's not Tag=5 then I'll drop it. But PC is not member of VLAN5, only dude that is member of VLAN5 is Port 24 so I'll listen to the incoming traffic from him only.
• VLAN Receive: I will only allow incoming packets without VLAN tag. Since the only dude that is not sending me tags is PC - I will accept untagged traffic from him only.

This is very confusing for me in the MikroTik world. Contradictory terms. I would like to understand what's the point of VLAN Mode and VLAN Receive. Both explanations are talking about ingress and egress but in the VLAN Configuration Example Wiki Ether 6 would be my WiFi (VLAN100) case, Ether 8 would be my Mgmt (VLAN5) case.
Having a second thought I understand this MikroTik's VLAN Mode terms as follows:

• VLAN Mode optional: Don't care about VLAN tags but if your ID is wrong I'll give you a proper one.
  
  • You don't have VLAN Tag? I'll let you go through.
  • You have it and it matches my VLAN table? You're most welcome.
  • You have a tag that is not in my VLAN table? Oh that's not a problem, let's pretend that you're untagged and I'll tag you as my Default VLAN ID
• VLAN Mode disabled: VLAN table? Waaaat?
  
  • You don't have the Tag? I'll let you go through.
  • You have it? I'll let you go only if it matches my Default VLAN ID and my buddy VLAN Receive says "any" but your tag stays with me and you go naked from now on.
• VLAN Mode enabled: Only tags that I know go through.
  
  • You coming to me with the Tag I don't know, get out of here!
  • Your tag matches my Default VLAN Tag! That's awesome dude! Cause now I can act like a proper access port! But leave the Tag here and go naked, will you?
• VLAN Mode strict: Jokes are over - hand over your passport, reason for travelling, destination and where you come from?
  
  • I don't know your Tag, it's not on my VLAN table. Get lost!
  • Got the Tag? Good. It's on my list and the dude that sent you is also member of the VLAN of your Tag - you can go.

In other words VLAN Mode is an inside club security guy that checks your ID before letting you go into the party.

And for VLAN Receive terms I think it is like another security guy but standing outside filtering and funneling the incoming guests so the VLAN Mode guy have less work to do:

• VLAN Receive: any
• Oh whatever... Come in - VLAN Mode will handle you
• VLAN Receive: only tagged
• Only Tags! And tell the VLAN Mode guy that you already have one so no need to give you a Default VLAN ID
• VLAN Receive: only untagged
• Come in naked people - we'll give you a Default VLAN ID

Having said that in a humorous way I think that ingress from OPNsense is already tagged with VLAN 5 so Port 24 must be:

VLAN Mode: strict
VLAN Receive: only tagged
Default VLAN ID: doesn't really matter cause VLAN Receive and VLAN Mode handles only tagged frames.
Force VLAN ID: Also doesn't matter. Only if the ingress comes with a tag and Default VLAN=1 it will basically strip all egress from tags and we don't want that cause OPNsense needs the tags.


Please let me know how far I am from fully understanding it.



As for mixing VLANs in OPNsense. Do I understand it right that I need to disable LAN0(igb0) DHCP and rely only on Mgmt(VLAN5) and WiFi(VLAN100) DHCP since they are the children of this physical interface? In other words - physical interface is like a container that only has an IP of 10.0.0.1 and it can even be 10.0.0.1/32 so OPNsense can handle the traffic, right?

First of all, do not mix untagged and tagged VLANs on OPNsense. Just dont, it may work but it may bring problems down the road.
https://docs.opnsense.org/manual/how-tos/vlan_and_lagg.html

I apologize but I don't understand. Where do I mix VLANs on OPNsense?

Second, In order to access a switch management in a VLAN, you need to tell that switch to have its management on a VLAN, for that consult the official Vendors documentation. (System TAB)

Once I put value "5" in Allow From VLAN I immediately lose connectivity to the Switch, to OPNsense etc.

[OPNsense:]

Hey!

So here's the problem. I want to set up a Management VLAN for the OPNsense and MikroTik CSS326-24G-2S+RM Switch.


OPNsense:

Iface	Physical iface	Virt. iface	DHCP
WAN	re0		IP assigned by ISP
LAN0	igb0		ISC-DHCP pool 10.0.0.1/24
LAN1	igb1		ISC-DHCP pool 10.1.0.1/24
Mgmt	Parent: igb0	vlan0005	ISC-DHCP pool 10.0.5.1/24
WiFi	Parent: igb0	vlan0100	ISC-DHCP pool 10.0.100.1/24

As you already see second octet is a physical interface, third one is VLAN (zero if not a VLAN).

CSS326 Switch (SwOS 2.17):

- Port 24 connected to LAN0(igb0)
- Port 22 connected to PC


CASE #1 (Switch has no config)
Starting both devices. My PC (connected to Port24) gets IP 10.0.0.2/24. I can access OPNsense GUI/SSH with no problem.
Switch only accessible if I manually assign IP address to my PC: 192.168.88.2 cause by default Switch is having 192.168.88.1

CASE #2 (Attempting to config Switch)
My intention is to have port 22 and 23 as Mgmt ports, one for my PC and the other one for the console. So in VLANs tab in SwOS I create two VLANs:

• ID: 1, Name: Default, all ports are members of this VLAN
• ID: 5, Name: Mgmt, all ports are members of this VLAN

Everything is working so far. I also need to go to VLAN tab where I have all ports listed with their default values:

• VLAN Mode: optional
• VLAN Receive: any
• Default VLAN ID: 1
• Force VLAN: unchecked

Since port 24 is going to be a trunk port then I set it up as follows:

• VLAN Mode: enabled
• VLAN Receive: any
• Default VLAN ID: 5
• Force VLAN: unchecked

After saving the config I lose my access to both OPNsense and switch


CASE #3 (Attempting to give Switch IP from Mgmt pool)
OPNsense has already everything configured. On SwOS I go to System tab and in the field named "Allow From VLAN" I put "5" cause thats the VLAN tag for the Mgmt interface on OPNsense, right?
Immediately after saving the config the connection is lost.

Questions:
I am upgrading my old Cisco switch where I was able to get IP from Mgmt DHCP pool. Port 22 and 23 was configured as Mgmt port on the switch but somehow I cannot understand how to do this on MikroTik?
What am I doing wrong?

How to get the following setup?

• Switch gets IP from Mgmt pool (10.0.5.2)
• Ports 22 and 23 are Mgmt ports
• Clients from Ports 1-16 get IPs from WiFi pool (10.0.100.1/24)

Seems like I don't understand how MikroTik switch is handling VLANs

Keep an eye on this; I'm assuming that once it's stable for you over some period of time, you can flag this thread as solved. It may come in handy for somebody else who will hit a similar issue.

11 days and no issues so far. Thread topic updated and case closed.

Thank you guys for your assistance and help in understanding dpinger.

So last night I've done something that I was thinking about it long before I've dug into dpinger and other activities helping me understand how's the connection between my OPNsense and ISP modem working.

So until last night my OPNsense instance was more than a year old. I mean it was updated all the time whenever new (full, not dev) version was released. But I installed it having version 21. Today we have 23.

And you know what? Everything simply went away!

Someone might say that I've probably messed up with the config but ummm - no, I don't think so. Before the clean install I've downloaded full config and took literally all screens from the GUI.

Once fresh install was done I did reproduce all setting based on the screenshots I've had from previous OPNsense install. 12 hours and no problem at all. So then I've backed up the manually (screen & click) recreated settings, restored the backup settings from the old install and when another 12 hours of testing were ongoing I've compared the XML (config) files in the mean time searching for any differences I could probably have doing the screen&click configuration. Basically no difference if you ask me.

And guess what? Another 12h test and nothing.

Interesting.  I've not looked that closely at the dpinger results.  Did you manually set your GW or did you let dpinger automatically get it?

GW is set to DHCP and that's it. Nothing else:



Other than that - here's what I had configured before Seimus and other guys suggested to remove (if there's no address then GW's IP is pinged by default):

Thank you Guys for the extensive knowledge I gained here.

I will definitely test it out. As a matter of fact I've removed 8's from my dpinger and now by default it is pinging my WAN_GW.

When the problem occurred again I've noticed something interesting.

Code Select Expand

<12>1 2023-05-19T02:28:33+02:00 fw.domain.it dpinger 28711 - [meta sequenceId="119"] WAN_DHCP xxx.xxx.12.1: sendto error: 64
<12>1 2023-05-19T02:28:34+02:00 fw.domain.it dpinger 28711 - [meta sequenceId="120"] WAN_DHCP xxx.xxx.12.1: sendto error: 64
<12>1 2023-05-19T02:28:35+02:00 fw.domain.it dpinger 28711 - [meta sequenceId="121"] WAN_DHCP xxx.xxx.12.1: sendto error: 64
<12>1 2023-05-19T02:28:36+02:00 fw.domain.it dpinger 28711 - [meta sequenceId="122"] WAN_DHCP xxx.xxx.12.1: sendto error: 64
<12>1 2023-05-19T02:28:37+02:00 fw.domain.it dpinger 28711 - [meta sequenceId="123"] WAN_DHCP xxx.xxx.12.1: sendto error: 65
<12>1 2023-05-19T02:28:38+02:00 fw.domain.it dpinger 28711 - [meta sequenceId="124"] WAN_DHCP xxx.xxx.12.1: sendto error: 65
<12>1 2023-05-19T02:28:39+02:00 fw.domain.it dpinger 28711 - [meta sequenceId="125"] WAN_DHCP xxx.xxx.12.1: sendto error: 65

When my GW goes down at fist I get error 64 which means:

64 EHOSTDOWN
Host is down.
A socket operation failed because the destination host was down.

And after a while E65 kicks in.

Which to my simple logic - ISPs CMTS (my bridge modem is a DOCSISv3 one) probably goes down and then when it gets up it tries to converge the network protocols again but it is responding. Hence why it's "EHOSTDOWN" first cause it's literally down, then it gets up but not ready yet and then it's "EHOSTUNREACH". Am I connecting the dots in the right way?

This is why I monitor the gateway ip and not some public ip.  By monitoring google dns you're adding in all kinds of additionally variables into the connection check.

Change dpinger to use the gateway and that should resolve your issue.

I'll run in this mode for a few hours/days.


One followup question though:
Once dpinger marks GW as down - how to mark it as online again besides flapping the WAN iface? I know that restoring connectivity is the proper way. I'm just asking for like a forced/ad-hoc restart and "tell" dpinger to stop treating my GW as down and perform the tests again. And I'm looking for a solution from CLI, not from OPNsense GUI.

[sendto error: 65]

Hello,

Firstly what is your setup? Do you have Dual WAN?

Single.

Because from the logs you post you monitor your GW via dpinger but you are actually probing google DNS. My question here is why? Why not probing the GW of your ISP?

Why not DNS? What if my ISP GW sucks and drops me every few minutes and I need to prove this? I need to reach beyond the ISP's GW. I literally need to see why I get these Internet cutouts.

1. Check your WAN interface for errors or problems
2. Check your Cable from your WAN to Telco device

Tested multiple times. Don't think I can think of about any other test.

sendto error: 65

65 EHOSTUNREACH
No route to host.
A socket operation was attempted to an unreachable host.

Either there is no possible route to the target locally, or status information was received from an upstream router that indicated the same condition elsewhere along the path to the target.

This can happen due to a lack of default route, missing interface link route, or similar conditions.

And here lays the problem. After I flap my WAN iface I do get this route working. Why suddenly route is being lost?

I literally have no idea why this is happening. How can prove my ISP that they eF-ed up something? OPNsense boots up, gets the IP from their DHCP (ISP modem is in bridge mode) and it works. Until... It stops. Then I need to flap/reboot in order to get it work.
This is exactly why I rely on Google DNS instead of monitoring their GW.

Than you for your extensive input and reply, Seimus nonetheless. You just gave me few other ideas to test things out.

dpinger is a service that does monitoring.

Disabling it makes no sense. Especially when you want to automate things.
Pinging gateway's IP address also makes no sense cause you don't test its own functionality.

The problem is not dpinger. Dpinger is a tool that helps you see that your GW goes down. The problem lays somewhere else.

Hey guys!

I'm experiencing a weird problem lately. Having latest version 23.1.7 my gateway goes offline couple of times a day. Up to even 10-12 times a day.
I blamed ISP at first but did some tests and it doesn't seem like it's them.

But to the point.

Gateway goes offline. I can do one of the two options:
 

   
• Reboot OPNsense
• Flap interface (ifconfig re0 down && ifconfig re0 up)

Both works. Flap is considerably faster.

So here are my two questions:

1. Anyone experiencing this as well? I am not losing DHCP lease. It's still there. Just dpinger marks my WAN_DHCP Gateway as Offline and it will remain until I reboot or flap WAN iface.

2. Any ideas for a quick workaround? I was thinking about small BASH script similar to Ooker's script but in my case I don't lose DHCP. It's just dpinger marks my GW as down and that's it. Until I flap/reboot.

Code Select Expand

<12>1 2023-05-15T13:16:37+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="13"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T13:27:54+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: Alarm latency 13957us stddev 14964us loss 22%
<13>1 2023-05-15T13:27:54+02:00 fw.domain.tld dpinger 40388 - [meta sequenceId="2"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 1 RTT: 13957us RTTd: 14964us Loss: 22%)
<12>1 2023-05-15T13:28:19+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="3"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T13:28:20+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="4"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T13:28:21+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="5"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T13:28:22+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="6"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T13:28:23+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="7"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T13:28:24+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="8"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T13:28:25+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="9"] exiting on signal 15
<12>1 2023-05-15T13:28:25+02:00 fw.domain.tld dpinger 89692 - [meta sequenceId="10"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T13:28:25+02:00 fw.domain.tld dpinger 89692 - [meta sequenceId="11"] exiting on signal 15
<12>1 2023-05-15T13:28:25+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="12"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T13:37:36+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: Alarm latency 532817us stddev 2881356us loss 0%
<13>1 2023-05-15T13:37:36+02:00 fw.domain.tld dpinger 38680 - [meta sequenceId="2"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 1 RTT: 532817us RTTd: 2881356us Loss: 0%)
<12>1 2023-05-15T13:37:52+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="3"] WAN_DHCP 8.8.8.8: Clear latency 342244us stddev 2499381us loss 0%
<13>1 2023-05-15T13:37:52+02:00 fw.domain.tld dpinger 52473 - [meta sequenceId="4"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 0 RTT: 342244us RTTd: 2499381us Loss: 0%)
<12>1 2023-05-15T17:55:16+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: Alarm latency 12785us stddev 13657us loss 22%
<13>1 2023-05-15T17:55:16+02:00 fw.domain.tld dpinger 24791 - [meta sequenceId="2"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 1 RTT: 12785us RTTd: 13657us Loss: 22%)
<12>1 2023-05-15T17:58:09+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:10+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="2"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:11+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="3"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:12+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="4"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:12+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="5"] exiting on signal 15
<12>1 2023-05-15T17:58:13+02:00 fw.domain.tld dpinger 12299 - [meta sequenceId="6"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T17:58:13+02:00 fw.domain.tld dpinger 12299 - [meta sequenceId="7"] exiting on signal 15
<12>1 2023-05-15T17:58:13+02:00 fw.domain.tld dpinger 16931 - [meta sequenceId="8"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T17:58:49+02:00 fw.domain.tld dpinger 16931 - [meta sequenceId="9"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:50+02:00 fw.domain.tld dpinger 16931 - [meta sequenceId="10"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:51+02:00 fw.domain.tld dpinger 16931 - [meta sequenceId="11"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:52+02:00 fw.domain.tld dpinger 16931 - [meta sequenceId="12"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:53+02:00 fw.domain.tld dpinger 16931 - [meta sequenceId="13"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:54+02:00 fw.domain.tld dpinger 16931 - [meta sequenceId="14"] exiting on signal 15
<12>1 2023-05-15T17:58:54+02:00 fw.domain.tld dpinger 41836 - [meta sequenceId="15"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T17:58:55+02:00 fw.domain.tld dpinger 41836 - [meta sequenceId="16"] exiting on signal 15
<12>1 2023-05-15T17:58:55+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="17"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T17:59:31+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="18"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:59:32+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="19"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:59:33+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="20"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:59:34+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="21"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:59:35+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="22"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:59:36+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="23"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:59:36+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="24"] exiting on signal 15
<12>1 2023-05-15T17:59:36+02:00 fw.domain.tld dpinger 89372 - [meta sequenceId="25"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T17:59:37+02:00 fw.domain.tld dpinger 89372 - [meta sequenceId="26"] exiting on signal 15
<12>1 2023-05-15T17:59:37+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="27"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T18:45:50+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: Alarm latency 15729us stddev 24430us loss 22%
<13>1 2023-05-15T18:45:50+02:00 fw.domain.tld dpinger 74095 - [meta sequenceId="2"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 1 RTT: 15729us RTTd: 24430us Loss: 22%)
<12>1 2023-05-15T18:46:59+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T18:47:00+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="2"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T18:47:01+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="3"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T18:47:02+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="4"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T18:47:03+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="5"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T18:47:03+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="6"] exiting on signal 15
<12>1 2023-05-15T18:47:03+02:00 fw.domain.tld dpinger 8158 - [meta sequenceId="7"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T18:47:04+02:00 fw.domain.tld dpinger 8158 - [meta sequenceId="8"] exiting on signal 15
<12>1 2023-05-15T18:47:04+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="9"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T19:01:52+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: Alarm latency 16162us stddev 15845us loss 22%
<13>1 2023-05-15T19:01:52+02:00 fw.domain.tld dpinger 62247 - [meta sequenceId="2"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 1 RTT: 16162us RTTd: 15845us Loss: 22%)
<12>1 2023-05-15T19:03:00+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:01+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="2"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:02+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="3"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:03+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="4"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:04+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="5"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:05+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="6"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:05+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="7"] exiting on signal 15
<12>1 2023-05-15T19:03:05+02:00 fw.domain.tld dpinger 69283 - [meta sequenceId="8"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T19:03:06+02:00 fw.domain.tld dpinger 69283 - [meta sequenceId="9"] exiting on signal 15
<12>1 2023-05-15T19:03:06+02:00 fw.domain.tld dpinger 73146 - [meta sequenceId="10"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T19:03:10+02:00 fw.domain.tld dpinger 73146 - [meta sequenceId="11"] WAN_DHCP 8.8.8.8: Alarm latency 17525us stddev 5156us loss 33%
<13>1 2023-05-15T19:03:10+02:00 fw.domain.tld dpinger 90669 - [meta sequenceId="12"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 1 RTT: 17525us RTTd: 5156us Loss: 33%)
<12>1 2023-05-15T19:03:50+02:00 fw.domain.tld dpinger 73146 - [meta sequenceId="13"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:51+02:00 fw.domain.tld dpinger 73146 - [meta sequenceId="14"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:52+02:00 fw.domain.tld dpinger 73146 - [meta sequenceId="15"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:53+02:00 fw.domain.tld dpinger 73146 - [meta sequenceId="16"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:54+02:00 fw.domain.tld dpinger 73146 - [meta sequenceId="17"] exiting on signal 15
<12>1 2023-05-15T19:03:54+02:00 fw.domain.tld dpinger 89699 - [meta sequenceId="18"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T19:03:55+02:00 fw.domain.tld dpinger 89699 - [meta sequenceId="19"] exiting on signal 15
<12>1 2023-05-15T19:03:55+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="20"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T19:21:16+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: Alarm latency 12331us stddev 4992us loss 22%
<13>1 2023-05-15T19:21:16+02:00 fw.domain.tld dpinger 13437 - [meta sequenceId="2"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 1 RTT: 12331us RTTd: 4992us Loss: 22%)
<12>1 2023-05-15T19:21:49+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="3"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:21:50+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="4"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:21:51+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="5"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:21:52+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="6"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:21:53+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="7"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:21:54+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="8"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:21:55+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="9"] exiting on signal 15<12>1 2023-05-15T13:16:37+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="13"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T13:27:54+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: Alarm latency 13957us stddev 14964us loss 22%
<13>1 2023-05-15T13:27:54+02:00 fw.domain.tld dpinger 40388 - [meta sequenceId="2"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 1 RTT: 13957us RTTd: 14964us Loss: 22%)
<12>1 2023-05-15T13:28:19+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="3"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T13:28:20+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="4"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T13:28:21+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="5"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T13:28:22+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="6"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T13:28:23+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="7"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T13:28:24+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="8"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T13:28:25+02:00 fw.domain.tld dpinger 98753 - [meta sequenceId="9"] exiting on signal 15
<12>1 2023-05-15T13:28:25+02:00 fw.domain.tld dpinger 89692 - [meta sequenceId="10"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T13:28:25+02:00 fw.domain.tld dpinger 89692 - [meta sequenceId="11"] exiting on signal 15
<12>1 2023-05-15T13:28:25+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="12"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T13:37:36+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: Alarm latency 532817us stddev 2881356us loss 0%
<13>1 2023-05-15T13:37:36+02:00 fw.domain.tld dpinger 38680 - [meta sequenceId="2"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 1 RTT: 532817us RTTd: 2881356us Loss: 0%)
<12>1 2023-05-15T13:37:52+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="3"] WAN_DHCP 8.8.8.8: Clear latency 342244us stddev 2499381us loss 0%
<13>1 2023-05-15T13:37:52+02:00 fw.domain.tld dpinger 52473 - [meta sequenceId="4"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 0 RTT: 342244us RTTd: 2499381us Loss: 0%)
<12>1 2023-05-15T17:55:16+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: Alarm latency 12785us stddev 13657us loss 22%
<13>1 2023-05-15T17:55:16+02:00 fw.domain.tld dpinger 24791 - [meta sequenceId="2"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 1 RTT: 12785us RTTd: 13657us Loss: 22%)
<12>1 2023-05-15T17:58:09+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:10+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="2"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:11+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="3"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:12+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="4"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:12+02:00 fw.domain.tld dpinger 93726 - [meta sequenceId="5"] exiting on signal 15
<12>1 2023-05-15T17:58:13+02:00 fw.domain.tld dpinger 12299 - [meta sequenceId="6"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T17:58:13+02:00 fw.domain.tld dpinger 12299 - [meta sequenceId="7"] exiting on signal 15
<12>1 2023-05-15T17:58:13+02:00 fw.domain.tld dpinger 16931 - [meta sequenceId="8"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T17:58:49+02:00 fw.domain.tld dpinger 16931 - [meta sequenceId="9"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:50+02:00 fw.domain.tld dpinger 16931 - [meta sequenceId="10"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:51+02:00 fw.domain.tld dpinger 16931 - [meta sequenceId="11"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:52+02:00 fw.domain.tld dpinger 16931 - [meta sequenceId="12"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:53+02:00 fw.domain.tld dpinger 16931 - [meta sequenceId="13"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:58:54+02:00 fw.domain.tld dpinger 16931 - [meta sequenceId="14"] exiting on signal 15
<12>1 2023-05-15T17:58:54+02:00 fw.domain.tld dpinger 41836 - [meta sequenceId="15"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T17:58:55+02:00 fw.domain.tld dpinger 41836 - [meta sequenceId="16"] exiting on signal 15
<12>1 2023-05-15T17:58:55+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="17"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T17:59:31+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="18"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:59:32+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="19"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:59:33+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="20"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:59:34+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="21"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:59:35+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="22"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:59:36+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="23"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T17:59:36+02:00 fw.domain.tld dpinger 46565 - [meta sequenceId="24"] exiting on signal 15
<12>1 2023-05-15T17:59:36+02:00 fw.domain.tld dpinger 89372 - [meta sequenceId="25"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T17:59:37+02:00 fw.domain.tld dpinger 89372 - [meta sequenceId="26"] exiting on signal 15
<12>1 2023-05-15T17:59:37+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="27"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T18:45:50+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: Alarm latency 15729us stddev 24430us loss 22%
<13>1 2023-05-15T18:45:50+02:00 fw.domain.tld dpinger 74095 - [meta sequenceId="2"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 1 RTT: 15729us RTTd: 24430us Loss: 22%)
<12>1 2023-05-15T18:46:59+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T18:47:00+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="2"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T18:47:01+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="3"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T18:47:02+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="4"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T18:47:03+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="5"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T18:47:03+02:00 fw.domain.tld dpinger 93279 - [meta sequenceId="6"] exiting on signal 15
<12>1 2023-05-15T18:47:03+02:00 fw.domain.tld dpinger 8158 - [meta sequenceId="7"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T18:47:04+02:00 fw.domain.tld dpinger 8158 - [meta sequenceId="8"] exiting on signal 15
<12>1 2023-05-15T18:47:04+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="9"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T19:01:52+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: Alarm latency 16162us stddev 15845us loss 22%
<13>1 2023-05-15T19:01:52+02:00 fw.domain.tld dpinger 62247 - [meta sequenceId="2"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 1 RTT: 16162us RTTd: 15845us Loss: 22%)
<12>1 2023-05-15T19:03:00+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:01+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="2"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:02+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="3"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:03+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="4"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:04+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="5"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:05+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="6"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:05+02:00 fw.domain.tld dpinger 15652 - [meta sequenceId="7"] exiting on signal 15
<12>1 2023-05-15T19:03:05+02:00 fw.domain.tld dpinger 69283 - [meta sequenceId="8"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T19:03:06+02:00 fw.domain.tld dpinger 69283 - [meta sequenceId="9"] exiting on signal 15
<12>1 2023-05-15T19:03:06+02:00 fw.domain.tld dpinger 73146 - [meta sequenceId="10"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T19:03:10+02:00 fw.domain.tld dpinger 73146 - [meta sequenceId="11"] WAN_DHCP 8.8.8.8: Alarm latency 17525us stddev 5156us loss 33%
<13>1 2023-05-15T19:03:10+02:00 fw.domain.tld dpinger 90669 - [meta sequenceId="12"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 1 RTT: 17525us RTTd: 5156us Loss: 33%)
<12>1 2023-05-15T19:03:50+02:00 fw.domain.tld dpinger 73146 - [meta sequenceId="13"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:51+02:00 fw.domain.tld dpinger 73146 - [meta sequenceId="14"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:52+02:00 fw.domain.tld dpinger 73146 - [meta sequenceId="15"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:53+02:00 fw.domain.tld dpinger 73146 - [meta sequenceId="16"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:03:54+02:00 fw.domain.tld dpinger 73146 - [meta sequenceId="17"] exiting on signal 15
<12>1 2023-05-15T19:03:54+02:00 fw.domain.tld dpinger 89699 - [meta sequenceId="18"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T19:03:55+02:00 fw.domain.tld dpinger 89699 - [meta sequenceId="19"] exiting on signal 15
<12>1 2023-05-15T19:03:55+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="20"] send_interval 1000ms  loss_interval 2000ms  time_period 60000ms  report_interval 0ms  data_len 0  alert_interval 1000ms  latency_alarm 500ms  loss_alarm 20%  alarm_hold 10000ms  dest_addr 8.8.8.8  bind_addr xxx.xxx.14.160  identifier "WAN_DHCP "
<12>1 2023-05-15T19:21:16+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="1"] WAN_DHCP 8.8.8.8: Alarm latency 12331us stddev 4992us loss 22%
<13>1 2023-05-15T19:21:16+02:00 fw.domain.tld dpinger 13437 - [meta sequenceId="2"] GATEWAY ALARM: WAN_DHCP (Addr: 8.8.8.8 Alarm: 1 RTT: 12331us RTTd: 4992us Loss: 22%)
<12>1 2023-05-15T19:21:49+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="3"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:21:50+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="4"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:21:51+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="5"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:21:52+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="6"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:21:53+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="7"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:21:54+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="8"] WAN_DHCP 8.8.8.8: sendto error: 65
<12>1 2023-05-15T19:21:55+02:00 fw.domain.tld dpinger 93153 - [meta sequenceId="9"] exiting on signal 15

Any ideas?


I even tried to downgrade my OPNsense to desired version following Shoterboyx's advice but I get errors when downgrading.

I'm experiencing kind of similar issue like you guys however I cannot perform the OPNsense downgrade to a specific version.

Code Select Expand


<truncated output>

Configuring login behaviour...done.
Configuring system logging...done.
=====
Message from opnsense-23.1.7_3:

--
I'm no chicken
Found local copy of 90f1d1d766, skipping fetch.
2 out of 2 hunks failed while patching etc/inc/interfaces.inc
4 out of 7 hunks failed while patching etc/rc.newwanip

Thank you for clarification, Meyergru

However like you said - it requires a trick to get it done anyway. And even if it works - it is kind of non standard solution. Not saying it's bad. It just needs some expertise in the area and time to test it out.

I'm happy though that you left a comment explaining this. Thank you once again! :)

[mission completed!]

So after days of testing here's what I achieved and I consider this as a mission completed! ;D.
Just to remind you all - this is what my current setup is:




OPNsense device:
1x re0 (built in NIC that acts as my WAN iface)
4x igb{0..3} (Intel I350-T4 NIC)

Interface ID: WAN
Assignment: re0
IP: DHCP
Connected to: My ISP router in modem mode

Interface ID: LAN0
Assignment: igb0
IP: 10.0.0.1/24
DHCP pool: 10.0.0.1 - 10.0.0.254
Connected to: Cisco switch

Interface ID: LAN1
Assignment: igb1
IP: 10.1.0.1/24
DHCP pool: 10.1.0.1 - 10.1.0.254
Connected to: UniFi AP U6-LR

igb2: not assigned
igb3: not assigned


Interface ID: Mgmt
Assignment: VLAN01
IP: 10.0.5.1/28
DHCP pool: 10.0.5.1/28

Interface ID: WiFi
Assignment: VLAN010
IP: 10.1.100.1/24

Interface ID: WiFi_IoT
Assignment: VLAN011
IP: 10.1.110.1/24

VLANS:

VLAN ID: Mgmt
Parent: igb0
tag: 5
Device name: vlan01

VLAN ID: WiFi
Parent: igb1
tag: 100
Device name: vlan010

VLAN ID: WiFi_IoT
Parent: igb1
tag: 110
Device name: vlan011



For those that prefer to see screens:






UniFi config:

SSID: WiFi
Network: VLAN100_Network

SSID: WiFi_IoT
Network: VLAN110_Network

Network: VLAN100_Network
Router: Third-party Gateway
VLAN ID: 100

Network: VLAN110_Network
Router: Third-party Gateway
VLAN ID: 110



Desired scenario:
Have AP to get IP from WiFi pool (static lease 10.1.100.2/24) whilst serving two SSIDs that are tagged with separate VLAN IDs so I can separate clients based on their VLAN IDs.

Key takeaways:

Understand that:

• AP is indeed connecting to LAN1 physical interface and it expects the IP from LAN1 DHCP pool as it doesn't communicate using any VLANs. It's a trunk after all. AP will need to have a static IP that is in LAN1's subnet or LAN1 will have to have DHCP enabled in order to assign one.
• Once client is connected to AP's SSID the SSID looks into UniFi Network configuration and if Network has a VLAN tagging every traffic that comes from this client connected to this particular SSID will be wrapped in a VLAN tag.
• If client traffic is wrapped in a VLAN tag then it gets IP from DHCP pool of tat VLAN interface configured on OPNsense and therefore it can be isolated/routed/NATed/whatever on OPNsense level.

And here's the result of it:
https://imgur.com/a/HgfbC7U