Very confusing incoming blocked logs

Started by opn69a, May 26, 2022, 03:23:25 AM

Previous topic - Next topic
May 26, 2022, 03:23:25 AM Last Edit: May 26, 2022, 03:27:03 AM by opn69a
Hi,

Last night, one of my alerts went off for a major number of blocks on the firewall (twice). When looking at these logs, I'm completely... confused. The source IP addresses show some external locations that are obviously not from me, but the destination IP address is also not my home IP address nor any of my VPN addresses. Also interesting, is MOST of the calls were reaching to the same destination port, 16393 (which I do not have open nor use on my network).

The most fascinating part of this was the destination IP does still belong to the ISP that I use and within the same area. However, I have  static IP address with this ISP (and have defined it as so in the opnsense rules), so it being changed to something else shouldn't have happen.

Here's a sample log that came through into syslog:
```
filterlog[94512]: 12,,,02f4bab031b57d1e30553ce08e0ec131,igb0,match,block,in,4,0x58,,52,51068,0,none,17,udp,1136,[[EXTERNAL NETWORK IP]],[[IP BELONGING TO SAME ISP, BUT NOT MINE]],57992,16393,1116
```
(just replaced the IP addresses for privacy reasons).

Over 18,000 network calls came in to the respective ports over 2 different minute times, 17 minutes apart from each other.

Has anyone seen anything like this or know how I can dig deeper into knowing what happen outside of the syslogs that came through? I'd contact my ISP to see if something happen, but I'm still at a loss on 1.) how they'd know anything and 2.) why my IP address would change when it's been statically defined for nearly 2 years now.

EDIT: Oh and just to mention it, the ISP ip address wasn't just one, but a range of them. It appears to have fallen into some /24 subnet or something. There were a total of 5 addresses used (last digit changed). None of the addresses even start with the same digits as mine so it wasn't obvious to me they even belonged to my ISP at first.

Only thing I can say is that port 16393 udp belongs to Apple FaceTime, maybe DOS, Portscan or misconfigured ISP Routers ...
OPNsense: Intel Core i5-6500, 16 GB RAM, 2x 120GB SSD ZFS-mirror, 4x Intel i350-T4

I often see erroneous log messages, and think the logging GUI may be broken.

E.g., the log will show traffic from my local 192.168.x.x to a public IP being blocked by the default deny/state violation rule. That should trigger on WAN in traffic only though, so something is amiss. (I reckon the display is erroneous - this error seems prevalent after changing the firewall rules)

Quote from: _Alchemist_ on May 26, 2022, 11:07:01 AM
Only thing I can say is that port 16393 udp belongs to Apple FaceTime, maybe DOS, Portscan or misconfigured ISP Routers ...

Yeah I saw that as well. But I don't have an ISP router/modem attached to the network (sitting in a box in the closet). It's just the OPNSense box plugged into the ISP's Cat5e cable and it basically acts as the modem in this case.

Today, it appears I'm seeing these logs come through again but "outgoing". This time port 138 over udp. It says the source and destination are both within the same subnet and both source/destination ports are 138. And same ISP, but my home IP address hasn't changed (blocking is happening on the WAN port but there are no surrounding logs indicating anything is causing these to happen).

At this time, those netbios calls appear to be happening every 10 minutes or so and started all happening at 3:30am

I'm kind of in agreement with abulafia that something must be up but idk. Seems weird that these addresses are from the same ISP and same location as myself. But I don't understand why the source and destination addresses would be addresses not part of my own network lol.

Not to bump this or anything, but some updates:

The odd requests stopped 5/27 for a few days and then started up again this morning a little after 4am.

The "source" address for some  of the requests are DNS servers: 1.1.1.1, 8.8.8.8
The "source" ports are always: 53, 443, 5060, 5061, or 5228 - outside the exception of one which was 27900
The "destination" address is the IP address that belongs to my ISP but is not mine
The "destination" port is always some random high port - outside the exception of one which was 443 (same as the one w/ source 27900, so they seemed to have flipped?)

Some of the "source" and "destination" addresses are still both addresses that belong to the ISP's subnet that I mentioned previously

I'm trying to find a pattern in these requests. The first 4 waves of requests this morning were once an hour, and then there was a pause for 2.5 hours. Then it happened 2 more times (1 hour apart) and hasn't happened again quite yet for the past 2 hours.

At the time I initially created this thread, there were two large waves of requests mostly matching the types of requests I mentioned above (port 443 being the popular one, but many requests being kinda random) - 5/24 @ 9pm. Then it was silent again until 5/26 @ ~3am. From there, there were 2 requests every 10 minutes for almost exactly 24 hours for port 138 (netbios)... then it just stopped and hasn't started up again. Both source and destination being the ISP address that's not a part of my network.

To reiterate something I mentioned earlier - I do not have the ISP modem attached to my network anymore. My OPNSense firewall acts as the modem as it has the ISP's cat5e cable plugged directly into port0 and then handles all that before going to my network.


I noticed there was an update to 22.1.8 on 5/25... This ALMOST lines up with the timeline when this started happening, but is slightly off since this all started the night of 5/24 (and my update cron is daily at 3am), so I'm not really sure they can be related or not. But at the same time, I haven't seen this in the past 2 years before and have had the same static IP from my ISP since starting all this. And yes, I've rebooted and checked for additional updates + plugin updates manually. I did not log into my firewall at all on 5/24 nor before that for several days, so no changes were made from me personally that could have caused this unless it was over a week before this started happening.

------------------------- PACKET CAPTURE -------------------------

So, after getting frustrated and scratching my head on all this constantly, I decided, "...Maybe I can do a packet capture and see exactly what's going on here?". Here's what I found:

There are TONS of ARP requests coming from the IP address subnet that I have been trying to solve. Looking at the 'source' in wireshark, it appears it's actually the mac address that belongs to my ISP (if I understand the ARP Table in OPNSense anyway). Mixed in with these ARP requests are TCP/UDP requests where the source and/or destination falls within the subnet doing these ARP requests... I'm unfamiliar with ARP spoofing/attacks in general, but is this a sign of someone attempting an ARP attack or something like that?

Outside that... I waited until the calls came through again and ran another packet capture (setting 'count' to 0 so there wasn't a limit on how many it captured)... And I found some REALLY interesting results.
There were a few ports for incoming/outgoing that were 80 and 5090, so I decided to inspect these in Wireshark. Apparently I'm picking up on packets that don't even belong to me! For the privacy of whoever this data belongs to, I'm filtering out their company name and phone number (ip address appears to be a private one), but here's one of the packets I picked up in the packet capture:

SIP/2.0 200 OK
Via: SIP/2.0/TCP 172.28.209.100:5115;branch=[[some hash value]];received=[[ISP subnet IP address]]
To: <sip:[[phone number]]@sip.ringcentral.com>;tag=[[some small hash]]
From: "[[company name]]" <sip:[[phone number]]@sip.ringcentral.com>;tag=[[some other small hash]]
Call-ID: [[some hash value]]
CSeq: 31 REGISTER
Contact: <sip:[[phone number]]@172.28.209.100:5115;transport=tcp>;expires=45;methods="INVITE,ACK,BYE,CANCEL,OPTIONS,INFO,MESSAGE,SUBSCRIBE,NOTIFY,PRACK,UPDATE,REFER"
Content-Length: 0

So... Seeing this kind of details in these packets that don't technically even belong to me - is this normal? Does this kind of traffic flow through the internet all the time? Or is this an actual vulnerability that the ISP is introducing to those who are on their network (glad I'm on a VPN!)? It almost feels like my network's playing "man-in-the-middle" without me actually doing anything. I'm really tempted to call my ISP at this rate... But would love to hear input from others, if there is any, before I go that route.

Other things I'm seeing: Apparently data that belong to NVRs/cameras, some HTTP request to gstatic... No DNS requests this time, but wouldn't doubt it if some came in if I ran a packet capture again.

During a DDOS attack, some routers or switches may become so overwhelmed with packets that they run out of ARP cache memory and go from ARP mode to bridged (broadcast) mode.

Usually, the network equipment of your ISP would isolate all customer's IPs despite they are all on the same subnet and would form a broadcast domain if connected to a switch. Effectively, you should never see any packets that are not destined for you (ideally, you should not even see any boardcasts that give away information).

But when the ISPs equipment goes into broadcast mode, any customer can see any other customer's packets. There are a variety of possibilities why this could happen - DDOS attacks and underprovisioning on your ISP's part being among them.
Intel N100, 4 x I226-V, 16 GByte, 256 GByte NVME, ZTE F6005

1100 down / 440 up, Bufferbloat A+