BGP network range traffic is blocked by firewall

[SillyPosition]

I have the following setup for some time now, everything used to work perfectly fine:
Couple of hardware nodes running a kubernetes cluster, publishes ip addresses in the 10.50.0.0/24 network range.
all nodes are connected to opnsense and advertises the routes using BGP.
BGP nodes are all connected, established and I can see the routes properly in the routing table.
My LAN network is 10.0.0.0/24.
To configure the BGP 'stuff' I followed this guide

Im running the latest firmware 22.1.1_3, Im not sure anymore whether I had this issue previously with 21.7.
Most of my firewall rules are generic and pretty much the default in LAN network (see attachment)

When I actually try to access a service, direction out is passing, yet direction in is blocked (see attachment)
Im pretty much stuck, I cant leverage 'rid' to investigate the blocking rule because its a default rule.
So I went ahead and tried to setup an explicit rule in the LAN firewall rules to allow exactly such connections, and also added an explicit deny all rule to troubleshoot whether 'rid' will lead to this rule, but it didn't help.

Im not sure how to proceed from here, how do I set explicit firewall rules betwen my LAN network (10.0.0.0/24) and my BGP network (10.50.0.0/24)?

[mimugmail]

This is a RST ACK packet of an already closed connection, this happens from time to time and is not related to BGP

[SillyPosition]

Hi, unrelated to the subject -  I really appreciate your work on the community repo :-)

I discovered this issue since I have a service which I access locally within LAN, and when I try to push data (POST ..., its a web service) I get connection resets.
I see it constantly, all the time.
When looking at the live view of the firewall logs, I see gigantic amount of such occurrances of dropped packets to the BGP network.
I dont see any such packets being dropped if I access remotely via openvpn.
I wouldnt say "from time to time", since the logs are full of such denied packets

[SillyPosition]

Perhps from a different point of view, but how in general do firewalling takes place when the network is internal, yet not part of "LAN"?
Or is the BGP network considered LAN even though the network address range differs?
If I would like to create a block rule for a specific IP on my LAN network, from accessing one of my IPs in my BGP network, how would I define that rule and in what firewall table?

[mimugmail]

Always in inbound direction and on the interface which see the packet first. So when traffic from DMZ to LAN needs filtering in DMZ in inbound direction

[SillyPosition]

What firewall table is responsible for allowing/blocking such traffic between those two subnets?
I keep seeing the issue, not only with reset-ack flags,but ack, fin-ack etc. It constantly reproduces and I have no idea what to look at.
Is there anything else that I can do in order to track these connections and what is blocking them?

[mimugmail]

Again, are you facing any kind of connectivity problem? Also find packets belong to already closed connections which happens when the software still wants to sent a packet but already signaled that its finished. You dont need to accept such a packet only cause you see it blocked

[SillyPosition]

I do, Im just not sure how to explain them explicitly.
I have a specific service running at home, in that BGP subnet, which at the beginning traffic is flowing OK, but then at some point I see alot of TCP retransmissions from my laptop (tracing with wireshark)
Is there a better way to trace this from the point of view of opnsense to maybe gain better understanding what exactly is breaking here?

If I put that same service regularly to listen on the physical hardware and get ip address that belongs to my home LAN, access to it is working perfect - so it removes suspicion of the service itself being problematic.
If I connect from remotely, via openvpn server (on opnsense), and get assigned with an IP that belongs to openvpn subnet, then access to this service is again working flawlessly.
It only happens while Im on LAN IP address range.

If I go to Firewall settings - Advanced, and check "Disable all packet filtering" everything works well, from within LAN network

[mimugmail]

Then your application is broken or not working correctly. You edit your firewall rule and in advanced settings set state to "none". Then it will work, but your Firewall isnt stateful anymore.

[SillyPosition]

How come it is the application that is broken, if the only thing that is causing the troubles is the way I communicate with it via my router?
Doest it mean pretty much that I have a misconfiguration with the router? Especially since when I configure it to bypass firewall rules, everything does work as it should

[mimugmail]

Ok, I reread your thread 3 times .. now I understand. You have BGP peers in 10.0.0.0 and announce 10.0.50 over it. Packet from your client go to OPNsense, it sends them back inside to peers and peers reply directly to your machine, so OPN only see half of the connection which is a bit ugly. You can circumvent the problem with and outbound NAT rule.

Out IF LAN, source LAN, dest 10.0.50, translate to interface address.

Then machines with 10.0.50 will see the OPN and the client and reply there back.
Maybe worth a try.

[SillyPosition]

Thank you!
It did the trick.
I dont know how it worked so far, in a partial way.
Maybe due to reflection settings? I really dont understand. I thought it happens under the hood automatically (SNAT to BGP networks)