OPNsense Forum

Archive => 21.1 Legacy Series => Topic started by: Matzke on June 08, 2021, 11:31:16 am

Title: Shared forwarding various failures when using it
Post by: Matzke on June 08, 2021, 11:31:16 am
Dear All,

I have a big problem with shared forwarding (firewall settings).

First of all, I have a multi-WAN (2 WANs) szenario.

I wanted to use Traffic Shaper - in order to work properly, I have to activate shared forwarding.

As soon as I activate shared forwarding, the following problems occur:
- OpenVPN with topology subnet won't work anymore - no connection to OPNSense.
- when using the deprecated topology net30 OpenVPN works again
- sporadically (every 1-5 Minutes) my internal Clients loose connection to the internet. When waiting some minutes, the connection is back again.

--> as soon as deactivating shared forwarding, the problems above disappear (only Traffic Shaper won't work as expected)

On a second OPNSense I checked this behavior (momentarily only single-WAN):
- OpenVPN with topology subnet won't work with shared forwarding
- I could not realize connection-aborts at Clients of the second OPNSense while shared forwarding was turned on, so I expect a Problem with more than one Gateway (MultiWAN) --> but I have to say, that behind this OPNSense are only 2 Clients at the moment (far few than behind OPNSense 1 with multi-gateways)
Title: Re: Shared forwarding various failures when using it
Post by: franco on June 08, 2021, 11:36:22 am
So shared forwarding allows you to use policy-based routing with captive portal or traffic shaper. It means you have policy-based routing firewall rules...

> On a second OPNSense I checked this behavior (momentarily only single-WAN):
> - OpenVPN with topology subnet won't work with shared forwarding

This really can't be a general issue so I would advise to review your policy-based routing firewall rules that seem to break this in the first place.


Cheers,
Franco
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 08, 2021, 11:48:34 am
Hello,
please explain this to me in a little more detail.

On OPNSense 1 I have some default firewall rules that should control the behavior. The last rule in the list then chooses a gateway group. This should fit exactly to the tutorial where exactly this scenario is covered (policy-based for internet, normal rules for local traffic).

So yes, on OPNSense 1 I use policy-based routing in a firewall rule to modify internet traffic. All local rules are not policy based.

This way I have the problem with OpenVPN as well as sporadically losing my clients internet connection.




On OPNSense 2 I have no policy based routing (or the gateway is set to default), here I only have problems with OpenVPN.


By the way - everything works fine without shared forwarding except Traffic Shaper

Can I provide any more information?

Translated with www.DeepL.com/Translator (free version)
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 08, 2021, 08:00:09 pm
Can somebody help me?

In my oppinion it's a bug.
Title: Re: Shared forwarding various failures when using it
Post by: franco on June 09, 2021, 10:03:03 am
First and foremost make sure to configure OpenVPN correctly on a default install with shared forwarding enabled. There are no problems with it I can assure you.

You probably have a bad firewall rule interfering with our setup.


Cheers,
Franco
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 09, 2021, 11:00:59 am
Can I send you a backup of my configuration or some screenshots of OpenVPN as well as Firewall rules?

I don't know where I should look for an config-error because the system works great (without shared forwarding) and I don't know where I should make changes.
Title: Re: Shared forwarding various failures when using it
Post by: franco on June 09, 2021, 11:08:55 am
You can append screenshots here for the community to take a closer look.

I'll just repeat one more time: if shared forwarding is the issue make sure you do not have any outgoing rules (floating or otherwise) that would block your traffic on the way out. When you disable shared forwarding these rules have no effect so it seems to be working. You can even use the firewall live log to search for dropped traffic that way given that you enable rule logging.

Finding the dropped traffic should be easy enough.


Cheers,
Franco
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 09, 2021, 11:30:04 am
I'll post the screenshots today in the afternoon.

Just one short question in Advance - the last rule on every interface is a block all rule. Every traffic which is allowed to pass firewall is explicitly allowed in rules above.

Now the question is - is this rule the problem or does this rule exclude the problem, since it ensures that a allow rule must be present, otherwise it would not work even without shared forwarding?

Thanks and many greetings
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 09, 2021, 11:33:04 am
and on the other hand - why do I have sporadically no connection to the internet with my LAN devices (when shared forwarding is turned on)

without changing the configuration this is alternating, I just have to wait some time.

Completely unrelated to the problem with OpenVPN.

(and if it is a configuration problem of the firewall, why does it work with deprecated OpenVPN topology net30 but not with topology subnet)?
Title: Re: Shared forwarding various failures when using it
Post by: franco on June 09, 2021, 01:21:11 pm
You do not need explicit block rules since the system already has these. The only exception is probably when you want to selectively log block information.

As for your apparent problems this is impossible to find out without enough information about your configuration and it probably escapes my available time for community support.


Cheers,
Franco
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 09, 2021, 02:07:02 pm
Attached you will find some Screenshots of my configuration.

Interesting ist OpenVPN_Roadwarrior_KS28 which is the Roadwarrior's interface and for example V30_intern which is my internal LAN. I skipped the other VLANs because they are not involved - if needed, I could Screenshot them too.

I can understand that it is hard and time consuming to support community members free of charge, but when I need help, most times there is really a bug in the software which I can help to solve with my logs and details or there is a problem in the documentation and I made a misconfiguration because of lacking docs.

Normally I am an experienced IT professional who does not need any help. So beginner's mistakes are not really to be expected - unless the docs didn't give it.

This is also "only" my private firewall at home - from the configuration you should see that it is not a forest and meadows (no idea how this phrase is called in English) configuration of a hobby IT professional. Unfortunately, however, a paid OPNSense for private is not affordable, so I fall back on the forum, which certainly helps many others with similar problems.
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 09, 2021, 02:07:34 pm
second part

Filesize-restriction is very hard
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 10, 2021, 03:46:40 pm
... can nobody help or even give me a hint ...

Are my firewall settings and rules okay?
Title: Re: Shared forwarding various failures when using it
Post by: mimugmail on June 10, 2021, 09:48:47 pm
Firewall : Settings : Advanced : Disable Force Gateway

This should ticked. Also, try disabling sticky for testing (only regarding the client timeout problem).
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 11, 2021, 03:43:53 pm
thanks,

I'll try and report what happened.
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 13, 2021, 02:42:18 pm
As soon as activating "disable force gateway" I sporadically loose Internet Connectivity (I will say I can't connect from clients in LAN to WAN).

As soon as deactivating it, everything works as expected (I tried it with and without shared forwarding enabled).

Therefore I didn't do other tests because basic functionality (Firewall/Routing from LAN to WAN) was lost.
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 14, 2021, 06:09:50 pm
Hello,
as already written yesterday, unfortunately it still does not work for me. Also, I have not entered any floating rules (see screenshots) that somehow spark in between.

Can anyone still help me? In my opinion, this is still a bug and I find it a pity that the OPNSense team is not investigating the issue more intensively here. As soon as a configuration error turns out, you can refer to the documentation and do not treat the thread further, but to put no further force into it I find very unfortunate.

@Franco: So again my question - what can I contribute to the error diagnosis?

Translated with www.DeepL.com/Translator (free version)
Title: Re: Shared forwarding various failures when using it
Post by: mimugmail on June 14, 2021, 07:18:48 pm
What happens when you disable the balancing rule or just allow it. Does Openvpn still have problems with topology?
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 15, 2021, 02:23:51 pm
Dear Mimugmail,

please tell me, what I should change in this rule?

a) disable this rule -> then I can't go into internet
b) just allow this rule -> I don't understand
c) change the gateway in this rule to * (but then I don't have policy based routing according to franco)

By the way (before changing anything) - when I dialin via openvpn and shared forwarding is enabled I even can't ping the firewall itself. I would assume that a rule on LAN interface shouldn't interfere here?

Title: Re: Shared forwarding various failures when using it
Post by: mimugmail on June 15, 2021, 09:27:00 pm
Just remove the Gateway in the rule and tell me if it works. Shared forwarding is enabled by default, if there would be a general problem you wouldnt be the first and only one having such phenomenons
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 15, 2021, 09:56:28 pm
I just forgot, I have a second OPNSense with only one Gateway and therefor no rule for gateway-switching.

There it is exactly the same. And when I remeber, shared forwarding was turned off in default (but I updated to OPNSense 21 it was no fresh installation).

By the way just to be sure - franco asked me if I have policy based routing. A fresh installation doesn't have a lot firewall rules (only some standard-floating), interface-rules are empty. When shared forwarding is enabled by default and it requires policy based routing it shouldn't work on any installation.
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 16, 2021, 09:49:17 am
Just to show - attached the firewall-rules of
a) LAN-Interface (vlan20 internal)
b) OpenVPN-Roadwarrior-Interface

c) Gateway-Tab - no multiple LAN Gateways

This is configuration of a second OPNSense - here also OpenVPN won't work woth topology subnet and shared forwarding turned on.
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 16, 2021, 11:33:25 am
... just verified with opnsense21.1-Install-ISO and a test in virtualbox:

- Shared forwarding is enabled by default
- OpenVPN Topology net30 is default (subnet isn't default although net30 is deprecated)
Title: Re: Shared forwarding various failures when using it
Post by: mimugmail on June 16, 2021, 12:08:32 pm
Is there a reason why you assign Openvpn as an Interface and add a gateway? Maybe this has strange side effects I'm not aware of
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 16, 2021, 02:23:06 pm
I just set an interface per OpenVPN Link - the gateway was set automatically. I also don't change something in interface settings - everything is left blank.

Attached you will see my VPN-Interface-Configuration (OPNSense Firewall 1).

I did this because I want to limit traffic from special roadwarriors or VPN-Members via firewall. Without the interface-assignment I was unable to create firewall rules which worked.

I tried to filter in the standard OpenVPN Interface (one interface for all different server/clients/tunnels) but it seems to be the encapsulated traffic at this interface. No rule with IP-Addresses of the VPN-Clients ever worked.

If this is wrong, please tell me how to solve this and I will delete the interface assignments.
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 20, 2021, 12:12:36 pm
Hello,

how can we best continue here - I am happy to help with tests and protocols.
How can I filter OpenVPN traffic via firewall without creating specific interfaces (as in the post before)?

Another question - all my internal network traffic is VLAN-tagged on one physical interface - can there be a problem here? The second firewall also has VLAN tagging active.

The fact is, the problem still exists and is also clearly on the two things:
openvpn topology subnet + shared forwarding on.

Furthermore, it is unfortunately not yet clarified why I sporadically lose the connection to the Internet with the multi-WAN firewall rule and it also comes back and disappears again (alternating) without any changes? This also occurs exclusively when shared forwarding is enabled.
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on June 23, 2021, 08:39:08 pm
Dear mimugmail,

I did a first try:

- deleted all OpenVPN Firewall rules
- deleted Interface assignment of OpenVPN tunnels (so I have now only one OpenVPN tab in Firewall
- restarted OpnSense
- turned on shared forwarding (OpenVPN topology subnet is already turned on)

I am very surprised, but OpenVPN now seems to work in this constellation.

I will continue to monitor it and get back to you later. But if this is really the case, then a note in the documentation would be very important not to assign OpenVPN interfaces. Maybe this should even be blocked or not offered in the web interface?

I will also observe whether the sporadic Internet outages are now gone and I will have a try with new Firewall-Rules in OpenVPN tab.
Title: Re: Shared forwarding various failures when using it
Post by: mimugmail on June 23, 2021, 11:13:59 pm
OK,good progress!
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on July 04, 2021, 08:28:06 pm
Dear Mimugmail,

I have been observing the behavior for some time now and can report the following (and would like to split the topic a bit, although it all has to do with shared-forwarding)

- after deleting the interface assignment of OpenVPN the topology subnet works like a charm

so here my first question
a) is this a bug, that assigned interface + shared forwarding + topology subnet won't work
b) I opened a separate thread but got no sufficient answer - when should I assign an interface to OpenVPN and when not (for example when using an NordVPN tunnel and I have to route specific traffic through it (guest network) I had to assign an interface for this OpenVPN instance.

- I still have some hickup with my internet-traffic. As soon as I use gateway-group and shared forwarding I have internet-dropouts (some seconds or minutes). After modifying firewall rule to use the default gateway instead of gateway-group it works like a charme, but it doesn't use my second gateway :-)

So I think, there is still a bug with shared forwarding and gateway group.

Please let me know, how I can assist the investigation
Title: Re: Shared forwarding various failures when using it
Post by: Matzke on July 07, 2021, 10:17:01 pm
Hi Mimugmail,

do you have an answer for my still existing problems or how can I assist in solving the problems?

Thanks a lot
Title: Re: Shared forwarding various failures when using it
Post by: mimugmail on July 08, 2021, 07:52:18 am
Its not possible to troubleshoot such complex things from remote. I dont assign interfaces on servers but I cant imagine why there should be a reason for it.