Messages

Can you please take Captive Portal with Multi-WAN in this release?

Hello Franco,

The comment on the piece of code says:

* If small enough for interface, or the interface will take
    * care of the fragmentation for us, we can just send directly.

So, how does this effect our case.

When gateway is set to 'default', ipfw process the packet first and sends it to captive portal. After authentication, ipfw passes it to pf.

As soon as route-to is set in pf, the connection is made directly with pf and ipfw does not come into play at all. How come route-to which comes later into the picture make ipfw not process the traffic.

ipfw should alway come into play no matter what. Shouldn't this be the case.

You need to create Port Forwards for each WAN (WAN1 and WAN2). This will let you access the webserver on each WAN IP i.e. http://WAN1 or http://WAN2. If you have a domain name, rather stick to one Port forward on only one WAN interface.

While configuring it if you let Opnsense create firewall rules, it will do so automatically under each WAN.

Multi WAN is used in case you want to failover WAN1 and WAN2 in scenarios when one goes down or load balance traffic between WAN1 and WAN2.

Franco,

Superb..

This is exactly the issue.

error = (*ifp->if_output)(ifp, m0, sintosa(&dst), NULL);

Hello Franco,

I agree it must be enforced. But the as soon as we use route-to in PF it breaks the pfil ordering. Somehow, ipfw is not coming into picture.

Guys,

I see Pfsense has some sysctl options to specify the order.

sysctl net.inet.ip.pfil

net.inet.ip.pfil.inbound=pf, ipfw*
net.inet.ip.pfil.outbound=pf, ipfw*

Can we implement this and hopefully this resolves the issue for everyone?

Good that it worked for you. It still does not work for me.

As Ad Schellevis rightly said this is a FreeBSD bug. I am quoting what he said in Github.

"@srijannandi just tried it on my end and can confirm it doesn't work. Looks like a kernel issue. Found this online from some time ago https://lists.freebsd.org/pipermail/freebsd-pf/2010-July/005739.html , not sure if its related, there doesn't seem to be a quick fix."

FreeBSD needs to get this resolved at their end.

Guys,

For me the packet capture goes like this:

1. When Default LAN Rule has gateway set to 'default'.

tcpdump on LAN, with port 80 or port 8000

15:41:45.247575 IP 172.16.1.10.51497 > X.X.X.X.80 (SYN to website)
15:41:45.247741 IP X.X.X.X.80 > 172.16.1.10.51497 (SYN+ACK from website)
15:41:45.247933 IP 172.16.1.10.51497 > X.X.X.X.80 (ACK from my machine)

A second connection is on port 8000 for Captive Portal
15:41:45.252389 IP 172.16.1.10.51498 > 172.16.1.1.8000 (SYN)
15:41:45.252524 IP 172.16.1.1.8000 > 172.16.1.10.51498 (SYN+ACK)
15:41:45.252700 IP 172.16.1.10.51498 > 172.16.1.1.8000 (ACK)

tcpdump on the loopback has the following:

15:41:45.356778 IP 127.0.0.1.18902 > 127.0.0.1.8999 (SYN)
15:41:45.356885 IP 127.0.0.1.8999 > 127.0.0.1.18902 (SYN+ACK)
15:41:45.356964 IP 127.0.0.1.18902 > 127.0.0.1.8999 (ACK)

2. As soon as I change the gateway on the Default Rule to point to Gateway Group, my machine makes a direct connection with the website without giving the Captive Portal.

15:46:07.667431 IP 172.16.1.10.51542 > X.X.X.X.80 (SYN to website)
15:46:07.923599 IP X.X.X.X.80 > 172.16.1.10.51542 (SYN+ACK from website)
15:46:07.923911 IP 172.16.1.10.51542 > X.X.X.X.80 (ACK)

My question is when there is an 'ipfw' rule that states that all connections on the LAN port destined for port 80 be redirected to 127.0.0.1:8000, why does this not come into picture when I do policy based routing with Gateway Group.

add 5000 fwd 127.0.0.1,8000 tcp from any to any dst-port 80 in via em0

Can someone help me do this, please?

Can you access the Captive Portal on http://GuestNetwork Address:8000 or on https (if you have enabled SSL Certificate), when you set the gateway to 'default' in the Firewall Rule?

Also the GuestNetwork has DHCP enabled and the machines in the GuestNetwork has their DNS set to the GuestNetwork Address.

If possible, please share a screenshot of the Firewall Rules for the GuestNetwork interface.

I am sure you might have done the following, however please re-check:

1. Captive Portal has the interface set to your GuestNetwork.
2. DNS Forwarder Enabled.
3. DNS Rule from GuestNetwork to GuestNetwork IP Address.
4. Rule to access Captive Portal on port 8000-10000 from GuestNetwork to Guest Address.
5. Rule to allow traffic from GuestNetwork to 'any'.

When you do not set the Gateway Group, this will allow you Load Balance the traffic out. When you set the Gateway Group, it will set let you out to the internet, but you might not see the Captive Portal.

To see the Gateway Failover Group, you need to create one, under System --> Gateways --> All. Then you will be able to see the Gateway Failover/Load Balancing group for Policy Based Routing. The DNS Rule should be above the Default LAN rule and that too the gateway should be set to 'Default' i.e. without Failover/Load Balancing group.

After which you need to change the Default LAN rule to pass traffic via the gateway Failover/Load Balancing group.

This will work and Captive Portal will appear if the Default LAN rule is set to have the gateway as 'Default'.

As soon as you apply Failover/Load Balancing as the gateway on the Default LAN rule, it might not work as I am facing a similar issue. Here, the 'ipfw' rule which caters to Captive Portal does not come into picture and all traffic is passed by the 'pf' rule set.

Can someone point to a way to get 'ipfw' to see this traffic in both the scenarios. 'ipfw' applies when Default LAN rule's gateway is set to 'Default'.

Guys,

I need help in getting Muti-WAN to work with Captive Portal.

I have two gateway, one PPPoE and another Static. Both are working properly. I have configured two Gateway groups, GWGroup1 with WAN1(PPPoE - Tier1) + WAN2(Static - Tier2) and GWGroup2 as WAN1(PPPoE - Tier2) + WAN2(Static - Tier1). This is done because I want to pass LAN traffic using GWGroup1 and LAN 2 traffic via GWGroup2. So, I will at all times have a failover group for both my LAN's. Monitoring IP's have been set and I have followed the link https://docs.opnsense.org/manual/how-tos/multiwan.html to the hilt.

Have configured Firewall Rules to pass DNS traffic using default GW and also traffic to LAN Address on ports 8000-10000 via default GW (though I wasn't sure I needed this). My default LAN to any rule has the gateway set to GWGroup1. Please refer to the screenshot attached.

In this scenario, the Captive does not appear. As soon as I set the gateway on the Default LAN rule to Default GW, everything starts working properly and I get the Captive Portal.

The PF rules are all proper:
1. With Policy Base Routing (using GWGroup1) in the LAN rule.
pass in quick on em0 route-to (pppoe0 X.X.X.X) inet from 172.16.1.0/24 to any flags S/SA keep state label "USER_RULE: Default allow LAN to any rule"
2. With Default GW in the LAN rule.
pass in quick on em0 inet from 172.16.1.0/24 to any flags S/SA keep state label "USER_RULE: Default allow LAN to any rule"

What I see is that as soon as I change the LAN rule to Default GW, I can see traffic hitting port 8999 in the loopback address.
18:07:01.460540 IP localhost.15623 > localhost.8999(SYN)
18:07:01.460648 IP localhost.8999 > localhost.15623(SYN+ACK)
18:07:01.460769 IP localhost.15623 > localhost.8999(ACK)
After which, another connection is initiated from the machine's IP to the LAN IP of the firewall on port 8000. And I get the Captive Portal login page.

When I change the LAN rule to use GWGroup1, I do not see any traffic on the loopback interface for port 8999 nor any traffic on the LAN IP on port 8000.

As soon as I use multi-WAN, Captive Portal Fails. I was of the opinion that the architecture of Opnsense is such that the 'ipfw' always comes before 'pf'. In that case, in both the scenarios, 'ipfw' should always pass the un-authenticated traffic to 127.0.0.1, 8000. But, in real scenario, as soon as I use multi-WAN Captive portal does not appear.

Opnsense is doing exactly what it is supposed to do.This is how it should work.

My bad. My networking fundamentals needed a little brush-up.

Thanks and Regards,
-=Srijan Nandi

Hello Everyone,

I need a bit of an explanation on how Port Forwarding works in Opnsense. I have a simple setup. One WAN (PPPoE) and one LAN and a server connected to LAN.

LAN Address is 172.16.1.1 and IP Address of Server is 172.16.1.80. Default Gateway on Server is set to 172.16.1.1.

I have created a Port Forwarding Rule on Opnsense, to allow traffic on the WAN interface on port 2226 to forward it to the server 172.16.1.80 on port 2224. These can be seen below:

rdr on pppoe0 inet proto tcp from any to X.X.X.X port 2226 -> 172.16.1.80 port 2224
# Reflection redirect
rdr on { em0 em3 enc0 } inet proto tcp from any to X.X.X.X port 2226 -> 172.16.1.80 port 2224
no nat on em0 proto tcp from em0 to 172.16.1.80 port 2224.

When i run a tcpdump on the pppoe interface. I can see the traffic as stated below:

08:14:11.753002 IP 1.39.86.152.32254 > X.X.X.X.2226
08:14:46.574063 IP 1.39.86.152.33519 > X.X.X.X.2226

Now when I run a tcpdump on the internal interface (em0) of Opnsense, I see traffic as below:

08:14:11.753157 IP 1.39.86.152.32254 > 172.16.1.80.2224
08:14:11.753768 IP 172.16.1.80.2224 > 1.39.86.152.32254

My question here is, if i have configured a port forwarding (rdr) rule, shouldn't I see the WAN IP of the Opnsense device when I run a tcpdump on the internal port (em0) of Opnsense.

The same problem I see when I run a tcpdump on the server as well. I am getting the public client IP address instead of the WAN IP Address of the Opnsense device.

Shouldn't I see X.X.X.X instead of the client IP of 1.39.86.152. Is this now how it is supposed to work. It would be great help if someone can explain me this.

Thanks and Regards,
-=Srijan Nandi

Hello Franco,

I changed the following parameters to:

dev.netmap.buf_num=200000
dev.netmap.ring_num=800
dev.netmap.buf_size=3096

But, still the same results. Therefore, it is very much clear that Intrusion Prevention requires minimum 1GB RAM.

Sorry for the botheration.

Thanks and Regards,
-=Srijan Nandi