OPNsense Forum

Archive => 18.1 Legacy Series => Topic started by: ssachse on January 30, 2018, 12:36:46 pm

Title: NAT, port aliases, redirect not working after upgrade
Post by: ssachse on January 30, 2018, 12:36:46 pm
Hi all, after upgrading to version 18.1, none of my port forwarding based on port aliases works. looking at /temp/rules.debug I just found following

......
UnifiPorts = "{ 8080 8443 8880 8843 27117 3478 }"
table <UNIFISERVER> persist
UNIFISERVER = "<UNIFISERVER>"
......

which seems ok, but

......
rdr on igb0 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on igb1_vlan10 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on igb2_vlan3 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on igb1_vlan1003 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on igb2_vlan1007 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on igb1 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on igb1_vlan1005 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on ovpnc2 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on openvpn inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on igb2_vlan8 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on igb0_vlan101 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on igb1_vlan10 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on igb2_vlan3 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on igb1_vlan1003 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on igb2_vlan1007 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on igb1 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on igb1_vlan1005 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on ovpnc2 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on openvpn inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080
rdr on igb2_vlan8 inet proto {tcp udp} from {any} to {(self)} port $UnifiPorts -> $UNIFISERVER port 8080

......

the rules for redirecting just resolve to the first port in the alias

What can I do ? in the 17 branch everything worked .... Is there a fix ? I have lots of those definitions in my rules. O do I have to revert to the 17 branch ? If, how is it done ?
This way I rendered mi production setup unusable  ;-(

Thanks a lot
Stefan
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: frank_p on January 30, 2018, 02:00:55 pm
I can confirm. same issue here.
seems that the first value of an alias (when more then one) is always fetch by the rule.
The NAT rule is not using all values or definitions.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on January 30, 2018, 02:33:16 pm
Tested here as well, and I can recreate the issue you are referring to.

Now I say that, to say this: I wasn't using Port aliases before, and was planning on doing so, but will most likely wait at this point.

My individual NAT Port Forward rules are working fine IF I'm not doing port aliases with them.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: marjohn56 on January 30, 2018, 03:05:14 pm
Port Aliases are working in my system using NAT. Still running 18.1.rc2.

So I have Binat rules as I have multiple external IP's. Create that and then create a WAN firewall rule any to my internal mail server with the ports that I have open 443,465 and 993 using a ports alias list and it works fine. I have just deleted the rules, checked that the ports were not accessible and then re-created them again, working fine.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on January 30, 2018, 03:07:15 pm
Well, I think that's the difference, this is on the production release.. so something changed between that RC and the production release it appears..
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: marjohn56 on January 30, 2018, 03:11:36 pm
I can check that but not until after 17:30 GMT, I cannot take opnsense down at the moment, I will get an ear bashing!  :)
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on January 30, 2018, 03:17:41 pm
Hahahaha.. I hear you :)
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: frank_p on January 30, 2018, 03:24:52 pm
Have one alias for web-traffic (ports 80 and 443).
Using that alias to forward traffic to respected servers (Mail and Cloud)
NAT rule uses ALWAYS the first parameter (it was 80). Means also HTTPS traffic was forwarded as port 80 traffic.
As i changed the sequence of parameter from 80,443 to 443,80, the traffic is now forwarded to 443, but not port 80 traffic.

Until V17 both parameter was handeled by the rule like expected 80->80 and 443->443

Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: marjohn56 on January 30, 2018, 04:25:29 pm
OK, I have just done a test with my 'TEST' unit, virgin 18.1

By default this unit had just the rules it comes with out of the box.

I created a port forward from the WAN to the LAN, forwarding ports 80,443,465 and 993, directing them at my laptop.

Then on my main machine I ran a piece of software called Hercules, if you do not know it then find it, it's a very useful tool, whatever, Hercules allows me to test by trying to connect on whatever port I specify to a given address.

I then ran wireshark on the laptop to see what was coming through the firewall.

I have to say, all the port forwards worked. I don't know what's going on with others, but on my test unit  - perfect.

This was create the port alias(es) then create the port forward, apply.

Note, before someone says, I had to turn of the block private networks as the WAN network was 192.168.1.0
and the LAN network was 192.168.3.0 - but it worked.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: frank_p on January 30, 2018, 05:08:05 pm
Probably it has to do with "translated NAT rules" from V17 -> V18 during update?
If yes, it can be the reason, that a fresh installation works and an updated installation not.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: marjohn56 on January 30, 2018, 05:14:59 pm
More than likely, I had issues when bouncing 17.7.11 to 18.1.rc1 & rc2.

I've drawn the conclusion that when doing a major upgrade for me it's best to do the configuration from scratch anyway. In the past, with pfS**** I've also had issues so now I just bite the bullet and get on with it.

I know it should not happen, but these things do, and I end up with less down time in the long run.

I got bitten by the ICMP - ICMP6 issue too, so there was a lot of swearing going on.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: ssachse on January 30, 2018, 05:31:00 pm
Well actually the upgrade did not work for me .... So I reinstalled and imported the config .... but sill the same with the aliases
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: marjohn56 on January 30, 2018, 05:55:09 pm
Try doing it with a fresh install and MANUALLY create the config. I know it's a PITA but see if that works, obviously something is not right, but I suspect its in converting the config.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on January 30, 2018, 05:55:38 pm
Probably it has to do with "translated NAT rules" from V17 -> V18 during update?
If yes, it can be the reason, that a fresh installation works and an updated installation not.

Then the question becomes this:  Do we have to delete all NAT-related rules, even for Outbound bypass rules for say, a VPN, then re-enter from scratch?
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: marjohn56 on January 30, 2018, 06:01:54 pm
Sadly there is only one way to prove it. My NAT and Aliases work on a fresh install.

I don't have time at this minute, but in a while I will compare my old 17.7.11 config to my 18.1.rc2 and see if there are any differences, I will also bounce my live firewall to 18.1 and see what happens... but as I said earlier, that needs to be after 17:30 GMT.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on January 30, 2018, 06:03:34 pm
Well, thank you so far for the troubleshooting.  I may pull one of my backup configs from before the upgrade and see what might be different as well.

Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: marjohn56 on January 30, 2018, 06:44:45 pm
So I've just bounced my 18.1.rc2 to 18.1_1 and all is working.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: AdSchellevis on January 30, 2018, 08:27:26 pm
I just installed an older version and checked the ruleset for a similar situation, it looks like the old version dropped the target port when a destination port alias was provided.

I'm not 100% sure this is intended behaviour for pf, but let me prepare a fix which does the same.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on January 30, 2018, 08:35:35 pm
Strange, because I tested this in a setup that did not use port aliases.

I added the port aliases, then redid the port forward rules, same thing happened, but I look forward to the patch to see if this fixes it.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: AdSchellevis on January 30, 2018, 08:51:24 pm
Ok, here's the patch

https://github.com/opnsense/core/commit/57f51d2943d964032770574605397006616e935c (https://github.com/opnsense/core/commit/57f51d2943d964032770574605397006616e935c)

installable using:

Code: [Select]
opnsense-patch 57f51d2943
Which in my test setup seems to deliver the same rule output.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on January 30, 2018, 09:09:50 pm
I've applied the patch.. .not sure what I need to look for.. because now I'm thinking it was working before, but meh.. I'm still testing.  THank for the patch though!
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on January 30, 2018, 09:17:36 pm
Well it does look like that Port aliases are working now, at least on my side.  Just tested by adding a new rule that included them all, then shut down each forwarded port in other rules one by one and tested. .. seems to have worked, but will wait for other confirmations before I move over to it as a permanent solution.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: tillsense on January 30, 2018, 09:20:34 pm
thanks ad!!
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: ssachse on January 30, 2018, 09:52:14 pm
I also installed an older version, got my opnsense working again and waiting for more confirmation before I switch again. Thanks a lot!
Stefan
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: frank_p on January 30, 2018, 11:22:40 pm
Looks like port-forwarding is working again :) on installation came from V17 and upgrade to V18
Thanks for the patch !!
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: franco on January 31, 2018, 07:36:29 am
Thank you for confirming. We will discuss releasing another hotfix for this and let you know soon. :)


Cheers,
Franco
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on January 31, 2018, 09:20:42 am
Hello

I noticed GeoIP Alias isn't working after upgrading to 18.1_1 and tried applying the hotfix, sadly it didn't helped.
I then tried with source any, which seemed to help, but after some time I am unable to connect again (OpenVPN in this case).

Sadly I can't provide any logs at the moment, because I'm not at home and I don't have a working VPN ;).

Regards
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: sibio on January 31, 2018, 11:30:18 am
Hi all,
patch was successful for us also but we had to "clone" the old rules and delete the original rules (after applying the patch and "reloading all services" on the console).

We had also a side effect of this bug: an old (useless) inbound NAT VOIP rule (using a port alias with SIP and some media ports) that prevented all OUTBOUND SIP connections (which was very surprising to us). After patching & cloning this side effect disappeared as well.

Robert wanted me to precise this in case this is useful to anybody.
Raynald
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: hirschferkel on January 31, 2018, 12:54:08 pm
The patch did not work here.
Ok, here's the patch

https://github.com/opnsense/core/commit/57f51d2943d964032770574605397006616e935c (https://github.com/opnsense/core/commit/57f51d2943d964032770574605397006616e935c)

installable using:

Code: [Select]
opnsense-patch 57f51d2943
Which in my test setup seems to deliver the same rule output.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on January 31, 2018, 02:31:34 pm
The patch did not work here.
Ok, here's the patch

https://github.com/opnsense/core/commit/57f51d2943d964032770574605397006616e935c (https://github.com/opnsense/core/commit/57f51d2943d964032770574605397006616e935c)

installable using:

Code: [Select]
opnsense-patch 57f51d2943
Which in my test setup seems to deliver the same rule output.


Same here.  I was having some oddities, not related to the patch or aliases, but outbound policy nat rules were messing up.

Just went through this morning and cleaned the entire firewall up and now it appears to be behaving... fingers crossed.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: hirschferkel on January 31, 2018, 03:29:31 pm
My forwarding rules are just > take all incoming connections on a range of ports to one destination and it's corresponding ports.
Host is defined as an Alias (but that's not the problem).
Port range is defined as another Alias.

But what I found is, that old imported rules can not be edited!
On the other hand I can edit a new rule, but this one will not be available with NAT port forwarding!

Something has gone quite wrong here...

At the moment it only works if I choose "pass" as an option, in a manual, single port forwarding. But I can't select new rules which are set to pass. I guess old rules loose their definition, as they can not be edited either. So in the end I cannot set a portrange to be passed... that's wired.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on January 31, 2018, 03:47:26 pm
I believe this is a hold over from then first jump to the 17.x series.

There was a change in how the rule was defined and some other options that were added/removed from the tabs/pages.

I had a similar issue originally, but spoke with franco and he suggested I recreate the rules and get rid of old ones.  Once I did that, things were back to normal.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on January 31, 2018, 03:49:22 pm
My forwarding rules are just > take all incoming connections on a range of ports to one destination and it's corresponding ports.
Host is defined as an Alias (but that's not the problem).
Port range is defined as another Alias.

But what I found is, that old imported rules can not be edited!
On the other hand I can edit a new rule, but this one will not be available with NAT port forwarding!

Something has gone quite wrong here...

At the moment it only works if I choose "pass" as an option, in a manual, single port forwarding. But I can't select new rules which are set to pass. I guess old rules loose their definition, as they can not be edited either. So in the end I cannot set a portrange to be passed... that's wired.
Is it possible that you configuered the old rules in the NAT > Port Forward menu? They should be editable there, they are only visible in the rules if you choosed "create new rule"
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: hirschferkel on January 31, 2018, 04:17:54 pm
AFAIK you can create rules only in the section Firewall > Rules > choose Interface > edit rules.
I created the old rules there, and the new ones.

The old ones stay not editable.

The new rules will not be available in Firewall > NAT > Port forward > edit forwarding rule > Filter rule association
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: franco on January 31, 2018, 05:16:41 pm
> The old ones stay not editable.
>
> The new rules will not be available in Firewall > NAT > Port forward > edit forwarding rule > Filter rule association

I think that's how the association always worked, no? Non-editable if auto-created via association or manually selectable if not.


Cheers,
Franco
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: marjohn56 on January 31, 2018, 05:36:39 pm
18.1_1 working well. Nice one guys.👍
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: hirschferkel on January 31, 2018, 05:41:41 pm
Hi Franco,

you're right, I missed that.

So if I autocreate a port forwarding, it will not work!
If I setup a rule one manually, it will not be available for a new port forwarding. So it won't work, anyway at the moment?
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: franco on January 31, 2018, 05:52:07 pm
Ah ok, that sounds like a viable theory. The problem with the auto-created association rules is that they are not real rules so their edit button was removed to prevent further breakage. Ideally, they shouldn't exist in a state that an user should feel the need to edit, but may therefore be in a twilight state that the new alias system cannot cope with yet. We'll take a closer look.


Thank you,
Franco
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: dcol on February 01, 2018, 05:57:48 pm
18.1 update also killed my NAT. Patch fixed it for me.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: franco on February 01, 2018, 06:33:28 pm
18.1.1 has been prepared and is ready for release tomorrow morning.


Cheers,
Franco
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Phobus on February 01, 2018, 07:04:10 pm
Hopefully 18.1.1 will fix the Alias problem.
18.1 breaks a lot of things for me; All Aliases not working, NAT-Patch not working, IPS Rule Updates not working...
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: opnsense_user12123 on February 01, 2018, 07:32:03 pm
Hopefully 18.1.1 will fix the Alias problem.
18.1 breaks a lot of things for me; All Aliases not working, NAT-Patch not working, IPS Rule Updates not working...

i have the same problems. on 17.7.12 everything was working fine.
Even a full reinstall didnĀ“t help!
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: elektroinside on February 01, 2018, 07:43:59 pm
Hopefully 18.1.1 will fix the Alias problem.
18.1 breaks a lot of things for me; All Aliases not working, NAT-Patch not working, IPS Rule Updates not working...

Aliases are working for me, though i have a cron job to update them.

Btw, @Franco, using the 'Aliases Resolve Interval' from Firewall: Settings: Advanced is indeed broken.

IPS rules / updates have a patch which fixed the issue.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 01, 2018, 08:05:55 pm
With the fix, port aliases are working, but GeoIP alias (still) isn't.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: marjohn56 on February 01, 2018, 08:08:33 pm
With the fix, port aliases are working, but GeoIP alias isn't.

Gesendet von meinem ONEPLUS A5000 mit Tapatalk

It's all a bit strange. My geo aliases and all others are working fine... I must have done something wrong . ???
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: franco on February 01, 2018, 11:51:13 pm
Maybe we should separate "not working" into two categories:

(a) Firewall: Diagnostics: pfTables -- alias empty
(b) generally not working in NAT or firewall rule

Then also check (b) under Firewall: Diagnostics: pfInfo (Rules) whether these non-working rules actually see traffic


Thanks,
Franco
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 02, 2018, 01:30:07 am
Maybe we should separate "not working" into two categories:

(a) Firewall: Diagnostics: pfTables -- alias empty
(b) generally not working in NAT or firewall rule

Then also check (b) under Firewall: Diagnostics: pfInfo (Rules) whether these non-working rules actually see traffic


Thanks,
Franco
Got it, my GeoIP alias falls under (a), the pfTable is empty and therefore there's nothing to compare to, since I'm using it as source nothing passes :)
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: franco on February 02, 2018, 09:13:53 am
Okay, that's good and bad... Good in the sense it's not a fundamental firewall issue, bad because whatever prevents your system from fetching the aliases may prevent it from reaching out in the first place...  Is that table populated when you run this from the console?

# configctl filter refresh_aliases
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 02, 2018, 09:29:14 am
Okay, that's good and bad... Good in the sense it's not a fundamental firewall issue, bad because whatever prevents your system from fetching the aliases may prevent it from reaching out in the first place...  Is that table populated when you run this from the console?

# configctl filter refresh_aliases
The command only returns 'OK'.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: franco on February 02, 2018, 09:34:34 am
Sure, now check the table...
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 02, 2018, 09:35:05 am
Sure, now check the table...
Seems it's still empty :(
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: franco on February 02, 2018, 09:38:01 am
What does this return then?

# ls -lah /var/db/aliastables/
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 02, 2018, 09:52:15 am
What does this return then?

# ls -lah /var/db/aliastables/
CH is my GeoIP alias, and it's empty, NAS contains the address I configured.(https://uploads.tapatalk-cdn.com/20180202/c666730ddf36d35618f39169288c8698.jpg)
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: franco on February 02, 2018, 09:57:37 am
Are you using the CH alias in a floating rule?
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 02, 2018, 10:00:22 am
Are you using the CH alias in a floating rule?
No, only in WAN rules, but currently it's removed from them because I tried to recreate the alias at the time displayed by the ls command.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: franco on February 02, 2018, 10:03:08 am
So you can't fetch the GeoIP alias even though it's not used?

We can try to increase the pressure:

# rm /var/db/aliastables/CH*
# configctl filter refresh_aliases

Still empty?
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Phobus on February 02, 2018, 10:19:07 am
Maybe we should separate "not working" into two categories:

(a) Firewall: Diagnostics: pfTables -- alias empty
(b) generally not working in NAT or firewall rule

Then also check (b) under Firewall: Diagnostics: pfInfo (Rules) whether these non-working rules actually see traffic


Thanks,
Franco

For me (Alias problem):
(a) Firewall: Diagnostics: pfTables -- alias empty

# configctl filter refresh_aliases
Still empty

# rm /var/db/aliastables/EBL*
# configctl filter refresh_aliases
Still empty
Output: Error (1)

Strange Output now files and Aliases are missing:

root@*****:~ # ls -lah /var/db/aliastables/
total 12
drwxr-x---   2 root  wheel   512B Feb  2 10:29 .
drwxr-xr-x  18 root  wheel   1.0K Feb  2 08:36 ..
-rw-r-----   1 root  wheel   257B Feb  2 10:29 EBL.self.txt
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 02, 2018, 10:20:14 am
So you can't fetch the GeoIP alias even though it's not used?

We can try to increase the pressure:

# rm /var/db/aliastables/CH*
# configctl filter refresh_aliases

Still empty?
Sadly yes, the three files are created but the txt file is still empty.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: franco on February 02, 2018, 10:37:29 am
Under Firewall: Settings: Advanced, is " Verify HTTPS certificates when downloading alias URLs" checked or unchecked? Are you using a proxy server in your network doing HTTPS MITM?


Cheers,
Franco
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 02, 2018, 10:39:53 am
Under Firewall: Settings: Advanced, is " Verify HTTPS certificates when downloading alias URLs" checked or unchecked? Are you using a proxy server in your network doing HTTPS MITM?


Cheers,
Franco
Setting is unchecked and I'm not using a proxy server who intercepts https..
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Phobus on February 02, 2018, 10:42:33 am
Quote
Under Firewall: Settings: Advanced, is " Verify HTTPS certificates when downloading alias URLs" checked or unchecked? Are you using a proxy server in your network doing HTTPS MITM?
In my Situation also:
Setting is unchecked and I'm not using a proxy server who intercepts https..
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Phobus on February 02, 2018, 03:45:58 pm
After the update to 18.1.1 "IDS rule update problem" seems to be solved.
Unfortunately Alias problem still exist - aliases aren't working e.g. hosts  :(
Same outputs as posted before...
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: opnsense_user12123 on February 02, 2018, 08:49:30 pm
NAT / Portforwarding is even in 18.1.1 not working correctly.

only if i disable port forwarding rule to proxy 127.0.0.1 (https) and disable blocking https rule, i get a 100% working connection!

On 17.7.12 these all works without any problems.

my no ssl bump list is the same as in version 17.7.12.


Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Phobus on February 03, 2018, 02:41:15 pm
OK I've found the "bug" with aliases (hosts) not working.
In my case I've a alias list with hosts they are used from MS for data collection.
One of them can't be resolved anymore so this entry should be skipped (in my opinion), but in that case it ended up with an error -> table generation (all) will be aborted -> aliases in that case will not work.

One deceased entry in an alias list is enough to stop the whole table generation  :o
This behavior should be changed to skip such entries.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: hirschferkel on February 07, 2018, 10:46:16 am
After Updating to 18.1.1 it runs again. Obviously there were some more issues.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 11, 2018, 02:22:03 pm
Tried reinstalling with 18.1 and updated to 18.1.2_2.
Still the same issue, geoip alias is empty..
Executed refresh_aliases and also deleted the tables and retried, still empty..
Same on test VM.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: franco on February 13, 2018, 05:32:47 pm
Another patch to try... https://github.com/opnsense/core/commit/b514992

# opnsense-patch b514992


Cheers,
Franco

Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 13, 2018, 05:40:44 pm
Another patch to try... https://github.com/opnsense/core/commit/b514992

# opnsense-patch b514992


Cheers,
Franco
Applied patch, retried the previous steps, still empty :(
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on February 13, 2018, 05:43:00 pm
How are you populating the alias?  Sorry if you mentioned it, but I don't want to dig through the entire thread.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 13, 2018, 05:53:15 pm
How are you populating the alias?  Sorry if you mentioned it, but I don't want to dig through the entire thread.
I open an alias and save it, apply the changes and check the pfTables result for the GeoIP alias.

Since it's still empty, I try these mentioned commands:

# rm /var/db/aliastables/CH*
# configctl filter refresh_aliases

And still empty.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on February 13, 2018, 05:53:58 pm
What is your configuration on the Alias itself?

Can you post a screenshot of the config?
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 13, 2018, 05:59:18 pm
What is your configuration on the Alias itself?

Can you post a screenshot of the config?
Only one country checked, even tried another one, same result..
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on February 13, 2018, 06:03:19 pm
So, I just tested a brand new alias, using this: https://iplists.firehol.org/files/firehol_level1.netset

Set the alias to URL Table (IPs) set the expiration to 1 day 0 hours (So it will refresh daily) and submitted, pfTables immediately shows them.

I've attached what the alias config looks like.  Can you screenshot YOUR alias similar to how I did mine so I can see what you're doing exactly.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: franco on February 13, 2018, 06:24:16 pm
@Evil_Sense it's not for populating aliases, it's for repairing alias usage in the outbound rules. run this before retesting:

# configctl filter reload


Cheers,
Franco
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 13, 2018, 06:33:11 pm
So, I just tested a brand new alias, using this: https://iplists.firehol.org/files/firehol_level1.netset

Set the alias to URL Table (IPs) set the expiration to 1 day 0 hours (So it will refresh daily) and submitted, pfTables immediately shows them.

I've attached what the alias config looks like.  Can you screenshot YOUR alias similar to how I did mine so I can see what you're doing exactly.
That's how I configured the GeoIP alias..

Edit: IP and port aliases are working and also populated(https://uploads.tapatalk-cdn.com/20180213/930c0100bab5f692e5d06bb793fade77.jpg)
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on February 13, 2018, 06:53:03 pm
Configured mine the same way, submitted, pfTables shows it is populated.
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 13, 2018, 07:08:00 pm
Configured mine the same way, submitted, pfTables shows it is populated.
Hmm.. Seems it doesn't get populated if the name is the same as the value

Edit: Deleting aliases also doesn't remove the tables..
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on February 13, 2018, 07:08:53 pm
That KIND of makes sense.  I set the name to GeoIP and just selected switzterland.. I didn't even think about the naming just as yours is in the interface, my bad. 
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 13, 2018, 07:41:18 pm
That KIND of makes sense.  I set the name to GeoIP and just selected switzterland.. I didn't even think about the naming just as yours is in the interface, my bad.
Thanks for testing and pointing me in the right direction :)
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: slackadelic on February 13, 2018, 07:42:14 pm
Did you get it working now?
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: Evil_Sense on February 13, 2018, 07:44:53 pm
Did you get it working now?
Yes, I changed the name and it's working perfectly fine :)

Also tested with other countries, it's true, if the name matches the value, nothing gets populated.

Now pfTables is showing some non existent tables, even after the refresh_aliases and reload command, but that doesn't bother me much ^^
Title: Re: NAT, port aliases, redirect not working after upgrade
Post by: franco on February 13, 2018, 07:53:06 pm
Ok, mystery almost solved. Added a follow-up for this https://github.com/opnsense/core/issues/2199 to discuss what to clean up to prevent users running into it.


Thanks!
Franco