OPNsense Forum

English Forums => 18.1 Legacy Series => Topic started by: mais_um on December 08, 2017, 01:29:54 am

Title: Aliases dosen't work(?)
Post by: mais_um on December 08, 2017, 01:29:54 am
Hi

OPNSense version: OPNsense 18.1.a_428-amd64

Have a roule from Wan (aliases 3 IPs) to firewall to manage the firewall and is not working.

The roule:        Proto  Source         Port   Destination   Port                  Gateway   Schedule   Description   
      IPv4 TCP   MaisUm         *        This Firewall   443 (HTTPS)   *


MaisUm with host: 192.168.100.100, 192.168.100.102 and 192.168.100.103.

If i put and IP; 192.168.100.100 or WAN Net (192.168.100.0/22) it works with aliases MaisUm doesn't.

I only have only one alias "MaisUm"
Thanks
Title: Re: Aliases dosen't work(?)
Post by: franco on December 08, 2017, 07:16:29 am
Hi there,

Thank you for running -devel!

Aliases are undergoing a larger rework which will take at least one more iteration, if you need them to be like they used to you have two options.

1. Switch back to the production release:

# opnsense-update -t opnsense

2. Go back to the last known good -devel package:

# opnsense-revert -r 17.7.8 opnsense-devel


Cheers,
Franco
Title: Re: Aliases dosen't work(?)
Post by: mais_um on December 08, 2017, 12:10:27 pm
Is in a test environment. So just saying. 
Title: Re: Aliases dosen't work(?)
Post by: franco on December 08, 2017, 12:13:43 pm
Ad said it only needs kickstart, e.g. giving it a reboot should make it work again. Make sure you update to 18.1.a_444 to avoid glitches. :)


Cheers,
Franco
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 08, 2018, 10:21:21 am
@franco

Has this been completed?

I only ask as my primary router relies heavily on Aliases, Secondary test router does not use them. If it's done I can move the primary to 18.*

Title: Re: Aliases dosen't work(?)
Post by: franco on January 08, 2018, 07:08:26 pm
There were 2 reports about /var/db/aliastables not being created which caused resolution to fail, but which we couldn't reproduce. The code is all in 18.1.r1, if you must you can wait till 18.1.r2 or 18.1 to move over. It's hard to say, but we are confident as other testers said to be happy.

Until 18.1 is out 17.7 will be updated alongside. :)


Cheers,
Franco
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 08, 2018, 07:35:30 pm
Thanks Franco,

It's fine,  I did find they work, I set up a couple of Aliases on my test unit and had no issues.

Better to leave the primary as it is for now. SWMBO works from home and if I break the net then I am in deep doggie doos...  ;D
Title: Re: Aliases dosen't work(?)
Post by: markusd on January 12, 2018, 03:26:50 pm
Hi
It seems, firewall-aliases don`t work for me after update to 18.1.r1...
And there is no /var/db/aliastables...

Any infos you need?

Thanks!

Markus
Title: Re: Aliases dosen't work(?)
Post by: franco on January 12, 2018, 03:55:25 pm
Hi Markus,

If you run this once on the console, will it work?

# configctl filter refresh_aliases


Cheers,
Franco
Title: Re: Aliases dosen't work(?)
Post by: markusd on January 12, 2018, 04:24:17 pm
Hi Franco,
thank you for your fast answer!

# configctl filter refresh_aliases
works after
# mkdir /var/db/aliastables

I now can generate a new HOST-Alias and find it in the directory, but
a new ports -Alias isn`t there?

Title: Re: Aliases dosen't work(?)
Post by: tillsense on January 12, 2018, 07:05:09 pm
I can confirm that. aliases are currently not active here with r1.

cheers till
Title: Re: Aliases dosen't work(?)
Post by: franco on January 12, 2018, 08:01:03 pm
Thanks guys, one patch here that should fix the mkdir thing.

https://github.com/opnsense/core/commit/60e4e8080

I'll do more tests with the transition of aliases from 17.7.x to 18.1.x which seems to be the key element to the reported behaviour on Monday. As far as I know port aliases are special as they don't change, saving them from the GUI should fix their usage in any case.


Cheers,
Franco
Title: Re: Aliases dosen't work(?)
Post by: nivek1612 on January 14, 2018, 12:35:51 am
Hi Franco

Alias seems broken for me also at 18.1. I use them extensively to avoid me having to list the ports and IP address in the WAN rules

After a reboot I cannot access servers from outside my firewall

"configctl filter refresh_aliases"

fixes the problem

but the issue returns after reboot and another "configctl filter refresh_aliases" is required

I pulled in my config from 17.7.11 into a fresh install of 18.1r

I have applied patch 60e4e80
Title: Re: Aliases dosen't work(?)
Post by: franco on January 14, 2018, 03:29:44 pm
Hi nivek,

Can you dump this output for me?

# df -h

I think you are using /var MFS through Nano image maybe?


Cheers,
Franco
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 14, 2018, 04:07:14 pm
Not sure what Kev has but this is my test unit, it's configured with the same config as my live unit except for the number of ports. Running dev version and suffering the same issue.


root@gateway:~ # df -h
Filesystem         Size    Used   Avail Capacity  Mounted on
/dev/gpt/rootfs     26G    1.6G     22G     7%    /
devfs              1.0K    1.0K      0B   100%    /dev
tmpfs              3.1G     15M    3.1G     0%    /var
tmpfs              3.1G     84K    3.1G     0%    /tmp
devfs              1.0K    1.0K      0B   100%    /var/dhcpd/dev
root@gateway:~ #

Before you ask, nothing in /var/db/ apart from pkg. :}

after reboot 'configctl filter refresh_aliases' just freezes when I run it. CTRL-C and run it again it says OK.
Title: Re: Aliases dosen't work(?)
Post by: nivek1612 on January 14, 2018, 04:15:27 pm
Hi nivek,

Can you dump this output for me?

# df -h

I think you are using /var MFS through Nano image maybe?


Cheers,
Franco

Sorry Franco not back at home until mid week, hopefully Martins reply will suffice . I know we are using the same settings as i think I stole them from him originally :-)

Also like Martin
after reboot 'configctl filter refresh_aliases' just freezes when I run it. CTRL-C and run it again it says OK.
Title: Re: Aliases dosen't work(?)
Post by: franco on January 14, 2018, 04:31:57 pm
Well, was it an install or Nano image, do you know? :)
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 14, 2018, 04:33:27 pm
mine was a serial image. 17.7. then bounced up.
Title: Re: Aliases dosen't work(?)
Post by: nivek1612 on January 14, 2018, 04:35:47 pm
I did a fresh install of a 18.1 vga image from European mirror on a new SSD

I then restored my config from 17.7.11
Title: Re: Aliases dosen't work(?)
Post by: franco on January 15, 2018, 08:47:39 am
Ok so a real install... that's a bit peculiar but let's see what a new round of testing does.

18.1-RC2 is coming out tomorrow so that won't be in there completely but 18.1 should be fine later.


Cheers,
Franco
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 16, 2018, 01:04:15 am
Running RC2....

Confirm aliases are working. :)

Seems that an import of the config from my live device running 17.7.11 is not liked by the APU running 18.1-rc2, possibly because I use all the ports in a LAN bridge on the Qotom. It seems it gets all confused even if I have removed the bridge and extra ports.

Clean install and config took a while but it all works nicely, well done Franco and all.
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 16, 2018, 02:01:52 am
Hold hard....

Did work until reboot then lost all the aliases.

Note, The aliastables folder is there. ZZZzzz time now, I'll check the contents of it tomorrow ( later today )!
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 16, 2018, 09:35:32 am
Don't know if this will help. Here are the aliases bits from rules.debug from a 17 live machine and 18.1-rc2 - 17 first.

# Other tables
table <virusprot>
table <bogons> persist file "/usr/local/etc/bogons"
table <bogonsv6> persist file "/usr/local/etc/bogonsv6"

# User Aliases
table <MAIL_SERVER_IPS> { 192.168.1.30 2a02:8010:6228:0:1:2:3:4 }
MAIL_SERVER_IPS = "<MAIL_SERVER_IPS>"
MAIL_SERVER_PORTS = "{ 443 465 587 993 }"
table <Mail_Server_WAN_IP> { 82.68.104.101 }
Mail_Server_WAN_IP = "<Mail_Server_WAN_IP>"
table <SPAM_HERO_IPS> { 108.60.195.218 108.60.195.213 108.60.195.222 208.53.48.218 208.53.48.191 208.53.48.71 }
SPAM_HERO_IPS = "<SPAM_HERO_IPS>"
Spam_Hero_Ports = "{ 2525 }"
table <V4Geoip> persist file "/var/db/aliastables/V4Geoip.txt"
V4Geoip = "<V4Geoip>"
table <V6GeoIP> persist file "/var/db/aliastables/V6GeoIP.txt"
V6GeoIP = "<V6GeoIP>"
Web_server_ports = "{ 80 443 }"


--------------------------------------------
from 18

# Other tables
table <virusprot>
table <bogons> persist file "/usr/local/etc/bogons"
table <bogonsv6> persist file "/usr/local/etc/bogonsv6"

# User Aliases
table <Geoip> persist
Geoip = "<Geoip>"
table <MAIL_SERVER_IPS> persist
MAIL_SERVER_IPS = "<MAIL_SERVER_IPS>"
MAIL_SERVER_PORTS = "{ 443 465 587 993 }"
table <Mail_Server_WAN_IP> persist
Mail_Server_WAN_IP = "<Mail_Server_WAN_IP>"
table <SPAM_HERO_IPS> persist
SPAM_HERO_IPS = "<SPAM_HERO_IPS>"
Spam_Hero_Ports = "{ 2525 }"
Web_server_ports = "{ 80 443 }"

# Plugins tables



Title: Re: Aliases dosen't work(?)
Post by: franco on January 16, 2018, 04:04:14 pm
Hey,

I still need "df -h" output here...

There is one patch that helps with /var MFS but it's not on 18.1.r2: https://github.com/opnsense/core/commit/6536510


Cheers,
Franco
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 16, 2018, 04:36:45 pm
Yes, I applied that patch before I rebooted.

OK, what I'll do after work is put the test unit back online and see what happens, I'll also get you the output you need.
Title: Re: Aliases dosen't work(?)
Post by: franco on January 16, 2018, 04:57:41 pm
The first reboot won't work because that's when it creates the persistent directory :D

Please provide "df -h" output of that box to be sure we're not talking about different things.
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 16, 2018, 05:06:17 pm
Nuts, I thought it was just the aliastables folder...

OK, let me bring it online. Might be an hour and a half or so, depending on the London traffic.
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 16, 2018, 05:42:19 pm
Ok, got home quicker than expected, it's still blocking my aliases, here's the output you wanted/

root@gateway:~ # df -h
Filesystem         Size    Used   Avail Capacity  Mounted on
/dev/gpt/rootfs     26G    1.0G     23G     4%    /
devfs              1.0K    1.0K      0B   100%    /dev
devfs              1.0K    1.0K      0B   100%    /var/dhcpd/dev
Title: Re: Aliases dosen't work(?)
Post by: franco on January 16, 2018, 06:02:53 pm
I miss London. ;(

Are you sure this is alias related and not https://forum.opnsense.org/index.php?topic=6891.0 ?


Cheers,
Franco
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 16, 2018, 06:11:44 pm
I don't miss London, I'd much rather be on a the beach in the Maldives.

So why does it affect only Aliases, if I manually enter a rule with the IP addresses it works?
Title: Re: Aliases dosen't work(?)
Post by: franco on January 16, 2018, 06:14:16 pm
Just double-checking...
Title: Re: Aliases dosen't work(?)
Post by: nas7 on January 16, 2018, 11:19:24 pm
Hi.

By the way, I had this bug in 18.1.r1 but now it seems to be working.  I had to create the /var/db/aliastable directory manually and perform the alias refresh after each reboot.  Now with 18.1.r2 it seems to be working with my current configuration.  I will test it a bit more and let you know if the issue appears again.
Title: Re: Aliases dosen't work(?)
Post by: franco on January 17, 2018, 02:25:19 pm
There seems to be one more which is probably the one that bit marjohn56:

https://github.com/opnsense/core/issues/2102

I'll work on this in the next two days.

Thanks to all for the help and patience :)
Title: Re: Aliases dosen't work(?)
Post by: franco on January 18, 2018, 10:08:16 am
This should help:

# opnsense-patch bba40c97 947718b

For reference...

https://github.com/opnsense/core/commit/bba40c97
https://github.com/opnsense/core/commit/947718b


Cheers,
Franco



Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 18, 2018, 11:50:29 am
Looking good...looking very good. Nice one! ;D
Title: Re: Aliases dosen't work(?)
Post by: nivek1612 on January 18, 2018, 11:57:51 am
So before i test (I'm now home)

We are using 18.1rc2 with

just patches bba40c97 947718b

or are there others to apply as well


Title: Re: Aliases dosen't work(?)
Post by: franco on January 18, 2018, 12:00:59 pm
Yes, 18.1.r2 and these two patches on top in that particular order.


Cheers,
Franco
Title: Re: Aliases dosen't work(?)
Post by: nivek1612 on January 18, 2018, 12:42:28 pm
Ok its working

I had to go into the alias section of GUI and re-save one of them first then re-apply,

After reboot its survived but it seemed to take a while for my servers to be accessible from outside my LAN. It may be that 4G link I was using was slow. But i did notice that my CPU load on the lobby dashboard was quiet high for a while almost implying the firewall was busy doing something - Alias table rebuild maybe

I'll leave it a while then give it another reboot and check if the same happens 
Title: Re: Aliases dosen't work(?)
Post by: franco on January 18, 2018, 01:44:48 pm
That's a couple of OKs now from different directions, good. Only one issue left in the tracker, then we are ready for tomorrows code freeze.... hopefully.  8)


Cheers,
Franco
Title: Re: Aliases dosen't work(?)
Post by: cardins2u on January 18, 2018, 03:40:22 pm
Umm how do you apply these patches?

Just run?

# opnsense-patch #####


??


Title: Re: Aliases dosen't work(?)
Post by: nivek1612 on January 18, 2018, 03:45:44 pm
That's a couple of OKs now from different directions, good. Only one issue left in the tracker, then we are ready for tomorrows code freeze.... hopefully.  8)


Cheers,
Franco

Just rebooted noticed again that after reboot the CPU usage is circa 25% for 3-4 minutes that drops to the more normal 3-4%. Never noticed this at 17.7.11, but it may have been the same. Also during this time the servers are very sluggish to requests from WAN again not something I'd observed before but may have been the same. Once the CPU has calmed down though all is working as expected and back to normal.

So maybe there is a little bit of sub optimal process intensive stuff going on at 18.1 compared to 17 
Title: Re: Aliases dosen't work(?)
Post by: nivek1612 on January 18, 2018, 03:46:52 pm
Umm how do you apply these patches?

Just run?

# opnsense-patch #####


??

once your at 18.1.r2 from console issue

opnsense-patch bba40c97 947718b



Title: Re: Aliases dosen't work(?)
Post by: franco on January 18, 2018, 03:46:59 pm
@cardins2u: Yes, run the command on top of 18.1.r2.

@nivek1612: I noticed this too today. Even on 17.7.12 where nothing changed. I'm guessing GeoIP alias?


Cheers,
Franco
Title: Re: Aliases dosen't work(?)
Post by: nivek1612 on January 18, 2018, 04:00:55 pm

@nivek1612: I noticed this too today. Even on 17.7.12 where nothing changed. I'm guessing GeoIP alias?


You are correct o wise one:-) GeoIP alias for both IPv4 and IPv6 are in use
Title: Re: Aliases dosen't work(?)
Post by: franco on January 18, 2018, 04:03:39 pm
Ok, yeah, heavy stuff. The truth is the new system is a wee bit better as it does not completely stall bootup anymore. That gave me a good scare this morning on 17.7.12. :D


Cheers,
Franco
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 18, 2018, 05:15:57 pm
Well I've just had a severe hour or two, going round in ever decreasing circles...

I had trouble with getting my system to work everytime I pulled in the config from 17.7.11, running a ping against google failed after every reboot. The only way to get it back was to disable all the rules except the default ones, apply it and back would come the ping, re-apply, all works.... reboot, lost pings again. :(

looked at rules debug, ahah... the gateway is different on a reboot... it was, it jumps between different gateways on boot, but that was not the issue...

No, it was the rules, namely the IPV6 ICMP rule, which now needs to be IPv6 ICMP.v6

I'm going for a pint.... or several!
Title: Re: Aliases dosen't work(?)
Post by: franco on January 18, 2018, 05:58:39 pm
This doesn't sound so good, something in the code to fix or add compat for in the rules generation?


Cheers,
Franco
Title: Re: Aliases dosen't work(?)
Post by: nivek1612 on January 18, 2018, 07:03:44 pm
I see you snuck out 17.7.12 when I was testing 18.1r2

thought my eyes where going for a minute when I looked in the System:Firmware 
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 18, 2018, 07:42:56 pm
This doesn't sound so good, something in the code to fix or add compat for in the rules generation?


Cheers,
Franco

If you make  it compatible the you dont need the option in the drop down list, it is a bit misleading, you. Select ipv6 then ICMP,  that's what gets you, you type Ic and up comes ICMP, and it is, but for v4. I wonder how many that will catch.  ;)
Title: Re: Aliases dosen't work(?)
Post by: franco on January 18, 2018, 07:53:42 pm
@nivek1612: keeping the 17.7.x folk happy hopefully. :)

@marjohn56: Ah so a user-config error? It's true that ICMP and ICMP6 are different IANA numbers. I thought it was something transcending the worlds between 17.7 and 18.1 causing an incompatibility. Discussing ways forward, the ICMP could be appended with 4 or v4 to make it more clear and cleanly sidestep the issue of ambiguity?
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 18, 2018, 07:56:33 pm
Or just select ICMP and let the backend work out which, v4 or V6 has already been selected or the drop down list should be empty. Failing that, start both labels with ICMP and add V6 v4 to the end of the label. At present if you enter 'Ic' you only get ICMP for v4.
Title: Re: Aliases dosen't work(?)
Post by: franco on January 19, 2018, 09:44:05 am
Enter the guy who reported this and said he would have a CVE assigned:

https://github.com/opnsense/core/commit/ae677059d

The story behind it was that although pf accepts these values, it cannot filter them. The CVE was never assigned in the end, though.

Moral of the story: don't try to outsmart 99% of the users for their own benefit, the 1% will try assign a CVE for when a security-related option does not do what it says it should. Since we can't be sure, we should simply make the choices less ambiguous.


Cheers,
Franco
Title: Re: Aliases dosen't work(?)
Post by: cardins2u on January 20, 2018, 02:20:17 am
After running the two patch my alias still doesn't work.



root@OPNsense:~ # df
Filesystem      1K-blocks    Used    Avail Capacity  Mounted on
/dev/gpt/rootfs  92421240 4125852 80901692     5%    /
devfs                   1       1        0   100%    /dev
devfs                   1       1        0   100%    /var/dhcpd/dev
root@OPNsense:~ #
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 20, 2018, 08:38:48 am
Was this with a clean install of 18.1.rc1?
Title: Re: Aliases dosen't work(?)
Post by: cardins2u on January 20, 2018, 12:38:02 pm
yes - clean install and restored configurations.
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 20, 2018, 01:28:13 pm
are the aliases showing in rules.debug?
Title: Re: Aliases dosen't work(?)
Post by: marjohn56 on January 20, 2018, 01:45:48 pm
yes - clean install and restored configurations.

Did you do just the two patches mentioned here or three patches? There is one more that MIGHT affect you.

653651046fda533983bfc818d087fee5f073c7f6

That was one to make sure the aliastables folder is there and behaving.