better way to import network address ranges for traffic shaping?

[tswalker]

This is frustrating beyond belief.  Trying to get lists of IP ranges into a traffic shaper rule and though they are all valid.. copy/paste from text file alway results in the "invalid IP address" message in the UI.

Is there a better way to get a list of address ranges into a shaper rule?

[petrus]

Hello!

No sorry I don't have a solution, and I find it very frustrating as well. I have the same issue at many input fields.
I cant input more then a single item at once and I cant copy the whole list at once.

I am sure there is something I have overlooked, but I just cant figure it out.

Any hints would be greatly appreciated from someone more knwoledgeable!

Thanks!

Petrus

[petrus]

Hello,

the only thing I was able to figure out is as follows:

• exporting the config via a config backup: https://<your router>/diag_backup.php.
  
• editing with some text editor
  
• import the xml with reboot checked
  

The issue with this is the reboot, which takes pretty long. I changed Services/Unbound DNS/Blacklist/Whitelist Domains successfully.

However, as long  as I don't reboot, the new whitelist does not appear.
That might be a bug...

A better workaround would still be nice!

Petrus

[Fright]

@tswalker @petrus
could you please describe step by step: what you want, what you do and what you get?

[petrus]

Hi Fright,
thanks!  The details:
OPNsense 20.7.8-amd64 at my home, serving my family in lock down. My children and my wife are at home and we are living our lives mostly through diverse messaging/collaboration applications. Generally MS Teams, Skype, Zoom are used for school, Webex & Signal for other purposes. 
I tried out Unbound blacklists, which do work fairly well, but sometimes they randomly block some of those applications. Especially M$ related sites find their way into those blacklists, then those applications stop working.
So I have started to create a whitelist, which grew and grew. It became too long to type it in. I like to extract it  run it through sort -u make additions then put it back into that whitelisting field.
Here it is in the config backup:

Code: [Select]

<unboundplus>
      <miscellaneous version="0.0.2">
        <privatedomain>lan,vie.lan,kk.lan</privatedomain>
        <dotservers/>
      </miscellaneous>
      <dnsbl version="0.0.1">
        <service_enabled/>
        <enabled>1</enabled>
        <type>aa,ag,bla0,bla,blf,blm,blp,blr,blr0,bls,blt,bly,el,ep,sa,st,ws,wsu,wse</type>
        <lists/>
        <whitelists>admin\.onedrive\.com,ajax\.aspnetcdn\.com,ajax\.googleapis\.com,.*\.akadns\.net,.*\.akamaihd\.net,.*\.assets-yammer\.com,.*\.azure\.com,.*\.azurewebsites\.net,.*\.cloudapp\.net,.*\.cloudappsecurity\.com,.*\.cloudfront\.net,.*\.hockeyapp\.net,.*/live\.com,.*\.live\.com,.*\.localytics\.com,.*/login\.live\.com,.*/lw\.skype\.com,.*\.lync\.com,.*\.microsoft\.com,.*\.microsoftonline.com,.*\.microsoftonline\.com,.*\.microsoftonline-p.net,.*\.microsoftonline-p\.net,.*\.msedge\.net,.*\.*\.msidentity\.com,.*\.msidentity\.com,.*\.msocdn\.com,.*\.oaspapps\.com,.*\.office365\.com,.*\.office\.com,.*\.office\.net,oneclient\.sfx\.ms,.*\.onmicrosoft\.com,.*\.outlook\.com,.*\.outlookgroups\.ms,.*\.sfbassets\.com,.*\.sharepoint\.com,.*\.sharepointonline\.com,.*/skypeassets\.com,.*/skype\.com,.*\.skype\.com,.*\.skypeforbusiness\.com,.*/skype\.net,.*\.svc\.ms,.*\.sway-cdn\.com,.*\.sway\.com,.*\.trafficmanager\.net,.*\.uservoice\.com,.*/web\.skype\.com,.*\.windowsazure\.com,.*\.windows\.net,.*\.yammer\.com,.*\.yammerusercontent\.com</whitelists>
      </dnsbl>
    </unboundplus>

Petrus

Ps. no I don't like most of those applications myself, so much so, that we started to use them only halve a year ago as they have started to get useful on our Linux Desktops. Since everybody else is using them, we have to use them too... 

[petrus]

Hi,
sorry for the confusion: my post does not apply to the Options of Unbound only. I think this is a general problem: there is no easy way to copy or insert a list into any field in the Opnsense WebGUI!
Or I might not be aware of that.

BR
Petrus

[Fright]

well, as far as I can see pasting data from the clipboard is supported on tokenized select fields. I checked on creating Alias (Network(s)).
now the comma delimiter is used, so I prepared the list by separating the values with commas. everything worked for me. and its passed all validations
I do not know why @tswalker's data does not pass validation, so I asked for a step-by-step description of the process.

another question that I'm asking developers to replace the delimiter so that regexes with quantifiers can be used.
https://github.com/opnsense/core/issues/4497
if the devs agree, it will be necessary to use a different separator when preparing the list before copying to the clipboard and pasting into the field

about copying - I saw no evidence that this is supported in tokenize2, so copying from the config seems logical to me (adding such a feature is technically possible, but is it really important?)

@petrus
specifically for unbound dnbls whitelists values you can try this in browser console to get comma separated list:

Code: [Select]

$("#dnsbl\\.whitelists").val().join();

[petrus]

Hello Fright,
thanks, I appreciate your answer!

It might just be me being not a dev, but your solution looks to me more tedious then downloading the config.xml. 

I think it is always important, that you can freely use the copy and paste functionality in whatever field you are on.
For Firewalls it's even more important then somewhere else, as they deal with a lots of ports/objects/entries.
Giving the user the ability to copy/paste something quickly, without the use of an API, is as important as ever.

This is one of the main reasons why people don't like to use the Windows GUI or are reluctant to stop using ASA Firewalls and buying a Firepower FW (that has APIs, as well, but copy and paste combined with grep, sed, awk, vim are something more readily accessible) ...

Maybe there is nothing wrong with tokenize2, but the GUI should just give the users the possibility switch it off if that gets into their way?

Regards!

Petrus

[Fright]

Hi
I'm not a dev either, but in my opinion it's always a conflict: "make a lot of convenient and beautiful functions" vs "keep it simple" vs "does it work? don't touch!" 

well, I quickly gave it a try - it's not that hard to add such functionality
https://drive.google.com/file/d/1Wrht-UtU-8FM6GCJWjJVEynGhuKYcUm3/view

The only question is how the devs look at it.
In any case, in my opinion the idea is interesting enough to make a FR on github

[petrus]

Hi Fright,

that would be great! As I have seen in that clip, you have already got some code for a copy to clipboard code!
That's not exactly what I mean (to give the option to the user to switch of the automatism for the input fields), but it's certainly a nice solution to this problem! Let's hope it'll find it's way into some forthcoming release.

Thanks!

Petrus

[Fright]

Hi
I just made sure for myself that it is not difficult to add this feature (adding a "copy to clipboard button") to the MVC.
I really think it's better for you to post the FR  yourself (I can join the discussion if you want or need to):
- no one can articulate your needs better than you
-I already have a bunch of requests and it seems to me that I already tired the devs a little