Messages

Just to clarify, since routing to an interface isn't possible i have to create a gateway for that interface right?
the IP for that gateway could technically could be anything right?

Then routing the specific subnet through that gateway would send the traffic through that interface?

I'w reacently started looking in to Tailscale, it solves most of the problems i had with wireguard and i'd like to try it as my site-to-site solution, i understand that it's using wireguard-go and it will perform a bit worse then the kmod we have all gotten used to by now (it's not default but seems to be very common anyway)

I'w currently installed Tailscale from mimugmail''s repo and got it working fine by using the tailscale IP to reach the remote site, but whenever i try to add subnet routing i get stuck...
Subnets are properly advertised on both sites but i cant figure out how to route the corresponding subnet to the right interface.

I understand that it's not possible to route traffic to a specific interface, but setting up a gateway for that interface and route traffic to that gateway should work right? well i couldn't make it work...

I also tried to set up outbound-nat to translate the remote subnet to tailscale net but couldn't get that to work either.

I noticed that pfsense had some guides for this since they also got a tailscale plugin now, not sure it that plugin does stuff differently behind the curtains, but i could not get any closer to success with any of those guides.

Basically i'd like to solve this and create a Guide for it since i am positive this will be helpful for many people when more people realize the pros of this system.

Is subneting/exit node working on this package? what could i be doing wrong? and most importantly what would be a good way to troubleshoot the problem? i'w watched the "Live View" and the traffic actually leaves LAN network and goes in to the Tailscale network... but the remote site never gets any traffic.

Just wanted to throw in an update, i noticed that everytime i disabled wireguard the cert updated as normal, i'w gone through all the rules that routes through wireguard but cant find why any of them would route the firewall through any of the wireguard gateways.... but a floating rule that lets "this firewall" pass to "any" solved the issue...

guess i got some rule-table cleaning to do :D

UPDATE: i have a remote site with the same letsencrypt setup to the same cloudflare account so it's not an cloudflare issue, must be something in the system.

I had this problem a while ago where updating certs gave an error but it was fixable by restarting the acme plugin, now i get errors again for some reason and this time it wasn't as easy to fix.

This is what i noticed in the logs:

Code Select Expand

2021-06-06T00:06:34 acme.sh[64775] ] Please check log file for more details: /var/log/acme.sh.log
2021-06-06T00:06:34 acme.sh[25970] ] Error add txt for domain:_acme-challenge.test.kieeps.com
2021-06-06T00:06:34 acme.sh[25810] ] invalid domain
2021-06-06T00:06:30 acme.sh[92861] ] Adding txt value: iHKzdf4agek_fsKB1Eadhw85eE6-0RiWUY8lwdn1yss for domain: _acme-challenge.test.kieeps.com
2021-06-06T00:06:30 acme.sh[60519] ] Getting webroot for domain='test.kieeps.com'
2021-06-06T00:06:27 acme.sh[86742] ] Getting domain auth token for each domain
2021-06-06T00:06:27 acme.sh[35025] ] Single domain='test.kieeps.com'
2021-06-06T00:06:27 acme.sh[82914] ] Using CA: https://acme-v02.api.letsencrypt.org/directory

and if i force the update i get this:

Code Select Expand

2021-06-06T09:25:15 acme.sh[28153] ] Please check log file for more details: /var/log/acme.sh.log
2021-06-06T09:25:15 acme.sh[77405] ] Error, can not get domain token entry test.kieeps.com for dns-01
2021-06-06T09:25:15 acme.sh[26529] ] The new-authz request is ok.
2021-06-06T09:25:14 acme.sh[5930] ] Please refer to https://curl.haxx.se/libcurl/c/libcurl-errors.html for error code: 3
2021-06-06T09:25:11 acme.sh[70692] ] Getting new-authz for domain='test.kieeps.com'
2021-06-06T09:25:11 acme.sh[24216] ] Getting webroot for domain='test.kieeps.com'
2021-06-06T09:25:11 acme.sh[18035] ] Getting domain auth token for each domain
2021-06-06T09:25:11 acme.sh[74817] ] Single domain='test.kieeps.com'
2021-06-06T09:25:11 acme.sh[14721] ] Using CA: https://acme-v02.api.letsencrypt.org/directory
2021-06-06T09:25:11 acme.sh[93510] ] Can not init api.
2021-06-06T09:25:11 acme.sh[72878] ] Please refer to https://curl.haxx.se/libcurl/c/libcurl-errors.html for error code: 7

and this is the content of /var/log/acme.sh.log:

Code Select Expand

[Sun Jun  6 09:23:55 CEST 2021] Using config home:/var/etc/acme-client/home
[Sun Jun  6 09:23:55 CEST 2021] Running cmd: issue
[Sun Jun  6 09:23:55 CEST 2021] _main_domain='test.kieeps.com'
[Sun Jun  6 09:23:55 CEST 2021] _alt_domains='no'
[Sun Jun  6 09:23:55 CEST 2021] Using config home:/var/etc/acme-client/home
[Sun Jun  6 09:23:55 CEST 2021] default_acme_server
[Sun Jun  6 09:23:55 CEST 2021] ACME_DIRECTORY='https://acme-v02.api.letsencrypt.org/directory'
[Sun Jun  6 09:23:55 CEST 2021] DOMAIN_PATH='/var/etc/acme-client/home/test.kieeps.com'
[Sun Jun  6 09:23:55 CEST 2021] Using ACME_DIRECTORY: https://acme-v02.api.letsencrypt.org/directory
[Sun Jun  6 09:23:55 CEST 2021] _init api for server: https://acme-v02.api.letsencrypt.org/directory
[Sun Jun  6 09:23:55 CEST 2021] GET
[Sun Jun  6 09:23:55 CEST 2021] url='https://acme-v02.api.letsencrypt.org/directory'
[Sun Jun  6 09:23:55 CEST 2021] timeout=
[Sun Jun  6 09:23:55 CEST 2021] _CURL='curl --silent --dump-header /var/etc/acme-client/home/http.header  -L '
[Sun Jun  6 09:25:11 CEST 2021] Please refer to https://curl.haxx.se/libcurl/c/libcurl-errors.html for error code: 7
[Sun Jun  6 09:25:11 CEST 2021] ret='7'
[Sun Jun  6 09:25:11 CEST 2021] Can not init api.
[Sun Jun  6 09:25:11 CEST 2021] Le_NextRenewTime
[Sun Jun  6 09:25:11 CEST 2021] Using CA: https://acme-v02.api.letsencrypt.org/directory
[Sun Jun  6 09:25:11 CEST 2021] _on_before_issue
[Sun Jun  6 09:25:11 CEST 2021] _chk_main_domain='test.kieeps.com'
[Sun Jun  6 09:25:11 CEST 2021] _chk_alt_domains
[Sun Jun  6 09:25:11 CEST 2021] Le_LocalAddress
[Sun Jun  6 09:25:11 CEST 2021] d='test.kieeps.com'
[Sun Jun  6 09:25:11 CEST 2021] Check for domain='test.kieeps.com'
[Sun Jun  6 09:25:11 CEST 2021] _currentRoot='dns_cf'
[Sun Jun  6 09:25:11 CEST 2021] d
[Sun Jun  6 09:25:11 CEST 2021] _saved_account_key_hash is not changed, skip register account.
[Sun Jun  6 09:25:11 CEST 2021] Read key length:4096
[Sun Jun  6 09:25:11 CEST 2021] _createcsr
[Sun Jun  6 09:25:11 CEST 2021] Single domain='test.kieeps.com'
[Sun Jun  6 09:25:11 CEST 2021] Getting domain auth token for each domain
[Sun Jun  6 09:25:11 CEST 2021] d='test.kieeps.com'
[Sun Jun  6 09:25:11 CEST 2021] Getting webroot for domain='test.kieeps.com'
[Sun Jun  6 09:25:11 CEST 2021] _w='dns_cf'
[Sun Jun  6 09:25:11 CEST 2021] _currentRoot='dns_cf'
[Sun Jun  6 09:25:11 CEST 2021] Getting new-authz for domain='test.kieeps.com'
[Sun Jun  6 09:25:11 CEST 2021] _init api for server: https://acme-v02.api.letsencrypt.org/directory
[Sun Jun  6 09:25:11 CEST 2021] GET
[Sun Jun  6 09:25:11 CEST 2021] url='https://acme-v02.api.letsencrypt.org/directory'
[Sun Jun  6 09:25:11 CEST 2021] timeout=
[Sun Jun  6 09:25:11 CEST 2021] _CURL='curl --silent --dump-header /var/etc/acme-client/home/http.header  -L '
[Sun Jun  6 09:25:12 CEST 2021] ret='0'
[Sun Jun  6 09:25:12 CEST 2021] ACME_KEY_CHANGE='https://acme-v02.api.letsencrypt.org/acme/key-change'
[Sun Jun  6 09:25:12 CEST 2021] ACME_NEW_AUTHZ
[Sun Jun  6 09:25:12 CEST 2021] ACME_NEW_ORDER='https://acme-v02.api.letsencrypt.org/acme/new-order'
[Sun Jun  6 09:25:12 CEST 2021] ACME_NEW_ACCOUNT='https://acme-v02.api.letsencrypt.org/acme/new-acct'
[Sun Jun  6 09:25:12 CEST 2021] ACME_REVOKE_CERT='https://acme-v02.api.letsencrypt.org/acme/revoke-cert'
[Sun Jun  6 09:25:12 CEST 2021] ACME_AGREEMENT='https://letsencrypt.org/documents/LE-SA-v1.2-November-15-2017.pdf'
[Sun Jun  6 09:25:12 CEST 2021] ACME_NEW_NONCE='https://acme-v02.api.letsencrypt.org/acme/new-nonce'
[Sun Jun  6 09:25:12 CEST 2021] ACME_VERSION='2'
[Sun Jun  6 09:25:12 CEST 2021] Try new-authz for the 0 time.
[Sun Jun  6 09:25:12 CEST 2021] url
[Sun Jun  6 09:25:12 CEST 2021] payload='{"resource": "new-authz", "identifier": {"type": "dns", "value": "test.kieeps.com"}}'
[Sun Jun  6 09:25:12 CEST 2021] RSA key
[Sun Jun  6 09:25:13 CEST 2021] HEAD
[Sun Jun  6 09:25:13 CEST 2021] _post_url='https://acme-v02.api.letsencrypt.org/acme/new-nonce'
[Sun Jun  6 09:25:13 CEST 2021] _CURL='curl --silent --dump-header /var/etc/acme-client/home/http.header  -L  -I  '
[Sun Jun  6 09:25:14 CEST 2021] _ret='0'
[Sun Jun  6 09:25:14 CEST 2021] POST
[Sun Jun  6 09:25:14 CEST 2021] _post_url
[Sun Jun  6 09:25:14 CEST 2021] _CURL='curl --silent --dump-header /var/etc/acme-client/home/http.header  -L '
[Sun Jun  6 09:25:14 CEST 2021] Please refer to https://curl.haxx.se/libcurl/c/libcurl-errors.html for error code: 3
[Sun Jun  6 09:25:14 CEST 2021] _ret='3'
[Sun Jun  6 09:25:14 CEST 2021] code
[Sun Jun  6 09:25:14 CEST 2021] The new-authz request is ok.
[Sun Jun  6 09:25:15 CEST 2021] entry
[Sun Jun  6 09:25:15 CEST 2021] Not a wildcard domain, lets check whether the validation is already valid.
[Sun Jun  6 09:25:15 CEST 2021] Error, can not get domain token entry test.kieeps.com for dns-01
[Sun Jun  6 09:25:15 CEST 2021] pid
[Sun Jun  6 09:25:15 CEST 2021] No need to restore nginx, skip.
[Sun Jun  6 09:25:15 CEST 2021] _clearupdns
[Sun Jun  6 09:25:15 CEST 2021] dns_entries
[Sun Jun  6 09:25:15 CEST 2021] skip dns.
[Sun Jun  6 09:25:15 CEST 2021] _on_issue_err
[Sun Jun  6 09:25:15 CEST 2021] Please check log file for more details: /var/log/acme.sh.log

Im using cloudflare DNS verification and as of now i use the Global API just to make sure it's not a API permission error.

Did cloudflare change something or did acme.sh break?

How is the wireguard tunnel set up?
If you use the plugin to do the routing you should only need to set 0.0.0.0/0 as the "allowed ip" on the data center side of the config and the home config should have the datacenter subnet as "allowed ip".

One other way to do it in opnsense is to assign the wgX to a interface and it's own gateway, that way you could controll it a bit better.

This thread helped a lot! tbh i had no idea my IPS wasn't working ;D but after checking everythin was "Allowed", followed the instructions here and now it's back to working again :)

thanks for this :D

I'w recently enabled IPS on my firewall, wanted to wait untill everything else was set up so i could put all my focus on IPS for a while, And of course i got some stuff in the log that i dont really care about so i disabled them.

Now i get a message at the top of the page saying:

We strongly advise to use policies instead of single rule based changes to limit the size of the configuration. A list of all manual changes can be revised in the policy editor

What is this anyway? is it bad practice to disable single rules?

Isn't discord more for gamers?

I guess this was the intention, cant say thats the case anymore :) i'w grown to like Discord mostly for it's great webhook support and good bot API, but i can understand why most companies and such doesn't manage their community over discord due to the gamer image it holds.

There is a signal or telegram group and this forum and IRC.
In my opinion IRC and forum should do it for network people, not all of us have time to monitor multiple apps or services.

Absolutely :) if i could get the server/channel for irc i'd love to join :D never really understood telegram...
[Edit] nwm found it! :)

Wouldn't it be nice to have an somewhat official Discord channel? not bashing on forums as a platform but... I kind of miss the option to discuss ideas and such :)

I know a lot of people dislike Discord, but there are a lot of other platforms that could be used instead, i'd even prefer something like irc or matrix over forums now days :)

I also understand that a forum has a nice interface for users to search for known issues, but one doesn't have to rule the other one out does it?

couldbiut be the switch?

could be. STP or something like that.
can try to enable PortFast on PC interface

eally don't know where to start troubleshooting

can try to connect directly to OPN and compare the results

I will look in to that tonight :-) thanks for the response!

First try what Fright said.

After a quick search it looks like in S2500 switch you can configure DHCP relay. In that case you can set your OPNsense as the DHCP server. Maybe it helps to speed things up.

As a note I wouldn't have removed the unifi switches and APs. Exchanging the router for an OPNsense was a good idea though.

I'll try the relaying aswell :-) i still have all the unifi stuff stored in a box :-) the main reason i switched was to learn networking, i got 3 aruba s2500 and a 505 ap from work, we get a lot of networking equipment that no one wants to keep, since they are layer 3 i fugured I'd try to learn some routing as well :-)

Good news :-) it'll be fun to see if it has any noticeable improvements over the current implementation :-D

Keep upnthe great work!

I switched from a complete unifi system a while back in favor of opnsense, an aruba s2500 switch and a aruba AP.
It's really worked well out of the box and i'w really learned a lot abot netwoking, and that was the entire point of switching... But i have one problem that i'w had since the beginning (i think, i didnt really notice untill later though)

Whenever i start my computer and log in to windows it takes a while for the dhcp to assign an IP, and when checking the logs for the dhcp server i can clearly see the dhcp trying to answer the request from the computer over and over and over... Eventually it actually gets an ip after 3ish minutes of requesting.  What could be the problem in the setup? I haven't changed anything in the native dhcp, i have added more dhpc's though but i also wiped opnsense once and started over from scratch and STILL had this behavior... Also re-installed my PC since i figured it was the culprit but it didn't fix it, nor did changibg the betwork card in the pc, couldbiut be the switch? Is there a way to activate a more extensive debugging/logging on the dhcp service?

I really don't know where to start troubleshooting :-)

Can you post screenshots of firewall rules and nat outbound rules?

Any new on the progress of the bsd kernel? Read somewhere a while back that it was being pushed to kernel, did it ever land?

And will the plugin currently in opnsense move from userspace to kernel when it gets implemented? :-)