So many problems ... what is going on!?

[Stitch10925]

Hey everyone,

With everything I do I seem to encounter some bug of some sort which has made setting up OpnSense a real pain in the butt. 2 evenings of work only got me as far as a working internet connection and internet from my DMZ and LAN towards my WAN... quite abysmal result if you ask me.

What has been happening

Problem 1, getting the internet to work over PPPoE:

This was with the previous OpnSense version I was running. For the life of me I could not get PPPoE to connect. I found in some forums that this could be caused by a race condition in which a blocking system call in the PPPoE deamon would not return in a timely fashion causing the connection request to drop.

I solved this by connecting through the internet using a router I had, and connecting OpnSense WAN to the router. I then upgraded OpnSense and all plugins to the latest version (OPNsense 18.7.10_3-amd64). After that PPPoE connection was instantaneous.

Problem 2, getting Dynamic DNS to work:

When I set up Dynamic DNS to update my Namecheap domain I get the following error when the service tries to start:
parser error : Space required after the Public Identifier in /usr/local/etc/inc/plugins.inc.d/dyndns/phpDynDNS.inc on line 1072


I logged a bug for this here: https://github.com/opnsense/plugins/issues/1156
No solution yet.

Problem 3, getting HA Proxy to start:

Had this working on the previous install, but now I cannot get it to start. I don't know why. In the log it only says: "HAProxy failed to start" and "returned exit status 1". Amazingly helpful. So if anyone has some pointers, I am all ears.

I had HAProxy problems before, here an update to the latest version did the trick. However, this latest version does not seem to be working for me again. Probably something stupid that I did or did not do, but no idea what.

-- update: 2019-02-02 --

I finally got HAProxy to start. The issue was the listen address in the public pool. I had it set to 127.0.0.1, and it should have been 0.0.0.0. The help description provided is thus very misleading since it suggest the following: "Configure listen addresses for this Public Service, i.e. 127.0.0.1:8080 or www.example.com:443". This should really be adjusted.

Found the solution by going through a few tutorials I found.

What else hasn't worked so far?

- DHCP instabilities:
-- Setting a DHCP range from x.x.x.5 to x.x.x.20 and having the client work with x.x.x.4 (??), even after disconnecting or forcing a reconnect to get a new IP via DHCP
-- Static DHCP leases not being used/detected (lease is added, but when the client asks for an IP, DHCP does not seem to realize it is a lease and give another IP (probably related to above)... and yes, the MAC address was correct, I checked multiple times.
- Wifi on the laptop switched between internet/no internet connection. The network stays connected, but all of a sudden a ping won't work or a website won't load. After a few seconds it works again for a short time, then it starts all over. I do not have this on my cellphone and only had it on my laptop since I use OpnSense as firewall/dns. It seems like the firewall crashes or it can't route to the internet anymore, ... something like that.

-- update: 2019-02-02 --

It appears I had an IP conflict on my network which caused most of the strange behavior mentioned above. Once I got that resolved everything seems to be much more stable.

- Getting DHCP hostname registration in DNS (or at least the resolving of them) to work
- NetBIOS resolving (still need to look into this, but probably the same as above?)

-- update: 2019-02-02 --

Switching to DnsMasq, seems to work now, but it might have been a misconfiguration in Unbound DNS. Not sure, but don't care. It's working

What's next?

I don't know what else I will encounter. I still have quite a lot of things to set up:
- Configure HAProxy (once it feels like starting) --> DONE - no issues
- Setting up Let's Encrypt
- Setting up VPN
- Routing RDP from LAN to DMZ --> DONE - no issues

Haven't even touched on:
- Intrusion Detection
- Clam AV
- Port forwarding

Obviously I am not "a linux guy", but I have managed before. Either way, to me this feels like an awful lot of issues/problems for the little I have been able to set up in 2 days time.

Anyone experiencing issues like this or know about any solutions to these issues? I am really getting frustrated with this, especially since I am hosting a few websites and I obviously haven't been able to get them online.

Some final specs:

OPNsense 18.7.10_3-amd64
FreeBSD 11.1-RELEASE-p18
OpenSSL 1.0.2q 20 Nov 2018

OPNSense is running as a virtual machine under Proxmox version 5.2-7.
WAN, DMZ and LAN are separate hardware NICs, each bridged to their own virtual NIC on OPNSense

[Stitch10925]

Added updates inline of original post

[xames]

Welcome to opnsense stich, the name could be opnproblemseveryday firewall.

[Stitch10925]

Guess I'm not the only one with problems then...

Still having internet dropouts. Oddly enough only on wireless... or at least it seems that way, maybe wired recovers faster, I don't know. Either way, the internet connection is not stable. I read that this might be because of using VirtIO drivers for the virtual nics (my opnsense is running in proxmox), but as for now I see no improvements.

I also tried to block any DMZ connections coming into my LAN. Set blocking rules on the LAN as well as on DMZ interfaces and it's still coming through. Oddly enough, when I try to access LAN from DMZ, in the firewall, the rule gets listed under the LAN interface, which seems odd to me.

The path is:
DMZ > DMZ Gateway > LAN
=
192.168.10.4 > 192.168.10.1 > 192.168.20.7

And the resulting rule in the Live Log is:
LAN Source =192.168.10.1, Destination = 192.168.20.7

How is 192.168.10.1 LAN? It's the DMZ gateway...

Apparently you need to change the "LAN can go anywhere" rule in order to block the DMZ stuff... this makes no sense to me. So either something is wrong or my understanding of this stuff is worse than I thought.

I though DMZ firewall rules would apply to anything coming into DMZ, WAN firewall rules to anything coming into WAN and LAN firewall rules anything coming into LAN. So I had set my LAN firewall rule, that it should block anything coming from DMZ... doesn't work though.

It's really disheartening sometimes...

[ruffy91]

DMZ rules apply to all packets going into the DMZ interface. So in other words they apply to all packets going out the DMZ zone.
It's very confusing if you worked with zone firewalls in the past.

[franco]

Welcome to opnsense stich, the name could be opnproblemseveryday firewall.

If you want to be cheeky, please be cheeky somewhere else.

In general please *do not* assume everything is weird and not working. Break down posts into smaller issues. It even makes troubleshooting easier, because if you have individual problems you will find individual answers have been posted in these very forums over the years.


Cheers,
Franco

[xames]

You are true, sorry for that, bad day i think, sorry.


Enviado desde mi iPhone utilizando Tapatalk

[mitsos]

Had to login just to post this reply:
I've been here since the very beginning (that's *days* after the project was started). I've used OPNsense in every imaginable configuration, from PPPoE connections to routed connections, to load balancing connections, to E-V-E-R-Y---S-I-N-G-L-E configuration that there is out there. LANs, WANs, Guest networks, DMZs, satellite, carrier pidgeon links, EVERYTHING. That's why my post count is so low, I literally have zero problems with the product, using it over the span of a few years since its initial creation, in multiple installations, both for myself and clients.

I always set up unbound as a full resolver (I have to get my DNSEC fix (not the bug kind, the snort-up-the-nose kind of fix)): zero problems, never seen unbound fail, even when pushed by (literally) hundreds of clients.

I always set up suricata in IPS mode: absolutely zero problems with suricata, except one single occasion where an Exchange server was corrupting PDFs (it's a Microsoft product: what, do you expect it to work? Have you heard the other one about the chicken crossing the street?). Not really an OPNsense issue, is it?

Never had a problem connecting with openvpn. If you are using any kind of other vpn, you are holding it wrong. When I click that mouse button, the tunnel is set up and I start working. Every single time.

I use two different DDNS providers, never had a refreshed connection fail to update to either one. As a reminder: over a combined number of multiple installations at clients' offices.

Having an IP conflict and complaining that the product is "ZOMG!!!11oneeleven BROKEN" is like driving your car off a cliff and expecting your ABS to save you on the way down  ;-)

Life may seem difficult and every piece of software you are using may seem as if it was specifically written to make your life a living hell. Both of those statements could not be further from the truth. When things aren't working, always start troubleshooting from the beginning and work your way towards the problem's solution.

To the er...guy that mentioned every day problems: I've been here long enough to remember a product that fits that exact description. Installations failing to boot up after a regular update (not even major updates), the firewall deciding in the middle of the day to stop routing packets because why not, etc... The other "oldtimers" remember it as well, do you? ;-) Hint: it rhimes with OPNsense, but I can assure you it most definitely isn't OPNsense.

[franco]

Uh, oh, you woke Demetris. But he reads well.


Cheers,
Franco

PS: Hi old friend! 

[Stitch10925]

I finally got most of it resolved, but it has been a hassle. It seems to be a lot of issues coming together which made it very difficult to figure out what was going wrong. But at the moment everything seems to be working quite well.

So yes, not necessarily OpnSense issues, but at some places the help text could be updated to be more clear or give a more truthful example, this would have set me in the right direction to fix the problem. Now it was a lot of online searching and trial and error.

@deZillium

The Ip conflict, in retrospect, appeared to not be an IP conflict, but a problem with the Fritz!Box I am using. If I put the Fritz!Box in Client-IP mode (become part of the existing network) the issues appear. When I just set it to use the existing LAN connection (NAT the connection and provide own IP addresses) I do not have the disconnection problems anymore. I tried the same with another, newer, Fritz!Box model, this one does not seem to be suffering from that problem. So in that one Client-IP works fine. But those things are really sh*t to figure out.