Topics

1

22.1 Legacy Series / Virtual IP Bind HAProxy Issue

« on: March 18, 2022, 12:58:33 pm »

Just updated to OPNsense 22.1.3-amd64 this morning and finally resolved an issue with HAProxy not starting.

HAProxy was working fine before the update, but after HAProxy would not start.  Config file reported no errors, but I could not find anything in log files as to why HAProxy was not starting.  So I finally connected via terminal and tried to start HAPRoxy manually and got an error that it could not bind to my internal Virtual IP.

SO I went to the VirtualIP settings in the GUI and confirmed that I had Allow service binding turned on.  As a test I re-saved and applied the settings.  After that HAProxy started up just fine.

I rebooted to test and the same thing happened.  I had to manually re-save the Virtual IP and apply the settings and the HAPRoxy could be started.

Is there some new setting somewhere that I need to activate to get this to work automatically like before?

2

General Discussion / Proxmox+OPNSense+VM Routing Issues?

« on: June 30, 2021, 03:59:46 pm »

I have been testing a setup of OPNSense running on Proxmox as a potential replacement for my old pfsense system that never seemed to be able to achieve 1 GB throughput (or near enough) for my fiber internet.  I believe it is hardware choking it as I can not seem to get it over the hump.

I am currently testing the new Proxmox system behind my pfsense system as I have some complicated setups with VLANS and HAPRoxy that I want to get setup on OPNsense before I transition to ensure limited downtime.

In my existing setup, I have a separate Ubuntu server running some tasks on separate hardware.  My hope is to roll that into Proxmox as a VM to reduce the amount of boxes.  So I have generated the Ubuntu VM on Proxmox.  This is where I am running into issues.  I cannot seem to get my Ubuntu VM (UVM) to connect to the internet.

My test setup is this:

WAN -> PFSense -> LAN (17216.1.1/16) -> HomeLab VLAN (10.0.1.1/24) -> UniFi switch w/ VLAN Tagged -> New Proxmox box w/ 6 ports

Proxmox box setup
Port 1 bridge setup with 10.0.1.254 for managing Proxmox from my LAN network
Port 6 bridged (vmbr6) to act as WAN port for OPNSense (vtnet0)
Port 5 bridged (vmbr5) to act as  LAN port for OPNSense (vtnet1)

I have tried the three following ways with varied results:

Option 1
- UVM w/ virtual port based on vmbr5 (port 5 above), tagged with VLAN 105
- Setup VLAN 105 on OPNSense
- Create OPNSense interface for VLAN w/ LAN Parent interface with IP 10.105.0.253
- DHCP 10.105.0.1/29 for VLAN Interface

In this setup, I get DHCP address assigned to UVM as 10.105.0.249, DNS seems to function as it revolves google.com to IP with ping.  However, I do not have internet connection, as ping fails.  I cannot curl or anything else.  I can ping OPNSense gateway at both 10.105.0.253 (setup as interface IP in OPNSense for VLAN 105) and 10.0.1.253 (how I access OPNSense from my main LAN).  But beyond that, no route is established.

Option 2
- Create new vmbr7 bridge in Proxmox, not tied to port with CDIR 10.105.0.1/29
- UVM w/ virtual port based on vmbr7, no VLAN tag as vtnet03
- Create OPNSense interface for  vtnet03 with IP 10.105.0.253
- DHCP 10.105.0.1/29 for vtnet03 Interface

In this setup, I get DHCP to assign IP to UVM, but nothing else works.  No DNS, no internet, cannot ping anything.

Option 3
- UVM w/ virtual port based on LAN (vmbr5 above), no VLAN tag
- Gets IP assigned from pfsense VLAN for HomeLab

In this setup, I get IP assigned from my pfsense box and I can DNS, reach internet, etc.  But of course, can ping OPNSense on 10.0.1.253.  In this scenario, I am essentially bypassing OPNSense.  So , this will not ultimately work in my scenario when I remove pfsense.

So, why can I not reach internet.  It seems like either 1 or 2 above would work.  I am not sure why the internet route is breaking down.  Based on my searches on this, I see posts saying to ensure there is a route from the VLAN interface to the gateway (WAN in my case?).  However, auto outbound NAT sows a connection between Ubuntu interfaces in both option 1 or 2 above.  Does that not establish that route? 

Note, in these scenarios, OPNSense can reach the internet, resolve DNS, etc.  So the breakdown apparently seems to be within OPNSense itself, but I cannot figure out where.  I do not think I am doing anything exotic that I haven't already done in my original pfsense setup.  Th only difference is that the Ubuntu setup is now virtual inside Proxmox instead of physical hardware.  I am just duplicating that setup (option 1) in this test setup.

I have turned off Firewall in Proxmox on the virtual ports to ensure it wasn't causing the issues.  I have also made sure I have Pass all traffic on the the interfaces in OPNSense in both option 1 and 2 above. 

Any help on getting this working would be greatly appreciated. 

3

21.1 Legacy Series / Proxmox Install Issues

« on: June 27, 2021, 11:21:17 pm »

I am running into a frustrating issue that I cannot seem to resolve and would appreciate any help or nudges in direction of tutorials that may help me resolve.

First a little background:
I have been running PFSense for several years on bare metal.  I have 1 gig fiber through AT&T and have never seemed to get close to my supposed 1GB limit.  I seem to be hovering around 300 or so.  I figured it was the hardware I was running on.  So I recently bought a new 6 port, i7 device from aliexpress.

I also have a separate Ubuntu server running on an Intel Nuc.  Through my investigation of new firewall hardware,  I cam across Proxmox.  So I decided it would be nice to roll my firewall and my Ubuntu as VM's.  I also came across OPNSense due to all the shenanigans related to pfsense changes.  Looked into it and thought it would be a good time to transition.

I have been trying for several days to get up and running a test.  Now I still have my pfsense box running on the front end and created a separate van to get the proxmox/opnsense/ubuntu system up and running before I swap it out.  I don't know if this is what is causing my issues, but am thinking not based on the issues I am encountering.

Hardware specs
CPU: Intel(R) Core(TM) i7-10510U CPU @ 1.80GHz
RAM: DDR4 32GB Kit (2 x 16GB) 2666MHz PC4-21300 CL19
HD:  mSATA SSD 256GB
Ethernet: 6 intel 1g ports
Proxmox version: 6.4-4
OPNSense version: 21.1

I have proxmox up and running.  I have tried to install OPNsense several times.  I always seem to get stuck after the second reboot.  I do the install, set up my interfaces, etc. reboots after install and do final setup.  Everything runs smooth.  It is after another reboot where things go hay-wire.

After initial install and first reboot, I have no issues checking for updates, or anything else.  After that next reboot, I can no longer access updates.  I can ping, but updates do not work.

For example, I can run the following commands from cli with no issues prior to the first reboot:
opnsense-update -M
pkg update -f

I can also run the update check from the web gui and no issues.

I have tried on subsequent installs not to update the system to latest 21.1.7, thinking that was the issue, and still it happens.

After the second reboot, web gui updates spins and spins and sometimes I get a timeout and sometimes I get no updates available.  I have tried several different mirrors, and all seem to have the same issue.

If I run the "pkg update -f" from cli, it takes up to 30-45 minutes to download packagesite.txz.  Prior to the second reboot, it downloads in a snap.

I downloaded PFSense and set it up as a VM to see if it has the same issues.  So far, I do not see the same behavior, but it doesn't have the same update methods as opnsesne, so not really sure what to check.  I even have rebooted it several times and even did a fetch of https://pkg.opnsense.org/FreeBSD:12:amd64/21.1/latest/packagesite.txz and no access or speed issues were encountered.

I have searched and I see a lot about ipv6 when it comes to seeing no packages or orphaned packages (something I also see after the second reboot).  I am not really running ipv6 and have no real need to up my understanding of it as my needs do not require it.  My pfsense box with AT&T doesn't have it setup.  And it doesn't appear to be affecting the pfsesne VM in this regard.

But, I don't think this is really my issue, as it works great after 1 reboot.  It is the second reboot, every time that causes the apparent shutdown.

So what changes after a second reboot to cause my system to effectively stop working?  I would really like to resolve this as I like OPNSense organization much better and not liking the direction pfsense is going with the whole proprietary bent.