1
20.7 Legacy Series / Re: Where disable remote syslog?
« on: January 08, 2021, 06:20:27 pm »
This looks to still be a problem as of 20.7.7_1. Did anyone ever find a solution beyond exporting and editing the config by hand?
Show Posts
This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.
Pages: [1] 2
2
19.1 Legacy Series / Re: Network performance and latency issues with haproxy
« on: April 30, 2019, 10:47:30 pm »
Here it is. redacted_http_test is the no-ssl backend I've been testing with.
Code: [Select]
#
# Automatically generated configuration.
# Do not edit this file manually.
#
global
# NOTE: Could be a security issue, but required for some feature.
uid 80
gid 80
chroot /var/haproxy
daemon
stats socket /var/run/haproxy.socket level admin
nbproc 1
nbthread 1
tune.ssl.default-dh-param 1024
spread-checks 0
tune.chksize 16384
tune.bufsize 16384
tune.lua.maxmem 0
log /var/run/log local0
defaults
log global
option redispatch -1
timeout client 30s
timeout connect 30s
timeout server 30s
retries 3
# WARNING: pass through options below this line
mode http
option httplog
# autogenerated entries for ACLs
# autogenerated entries for config in backends/frontends
# autogenerated entries for stats
# Frontend: redacted_ssl ()
frontend redacted_ssl
bind *:443 name *:443 ssl crt-list /tmp/haproxy/ssl/5cc4fcfb4d50d1.96664841.certlist
mode http
option http-keep-alive
option forwardfor
# tuning options
timeout client 30s
# logging options
# ACL: host_redacted
acl acl_5cc4fe5883d959.03367346 hdr_end(host) -i redacted
# ACTION: map_redacted
use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/5cc509feaa78f3.47022982.txt)] if acl_5cc4fe5883d959.03367346
# Frontend: letsencrypt ()
frontend letsencrypt
bind *:80 name *:80
mode http
option http-keep-alive
default_backend acme_challenge_backend
# tuning options
timeout client 30s
# logging options
# Frontend: redacted_http_test ()
frontend redacted_http_test
bind *:8080 name *:8080
mode http
option http-keep-alive
default_backend server1_apache
option forwardfor
# tuning options
timeout client 30s
# logging options
# ACL: host_redacted
acl acl_5cc4fe5883d959.03367346 hdr_end(host) -i redacted
# ACTION: map_redacted
use_backend %[req.hdr(host),lower,map_dom(/tmp/haproxy/mapfiles/5cc509feaa78f3.47022982.txt)] if acl_5cc4fe5883d959.03367346
# Backend: acme_challenge_backend (Added by Let's Encrypt plugin)
backend acme_challenge_backend
# health checking is DISABLED
mode http
balance source
# stickiness
stick-table type ip size 50k expire 30m
stick on src
# tuning options
timeout connect 30s
timeout server 30s
http-reuse never
server acme_challenge_host 127.0.0.1:43580
# Backend: server1_apache ()
backend server1_apache
# health checking is DISABLED
mode http
balance source
# stickiness
stick-table type ip size 50k expire 30m
stick on src
# tuning options
timeout connect 30s
timeout server 30s
http-reuse never
server server1_apache backend.lan:80
# Backend: backend_grafana ()
backend backend_grafana
# health checking is DISABLED
mode http
balance source
# stickiness
stick-table type ip size 50k expire 30m
stick on src
# tuning options
timeout connect 30s
timeout server 30s
http-reuse never
server backend_grafana grafana.lan:80
3
19.1 Legacy Series / Network performance and latency issues with haproxy
« on: April 28, 2019, 05:44:21 pm »
First, I'm quite impressed by the HAProxy and Let's Encrypt plugins, and how they work together.
I'm running opnsense on an APU2 board, and I'm noticing:
For example, the backend web server (a basic Apache setup) can saturate the 1Gb network connection with no issues. If I do a straight HTTP proxy through HAProxy in opnsense (no SSL offloading), performance caps out at about 200Mbit. I expected closer to 300 or 400 Mbits. During a transfer, the HAProxy process is using close to 100% of a CPU core (the board has 4 cores) with 50% system CPU usage. If I ping a host during the speed test from opnsense, latency goes up significantly for all connections going through the firewall.
So far, I noticed that at http://www.haproxy.org it's mentioned that pf causes quite a performance hit compared to Linux, but it's not clear how recent that note is.
Any ideas on how to solve the latency issues? I can live with slower transfer performance, but lag spikes will be a deal breaker for me.
I'm running opnsense on an APU2 board, and I'm noticing:
- Slow transfer rates with high CPU usage
- Latency spikes for unrelated network traffic when HAProxy is under load
For example, the backend web server (a basic Apache setup) can saturate the 1Gb network connection with no issues. If I do a straight HTTP proxy through HAProxy in opnsense (no SSL offloading), performance caps out at about 200Mbit. I expected closer to 300 or 400 Mbits. During a transfer, the HAProxy process is using close to 100% of a CPU core (the board has 4 cores) with 50% system CPU usage. If I ping a host during the speed test from opnsense, latency goes up significantly for all connections going through the firewall.
So far, I noticed that at http://www.haproxy.org it's mentioned that pf causes quite a performance hit compared to Linux, but it's not clear how recent that note is.
Any ideas on how to solve the latency issues? I can live with slower transfer performance, but lag spikes will be a deal breaker for me.
4
18.7 Legacy Series / Stale WAN IP used for SIP NAT mapping (Dynamic state reset)
« on: November 23, 2018, 07:27:30 pm »
OPNSense is connected via DSL / pppoe for it's WAN connection. I have an OBiHAI SIP bridge for VoIP access. If the WAN IP changes, the old NAT mapping are still used, causing packets to be sent with the wrong source IP address. This breaks WAN connectivity until the states are killed.
I've verified the wrong source IP is being sent from a packet capture of the pppoe interface. What's surprising to me is that nothing else other than this one mapping appears to be affected by this.
In the firewall advanced options, I found "Dynamic state reset" which was not enabled. Turning that on fixed the stale mappings. Is there any reason why that option shouldn't be on by default?
This could be related to switching ISPs from one that used DHCP to DSL and PPPoE. Is this a setting normally set during the setup wizard, which would be missed if you manually changed WAN settings after the initial install?
- In the firewall states dump, I filter on port 5060 to see the inbound and outbound mappings.
- Note your current WAN IP, and click "reload" at the WAN interface in the overview to force a new connection.
- After the IP has renewed, reload the states dump and note the outbound IP address is the old IP address and not the new one.
- Killing the states restores WAN connectivity to the SIP bridge.
I've verified the wrong source IP is being sent from a packet capture of the pppoe interface. What's surprising to me is that nothing else other than this one mapping appears to be affected by this.
In the firewall advanced options, I found "Dynamic state reset" which was not enabled. Turning that on fixed the stale mappings. Is there any reason why that option shouldn't be on by default?
This could be related to switching ISPs from one that used DHCP to DSL and PPPoE. Is this a setting normally set during the setup wizard, which would be missed if you manually changed WAN settings after the initial install?
5
17.7 Legacy Series / Re: Cannot ping ISPs DNS servers since upgrade from 17.1.11 to 17.7
« on: August 08, 2017, 04:28:26 pm »
The patch didn't do anything, regardless of the override setting. I did have that enabled.
However, this pointed out a better workaround for now - if I statically set DNS, the bad routes aren't added.
However, this pointed out a better workaround for now - if I statically set DNS, the bad routes aren't added.
6
17.7 Legacy Series / Re: Cannot ping ISPs DNS servers since upgrade from 17.1.11 to 17.7
« on: August 07, 2017, 05:55:38 pm »
I've got exactly the same problem since upgrading. Deleting the two routes for each DNS server fixes the problem, and renewing the WAN IP brings them back.
7
17.1 Legacy Series / Re: How is netflow data supposed to be rotated?
« on: June 27, 2017, 02:32:56 am »
One more bit - I just noticed that the UI isn't showing any data for the past 2 weeks. However, disk use has been steadily increasing in that time.
8
17.1 Legacy Series / How is netflow data supposed to be rotated?
« on: June 27, 2017, 02:26:35 am »
I re-enabled netflow about a month ago, and ever since it's been slowly growing in disk use. This is on a home connection, so it's not /that/ much traffic to log. flowd and flowd_aggregate are both running:
root@blackbox:/var # service flowd status
flowd is running as pid 36517 36532.
root@blackbox:/var # service flowd_aggregate status
flowd_aggregate is running as pid 42215.
root@blackbox:/var # du -sh log/flowd* netflow
3.6G log/flowd.log
11M log/flowd.log.000001
11M log/flowd.log.000002
11M log/flowd.log.000003
11M log/flowd.log.000004
12M log/flowd.log.000005
11M log/flowd.log.000006
11M log/flowd.log.000007
11M log/flowd.log.000008
11M log/flowd.log.000009
11M log/flowd.log.000010
5.3G netflow
I'm going to have to reset the data manually as I'm almost out of disk space. My understanding is simply running flowd_aggregate should be enough here. Is there anything else to check?
root@blackbox:/var # service flowd status
flowd is running as pid 36517 36532.
root@blackbox:/var # service flowd_aggregate status
flowd_aggregate is running as pid 42215.
root@blackbox:/var # du -sh log/flowd* netflow
3.6G log/flowd.log
11M log/flowd.log.000001
11M log/flowd.log.000002
11M log/flowd.log.000003
11M log/flowd.log.000004
12M log/flowd.log.000005
11M log/flowd.log.000006
11M log/flowd.log.000007
11M log/flowd.log.000008
11M log/flowd.log.000009
11M log/flowd.log.000010
5.3G netflow
I'm going to have to reset the data manually as I'm almost out of disk space. My understanding is simply running flowd_aggregate should be enough here. Is there anything else to check?
9
17.1 Legacy Series / Re: collectd network plugin is missing encryption support
« on: June 09, 2017, 11:10:38 pm »
It did! Thanks.
10
17.1 Legacy Series / Re: collectd network plugin is missing encryption support
« on: May 24, 2017, 03:15:30 pm »
Of course, the above ldd paste was on the colletctd binary, not network.so 
libgcrypt is in the network plugin, using the default build options:
libgcrypt is in the network plugin, using the default build options:
Code: [Select]
root@blackbox:~ # ldd /usr/local/lib/collectd/network.so
/usr/local/lib/collectd/network.so:
libgcrypt.so.20 => /usr/local/lib/libgcrypt.so.20 (0x2e31a0b000)
libgpg-error.so.0 => /usr/local/lib/libgpg-error.so.0 (0x2e31d1d000)
libc.so.7 => /lib/libc.so.7 (0x2e3102c000)
11
17.1 Legacy Series / Re: Are backups of "volatile" files required on reboots for non-nano installs?
« on: May 24, 2017, 03:09:45 pm »
Well, at one point, the disk (16GB) actually filled up completely with netflow data and I had to manually clear it out. I haven't replicated that. /var doesn't look to be RAM mounted, which is why I don't see why there needs to be a backup and restore process at all.
12
17.1 Legacy Series / Re: collectd network plugin is missing encryption support
« on: May 22, 2017, 02:24:38 pm »
Here's ldd after install:
You're right, there is no gcrypt support (nor shown in pkg info as well). However, encryption support is working fine. Perhaps there's multiple libraries it can use to support it, but I'm not finding anything obvious. I'd probably lean towards enabling gcrypt just to match what previous builds did.
I have a cron job for a nightly firmware upgrade. It looks like collectd has been replaced by the binary version on me from the repos. I thought I locked the package after install, but perhaps I didn't. Is there anything else to do when installing ports to keep them in place?
Code: [Select]
root@blackbox:/usr/ports/net-mgmt/collectd5 # ldd /usr/local/sbin/collectd������ /usr/local/sbin/collectd:
libm.so.5 => /lib/libm.so.5 (0x3442ba34000)
libthr.so.3 => /lib/libthr.so.3 (0x3442bc5f000)
libstatgrab.so.10 => /usr/local/lib/libstatgrab.so.10 (0x3442be87000)
libdevstat.so.7 => /lib/libdevstat.so.7 (0x3442c099000)
libltdl.so.7 => /usr/local/lib/libltdl.so.7 (0x3442c29f000)
libc.so.7 => /lib/libc.so.7 (0x3442af14000)
libkvm.so.7 => /lib/libkvm.so.7 (0x3442c4a8000)
libelf.so.2 => /lib/libelf.so.2 (0x3442c6b7000)You're right, there is no gcrypt support (nor shown in pkg info as well). However, encryption support is working fine. Perhaps there's multiple libraries it can use to support it, but I'm not finding anything obvious. I'd probably lean towards enabling gcrypt just to match what previous builds did.
I have a cron job for a nightly firmware upgrade. It looks like collectd has been replaced by the binary version on me from the repos. I thought I locked the package after install, but perhaps I didn't. Is there anything else to do when installing ports to keep them in place?
13
17.1 Legacy Series / Are backups of "volatile" files required on reboots for non-nano installs?
« on: May 21, 2017, 02:35:43 am »
I have OPNsense installed on a 16GB SSD. I noticed that reboots were taking a long time. It turns out the culprit was the netflow database; gzip'ing was taking 5-10 minutes, blocking a reboot. This commit illustrates the code fairly well. Any reason I can't just comment them out?
14
17.1 Legacy Series / Re: collectd network plugin is missing encryption support
« on: May 18, 2017, 10:45:50 pm »
Turns out that running the build from 16.7 didn't throw any errors, but did send wrong data in the networking plugin. Following https://forum.opnsense.org/index.php?topic=2004 I installed collectd from ports and crypt support is working fine.
15
17.1 Legacy Series / Re: dhclient exits on WAN IP renewal
« on: May 11, 2017, 08:15:34 pm »
It looks like this is still an issue on 17.1.6. Is there an upstream issue I can track?
Pages: [1] 2