1
General Discussion / Emulation of VSAT speed and latency with OpnSense
« on: January 15, 2017, 06:00:30 pm »
Hello dear community members. A few weeks ago I was charged with a task to find a way to limit our office Internet to a specific rack we are building with the bandwidth and latency of a slow satellite connection (VSAT). I was playing a lot and read tons of howto's and followed tutorials one after the other, however I had some issues that I could not resolve myself. I've been working as an IT for all my life, however I did never played with OpnSense or used any other software as bandwidth limiting/shaping.
Let me give some more detailed information and what steps I took to make it work. Most likely I have made a simple logic mistake, however I am unable to find it.
The WAN interface of the box is behind our private network and have the following info:
IP: 192.168.9.112
Mask: 24
Gateway: 192.168.9.254
The LAN interface of the box is set to communicate with the newly built rack and has the following details:
IP: 192.168.170.1
Mask: 24
Gateway: N/A
The VSAT speed I am trying to emulate is: Up: 3Mbps/Down 1Mbps with latency ~700ms.
The goal is to allow traffic from any LAN interface to Internet using the bandwidth and latency limiter as well as to allow connections to the new rack LAN from the WAN interface (which is our private network)
Following a few simple tutorials, I created two limiters in Firewall->Traffic Shaper->Limitters. One was called VSATLimitUP having bandwidth of 3Mbps and 350ms latency. The other was called VSATLimitDown having 1Mbps limit with latency 350 (the reason I put 350 instead of 700 was the fact that the ping reply became doubled prior to in/out communication so I split the desired latency in two and set the result to both limiters).
From there everything seemed to be simple, creating a rule in the firewall with the following settings:
Action: Pass
Interface: LAN
Protocol: any
Source: LAN Subnet
Destination: WAN Subnet
Description: LAN to WAN over VSAT
In = VSATLimitUP & Out = VSATLimitDown
Moved the rule above all others and tested to ping. From here I had different results - once I had full LAN speed of <1ms to lan/wan or had a proper latency of 700ms to any IP (lan or wan). It is good if I have a ping to WAN with the latency, however to the LAN I should not be limited and should have <1ms to any device behind 192.168.170.0/24 network.
A similar WAN rule was created:
Action: Pass
Interface: WAN
Protocol: any
Source: WAN Subnet
Destination: LAN Subnet
Description: WAN to LAN over VSAT
In = VSATLimitDown & Out = VSATLimitUP (reversed order of the LAN rule as per most manuals and tutorials)
As a result, I was able to get some traffic being limited, however it did not apply to the limits I set (3Mbps = ~250-300KBps and 1Mbps ~110-120KBps) I get around 50-60K of download speed and is not affected by any change in the limitters.
On the other hand, any attempt to access the LAN network from the WAN (192.168.9.X to 192.168.170.X) is blocked even though on the client remote windows machine I added a route: route add 192.168.170.0 mask 255.255.255.0 192.168.9.112
So far I am able to access the OpnSense over the WAN (I added another rule from any to This Firewall) and ping it, but am unable to ping the second interface of the LAN (192.168.170.1) I feel like I am missing a rule to pass the traffic from 192.168.9.112 to 192.168.170.1 but tried to do that with no limitters and was not able to.
On top of that, looking at our network syslog I noticed that the box is trying to have connections to external network around midnight almost every minute. I went back to the firewall on port 53. I went back to the firewall and stopped any DNS services as well as NTP protocol thinking that the box would stop doing that. The next morning I discovered that there was still the same attempt to go out of the box to the same IP's. I thought that this might be the update attempt, but the box was already updated manually by me, so there is some connection that is still blocked and I need to stop this service as it is flooding the network and slowing down the ASA we have as a firewall.
So any shared thoughts about how to setup the limiters in my case and make the traffic goes both ways from LAN (192.168.170.0/24) to Internet and reverse, but at the same time not limiting the ping to any local LAN IP's is highly appreciated. I know there are tons of howto's out there, however most of them are reverse to my need - reducing the bandwidth and reducing the latency instead of increasing it.
Here are two of the best howto's I found so far and used as a guide but I was not able to fully accomplish the task:
http://www.squidworks.net/2012/08/pfsense-2-0-limiting-users-upload-and-download-speeds-by-limiting-bandwidth/
https://www.reddit.com/r/PFSENSE/comments/3e67dk/flexible_vs_fixed_limiters_troubleshooting_with/
Any info, any spotted mistakes or needed corrections would make my day
Let me give some more detailed information and what steps I took to make it work. Most likely I have made a simple logic mistake, however I am unable to find it.
The WAN interface of the box is behind our private network and have the following info:
IP: 192.168.9.112
Mask: 24
Gateway: 192.168.9.254
The LAN interface of the box is set to communicate with the newly built rack and has the following details:
IP: 192.168.170.1
Mask: 24
Gateway: N/A
The VSAT speed I am trying to emulate is: Up: 3Mbps/Down 1Mbps with latency ~700ms.
The goal is to allow traffic from any LAN interface to Internet using the bandwidth and latency limiter as well as to allow connections to the new rack LAN from the WAN interface (which is our private network)
Following a few simple tutorials, I created two limiters in Firewall->Traffic Shaper->Limitters. One was called VSATLimitUP having bandwidth of 3Mbps and 350ms latency. The other was called VSATLimitDown having 1Mbps limit with latency 350 (the reason I put 350 instead of 700 was the fact that the ping reply became doubled prior to in/out communication so I split the desired latency in two and set the result to both limiters).
From there everything seemed to be simple, creating a rule in the firewall with the following settings:
Action: Pass
Interface: LAN
Protocol: any
Source: LAN Subnet
Destination: WAN Subnet
Description: LAN to WAN over VSAT
In = VSATLimitUP & Out = VSATLimitDown
Moved the rule above all others and tested to ping. From here I had different results - once I had full LAN speed of <1ms to lan/wan or had a proper latency of 700ms to any IP (lan or wan). It is good if I have a ping to WAN with the latency, however to the LAN I should not be limited and should have <1ms to any device behind 192.168.170.0/24 network.
A similar WAN rule was created:
Action: Pass
Interface: WAN
Protocol: any
Source: WAN Subnet
Destination: LAN Subnet
Description: WAN to LAN over VSAT
In = VSATLimitDown & Out = VSATLimitUP (reversed order of the LAN rule as per most manuals and tutorials)
As a result, I was able to get some traffic being limited, however it did not apply to the limits I set (3Mbps = ~250-300KBps and 1Mbps ~110-120KBps) I get around 50-60K of download speed and is not affected by any change in the limitters.
On the other hand, any attempt to access the LAN network from the WAN (192.168.9.X to 192.168.170.X) is blocked even though on the client remote windows machine I added a route: route add 192.168.170.0 mask 255.255.255.0 192.168.9.112
So far I am able to access the OpnSense over the WAN (I added another rule from any to This Firewall) and ping it, but am unable to ping the second interface of the LAN (192.168.170.1) I feel like I am missing a rule to pass the traffic from 192.168.9.112 to 192.168.170.1 but tried to do that with no limitters and was not able to.
On top of that, looking at our network syslog I noticed that the box is trying to have connections to external network around midnight almost every minute. I went back to the firewall on port 53. I went back to the firewall and stopped any DNS services as well as NTP protocol thinking that the box would stop doing that. The next morning I discovered that there was still the same attempt to go out of the box to the same IP's. I thought that this might be the update attempt, but the box was already updated manually by me, so there is some connection that is still blocked and I need to stop this service as it is flooding the network and slowing down the ASA we have as a firewall.
So any shared thoughts about how to setup the limiters in my case and make the traffic goes both ways from LAN (192.168.170.0/24) to Internet and reverse, but at the same time not limiting the ping to any local LAN IP's is highly appreciated. I know there are tons of howto's out there, however most of them are reverse to my need - reducing the bandwidth and reducing the latency instead of increasing it.
Here are two of the best howto's I found so far and used as a guide but I was not able to fully accomplish the task:
http://www.squidworks.net/2012/08/pfsense-2-0-limiting-users-upload-and-download-speeds-by-limiting-bandwidth/
https://www.reddit.com/r/PFSENSE/comments/3e67dk/flexible_vs_fixed_limiters_troubleshooting_with/
Any info, any spotted mistakes or needed corrections would make my day