[MultiWan]Clients notice poor and inconsistent connectivity when using MultiWan

Started by akshaykalia, June 15, 2024, 07:46:54 AM

Previous topic - Next topic
Problem Statement:
  - MultiWan enabled using policy-based routing via firewall rule
  - All Lan traffic other than DNS directed to gateway group
  - Clients notice poor and inconsistent connectivity 
  - DNS resolution works 100% of time
  - However, Apps and Browsers report timeout while loading the data
- To make things work I have to keep policy-based routing disabled or keep the 3 GWs in separate tier
  - The setup worked on version 24.1.6


System details:
  - Versions: OPNsense 24.1.8-amd64 FreeBSD 13.2-RELEASE-p11 OpenSSL 3.0.13
  - CPU type: AMD Athlon 200GE with Radeon Vega Graphics (2 cores, 4 threads)
  - Memory: 16 GB
  - Storage: 256 GB NVME

Interface Details:
  - Lan:  re0 Realtek PCI-e Nic 2.5 Gbps
          - Static IPv4
          - IPv6 Disabled
          - Connected to Wi-Fi router

  - opt1: igb0 Dula port intel PCI-e Nic 1 Gbps
          - DHCP IPv4
          - IPv6 Disabled 
          - Connected to DSL Router (40 Mbps)
  - opt2: igb1 Dula port intel PCI-e Nic 1 Gbps
          - DHCP IPv4
          - IPv6 Disabled
          - Connected to 4G router (30 Mbps)
  - wan:  re1 Realtek Onboard Nic 1 Gbps
          - DHCP IPv4
          - IPv6 Disabled
          - Connected to Fiber Router (600 Mbps)

Gateway Details:
  - WAN_GW (re1):
      - Priority: 200
      - Monitoring: Enabled
      - Weight: 5
      - Probe Interval: 5
  - DSL_DHCP (igb0):
      - Priority: 250
      - Monitoring: Enabled
      - Weight: 3
      - Probe Interval: 5
  - 4g_DHCP (igb1):
      - Priority: 254
      - Monitoring: Enabled
      - Weight: 1
      - Probe Interval: 5
  - LAN_6TO4(6 to 4 tunnel):
      - Priority: 255
      - Monitoring: Enabled
      - Weight: 1
      - Probe Interval: 5

Firewall Rules:
  - Attached as a screenshot

I am going something worng?