Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Topics - glgontijo

#1
Hi everyone,

I'm reaching out because I've encountered a strange connectivity issue with my current VPN architecture. Here is the setup:

Server: OPNsense 25.7.11

Clients: 3x OPNsense nodes (v26.1.2), though the issue persisted even when they were on 25.7.

VPN: WireGuard for Site-to-Site connectivity.

Routing/HA: BGP + BFD for High Availability (2x WAN links on the server side).

L2 Extension: VxLAN over WireGuard + Bridge with local VLANs to maintain a single L2 broadcast domain across sites.

The Problem:
Some hosts on the Headquarters (HQ) LAN cannot see the clients, while others can. This behavior is reciprocal (affects both directions).

If I disable the packet filter (pf), everything responds normally.

When pf is active, only specific hosts can communicate.

Interestingly, broadcast traffic works perfectly across the entire network.

What I've tried so far:

Rules: I have "Allow All" (any-any) rules on the Bridge, VxLAN, and VLAN interfaces. I'm also using Floating Rules covering all member interfaces.

Tunables: I've toggled net.link.bridge.pfil_bridge and net.link.bridge.pfil_member in various combinations (0/1, 1/0, 0/0) with no success.

MSS Clamping: Enabled and disabled; no change.

BGP/Routing: I tried disabling the VPN on one of the WAN links to force traffic through a single path (ruling out BGP routing ARP to the wrong interface), but the behavior remained the same.

Constraints:
I must stick with an L2 architecture because I require L2 broadcast propagation for CCTV, VoIP, and WiFi Controllers. I considered switching to OpenVPN in TAP mode (which would also simplify the HA setup), but I prefer WireGuard for its superior performance with VoIP traffic.

Other hosts behind the client nodes are working fine and show no errors in broadcast or standard traffic. It seems to be a selective filtering issue despite the "allow all" rules.

Does anyone have insight into why the firewall might be dropping this traffic only for specific hosts within an L2 bridge?

Any help would be greatly appreciated!
#2
Caros,

Tenho um servidor VPN em OPNsense e clientes também em OPN.

A estrutura é a seguinte:

VPN Wireguard conecta os clientes ao servidor
BGP+BFD garante a alta disponibilidade (02 links de WAN no servidor WG)
VxLAN + Bridge com as VLANs de cada lado. Garantindo o funcionamento da rede como uma só VLAN.

Todos os 3 clientes configurados de forma igual. Com a mesma versão do OPN 26.1.2
Somente o servidor que está na 25.7.11 (Mas o problema vem desde quando todos clientes também estavam com a 25.7)

Alguns hosts da rede local na sede, não enxergam o cliente, mas outros sim.
Se desativo o pf, ele responde normalmente, se reativo, somente alguns hosts da sede respondem.
O comportamento é mútuo, também atinge o cliente.
Broadcast trafega sem problemas na rede.

As regras são all-all any-any para a bridge, vxlan e vlan (floating com todas as interfaces que compõem a br).
Já ativei e desativei MSS Clamping.
Já ativei e desativei net.link.pfil.bridge e member (0,1 / 1,0 / 0,0) e nada.

Outros hosts atrás do cliente respondem sem erros e não tem problemas em tráfego de broadcast.

Preciso entender o que está acontecendo.

Não posso trabalhar com outra arquitetura, pois preciso de propagação de broadcast em L2 para CFTV, VOIP e controladora de Wifi.

Pensei em usar OpenVPN TAP para obter L2 (isso também eliminaria a necessidade do BGP, já que o OVPN faz isso diretamente), mas o WG tem mais qualidade para o VOIP.

AH!
Também já tentei desativar a VPN de um dos links, e forçar tudo em um só (pra ver se não era BGP encaminhando ARP em interface errada). Não houve alteração no comportamento.

Agradeço quaisquer ajudas.
#3
Hello guys,

I need the TAP VPN (it's... it has to be TAP) not create the default gateway (route 0.0.0.0).
I've tried "route-nopull", "route-noexec" (server, client via CSC, export file).

Let's specify what I need.
  • Customers authenticate with user and password (freeradius, so far it is perfect)
  • Only customers with CSC configuration can authenticate on the VPN
  • Customers receive IP from LAN via CSC
  • Only LAN routes must be created on the customer. But it creates the default route and with metric below the existing one. HERE IS MY PROBLEM!

Follow the files (I edited to remove sensitive data)

Server: # cat /var/etc/openvpn/*.conf | sed -n '1,200p'
dev ovpns1
ping-timer-rem
topology subnet
dh /usr/local/etc/inc/plugins.inc.d/openvpn/dh.rfc7919
verify-client-cert require
remote-cert-tls client
server-bridge
username-as-common-name
client-config-dir /var/etc/openvpn-csc/1
auth-user-pass-verify "/usr/local/opnsense/scripts/openvpn/ovpn_event.py --defer '29533187-c920-428c-b82f-6fd2c670ad14'" via-env
learn-address "/usr/local/opnsense/scripts/openvpn/ovpn_event.py '1'"
client-disconnect "/usr/local/opnsense/scripts/openvpn/ovpn_event.py '29533187-c920-428c-b82f-6fd2c670ad14'"
tls-verify "/usr/local/opnsense/scripts/openvpn/ovpn_event.py '29533187-c920-428c-b82f-6fd2c670ad14'"
Multihome
push "explicit-exit-notify"
push "route 172.16.0.0 255.255.0.0"
route 172.16.0.0 255.255.0.0
persist-tun
persist-key
keepalive 10 60
dev-type tap
dev-node /dev/tap1
script-security 3
writepid /var/run/ovpn-instance-29533187-c920-428c-b82f-6fd2c670ad14.pid
daemon openvpn_server1
management /var/etc/openvpn/instance-29533187-c920-428c-b82f-6fd2c670ad14.sock unix
proto udp4
verb 7
disable-dco
up /usr/local/etc/inc/plugins.inc.d/openvpn/ovpn-linkup
down /usr/local/etc/inc/plugins.inc.d/openvpn/ovpn-linkdown
port 1194
date-ciphers AES-256-GCM:AES-128-GCM:CHACHA20-POLY1305
data-ciphers-fallback AES-256-GCM
block-ipv6
float
explicit-exit-notify
fast-io
<tls-crypt>
#
#2048 bit OpenVPN static key
#
-----BEGIN OpenVPN Static key V1-----


Client CSC: # cat /var/etc/openvpn-csc/1/guilherme.gontijo@uftm.edu.br | sed -n '1,200p'
ifconfig-push 172.16.7.2 255.255.0.0



Exported File:
dev tap
persist-tun
persist-key
data-ciphers-fallback AES-256-GCM
client
resolv-retry infinite
remote 186.248.203.214 1194 udp4
remote 200.131.62.250 1194 udp4
lport 0
verify-x509-name "C=BR, ST=MG, L=Uberaba, O=UFTM, OU=PROTIC, CN=vpnserver-certificate.uftm.br" subject
remote-cert-tls server
auth-user-pass
auth-nocache
route-noexec
route-nopull
<ca>
-----BEGIN CERTIFICATE-----


NOTE: Chat GPT and Gemini couldn't help me in this... kkkk 

Sorry for bad translation!
#4
Olá pessoal,

Preciso que a VPN TAP (é... tem que ser TAP) não crie o gateway default (rota 0.0.0.0).
Já tentei "route-nopull", "route-noexec" (servidor, cliente via CSC, arquivo de exportação).

Vamos especificar o que eu preciso.
  • Clientes autenticam com usuário e senha (freeradius, até aqui está perfeito)
  • Somente clientes com configuração do CSC podem autenticar na VPN
  • Clientes recebem IP da LAN via CSC
  • Somente as rotas da LAN devem ser criadas no cliente. Mas ele cria a rota default e com métrica abaixo da já existente.  AQUI ESTÁ MEU PROBLEMA!

Seguem os arquivos (editei para remover dados sensíveis)

Servidor: # cat /var/etc/openvpn/*.conf | sed -n '1,200p'
dev ovpns1
ping-timer-rem
topology subnet
dh /usr/local/etc/inc/plugins.inc.d/openvpn/dh.rfc7919
verify-client-cert require
remote-cert-tls client
server-bridge
username-as-common-name
client-config-dir /var/etc/openvpn-csc/1
auth-user-pass-verify "/usr/local/opnsense/scripts/openvpn/ovpn_event.py --defer '29533187-c920-428c-b82f-6fd2c670ad14'" via-env
learn-address "/usr/local/opnsense/scripts/openvpn/ovpn_event.py '1'"
client-disconnect "/usr/local/opnsense/scripts/openvpn/ovpn_event.py '29533187-c920-428c-b82f-6fd2c670ad14'"
tls-verify "/usr/local/opnsense/scripts/openvpn/ovpn_event.py '29533187-c920-428c-b82f-6fd2c670ad14'"
multihome
push "explicit-exit-notify"
push "route 172.16.0.0 255.255.0.0"
route 172.16.0.0 255.255.0.0
persist-tun
persist-key
keepalive 10 60
dev-type tap
dev-node /dev/tap1
script-security 3
writepid /var/run/ovpn-instance-29533187-c920-428c-b82f-6fd2c670ad14.pid
daemon openvpn_server1
management /var/etc/openvpn/instance-29533187-c920-428c-b82f-6fd2c670ad14.sock unix
proto udp4
verb 7
disable-dco
up /usr/local/etc/inc/plugins.inc.d/openvpn/ovpn-linkup
down /usr/local/etc/inc/plugins.inc.d/openvpn/ovpn-linkdown
port 1194
data-ciphers AES-256-GCM:AES-128-GCM:CHACHA20-POLY1305
data-ciphers-fallback AES-256-GCM
block-ipv6
float
explicit-exit-notify
fast-io
<tls-crypt>
#
# 2048 bit OpenVPN static key
#
-----BEGIN OpenVPN Static key V1-----


Cliente CSC:  # cat /var/etc/openvpn-csc/1/guilherme.gontijo@uftm.edu.br  | sed -n '1,200p'
ifconfig-push 172.16.7.2 255.255.0.0


Arquivo Exportado:
dev tap
persist-tun
persist-key
data-ciphers-fallback AES-256-GCM
client
resolv-retry infinite
remote 186.248.203.214 1194 udp4
remote 200.131.62.250 1194 udp4
lport 0
verify-x509-name "C=BR, ST=MG, L=Uberaba, O=UFTM, OU=PROTIC, CN=vpnserver-certificate.uftm.br" subject
remote-cert-tls server
auth-user-pass
auth-nocache
route-noexec
route-nopull
<ca>
-----BEGIN CERTIFICATE-----


OBS.: Chat GPT e Gemini não conseguiram me ajudar nessa... kkkk