IPSec tunnel is up but no traffic (no bytes in or bytes out)

Started by wshamroukh, November 28, 2024, 01:54:04 PM

Previous topic - Next topic
Print
Go Down Pages1
User actions
November 28, 2024, 01:54:04 PM
I am testing opnsese on two VMs on Azure both live in two different virtual network and each has a single public ip address and a since nic as shown the below diagram.

I managed to get the S2S tunnel up but there is no traffic between the two opnsense servers.

I tried to ping a vm (or even the other opnsense) from opnsense1 server, but I get this message:
Code Select
root@OPNsense:~ # ping 10.2.1.4
PING 10.2.1.4 (10.2.1.4): 56 data bytes
ping: sendto: Permission denied
ping: sendto: Permission denied

Same error I get when I try to ssh into the vm on the other side:
Code Select
root@OPNsense:~ # ssh 10.2.1.4
ssh: connect to host 10.2.1.4 port 22: Permission denied

Any help is really appreciated.

November 28, 2024, 02:21:23 PM #1
Did you create firewall rules that allow traffic?
Hardware:
DEC740
November 28, 2024, 03:36:24 PM #2
Quote from: Monviech (Cedrik) on November 28, 2024, 02:21:23 PM
Did you create firewall rules that allow traffic?

If you mean the IPSec firewall rule, yes. I was following this article https://docs.opnsense.org/manual/how-tos/ipsec-s2s.html
Is there anything else to be allowed?
November 28, 2024, 04:24:04 PM #3
Did you even allow remote access in Azure?
November 28, 2024, 04:26:03 PM #4
Quote from: viragomann on November 28, 2024, 04:24:04 PM
Did you even allow remote access in Azure?
Yes - I can ssh into each opnsense machine and I can access the https portal just fine
November 28, 2024, 04:38:53 PM #5
Here is some outputs if you can spot anything wrong:

site1:
Code Select
swanctl --list-sas
no files found matching '/usr/local/etc/strongswan.opnsense.d/*.conf'
con1: #1, ESTABLISHED, IKEv2, 233b879dde1990fc_i* 7f1b33d2a0738eca_r
  local  '4.213.xx.xx' @ 10.1.0.250[4500]
  remote '4.188.xx.xx' @ 4.188.xx.xx[4500]
  AES_CBC-256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
  established 1073s ago, rekeying in 12368s
  con1: #2, reqid 1, INSTALLED, TUNNEL-in-UDP, ESP:AES_GCM_16-256
    installed 1073s ago, rekeying in 2050s, expires in 2527s
    in  cb2eaa17,      0 bytes,     0 packets
    out ca26585e,      0 bytes,     0 packets,   186s ago
    local  10.1.0.0/16
    remote 10.2.0.0/16

site2:
Code Select
root@OPNsense:~ # swanctl --list-sas
no files found matching '/usr/local/etc/strongswan.opnsense.d/*.conf'
con1: #1, ESTABLISHED, IKEv2, 233b879dde1990fc_i 7f1b33d2a0738eca_r*
  local  '4.188.xx.xx' @ 10.2.0.250[4500]
  remote '4.213.xx.xx' @ 4.213.xx.xx[4500]
  AES_CBC-256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
  established 1067s ago, rekeying in 12126s
  con1: #2, reqid 1, INSTALLED, TUNNEL-in-UDP, ESP:AES_GCM_16-256
    installed 1067s ago, rekeying in 2128s, expires in 2533s
    in  ca26585e,      0 bytes,     0 packets
    out cb2eaa17,      0 bytes,     0 packets,   180s ago
    local  10.2.0.0/16
    remote 10.1.0.0/16

site1: ipsec logs:
Code Select
2024-11-28T09:36:33-06:00 Informational charon 13[IKE] <con1|2> sending keep alive to 4.188.xx.xx[4500]
2024-11-28T09:36:01-06:00 Informational charon 12[NET] <con1|2> sending packet: from 10.1.0.250[4500] to 4.188.xx.xx[4500] (224 bytes)
2024-11-28T09:36:01-06:00 Informational charon 12[ENC] <con1|2> generating IKE_AUTH response 1 [ IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr ]
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> CHILD_SA con1{2} established with SPIs c18aeef8_i cdd9b5d2_o and TS 10.1.0.0/16 === 10.2.0.0/16
2024-11-28T09:36:01-06:00 Informational charon 12[CFG] <con1|2> selected proposal: ESP:AES_GCM_16_256/NO_EXT_SEQ
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> maximum IKE_SA lifetime 15691s
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> scheduling rekeying in 14251s
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> IKE_SA con1[2] established between 10.1.0.250[4.213.xx.xx]...4.188.xx.xx[4.188.xx.xx]
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> authentication of '4.213.xx.xx' (myself) with pre-shared key
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> received ESP_TFC_PADDING_NOT_SUPPORTED, not using ESPv3 TFC padding
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> authentication of '4.188.xx.xx' with pre-shared key successful
2024-11-28T09:36:01-06:00 Informational charon 12[CFG] <con1|2> selected peer config 'con1'
2024-11-28T09:36:01-06:00 Informational charon 12[CFG] <2> looking for peer configs matching 10.1.0.250[4.213.xx.xx]...4.188.xx.xx[4.188.xx.xx]
2024-11-28T09:36:01-06:00 Informational charon 12[ENC] <2> parsed IKE_AUTH request 1 [ IDi N(INIT_CONTACT) IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr N(MULT_AUTH) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
2024-11-28T09:36:01-06:00 Informational charon 12[NET] <2> received packet: from 4.188.xx.xx[4500] to 10.1.0.250[4500] (256 bytes)
2024-11-28T09:36:01-06:00 Informational charon 12[NET] <2> sending packet: from 10.1.0.250[500] to 4.188.xx.xx[500] (472 bytes)
2024-11-28T09:36:01-06:00 Informational charon 12[ENC] <2> generating IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(CHDLESS_SUP) N(MULT_AUTH) ]
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <2> remote host is behind NAT
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <2> local host is behind NAT, sending keep alives
2024-11-28T09:36:01-06:00 Informational charon 12[CFG] <2> selected proposal: IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <2> 4.188.xx.xx is initiating an IKE_SA
2024-11-28T09:36:01-06:00 Informational charon 12[ENC] <2> parsed IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
2024-11-28T09:36:01-06:00 Informational charon 12[NET] <2> received packet: from 4.188.xx.xx[500] to 10.1.0.250[500] (464 bytes)
2024-11-28T09:36:00-06:00 Informational charon 12[IKE] <con1|1> received NO_PROPOSAL_CHOSEN notify error
2024-11-28T09:36:00-06:00 Informational charon 12[ENC] <con1|1> parsed IKE_SA_INIT response 0 [ N(NO_PROP) ]
2024-11-28T09:36:00-06:00 Informational charon 12[NET] <con1|1> received packet: from 4.188.xx.xx[500] to 10.1.0.250[500] (36 bytes)
2024-11-28T09:36:00-06:00 Informational charon 13[NET] <con1|1> sending packet: from 10.1.0.250[500] to 4.188.xx.xx[500] (464 bytes)
2024-11-28T09:36:00-06:00 Informational charon 13[ENC] <con1|1> generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
2024-11-28T09:36:00-06:00 Informational charon 13[IKE] <con1|1> initiating IKE_SA con1[1] to 4.188.xx.xx
2024-11-28T09:36:00-06:00 Informational charon 13[KNL] creating acquire job for policy 10.1.0.250/32 === 4.188.xx.xx/32 with reqid {1}
2024-11-28T09:35:46-06:00 Informational charon 13[CFG] installing 'con1'
2024-11-28T09:35:46-06:00 Informational charon 13[CFG] added vici connection: con1
2024-11-28T09:35:46-06:00 Informational charon 13[CFG] loaded IKE shared key with id 'ike-p1-0' for: '4.188.xx.xx'
2024-11-28T09:35:46-06:00 Informational charon 00[JOB] spawning 16 worker threads
2024-11-28T09:35:46-06:00 Informational charon 00[LIB] loaded plugins: charon aes des blowfish rc2 sha2 sha1 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs12 pgp dnskey sshkey pem openssl pkcs8 fips-prf curve25519 xcbc cmac hmac kdf gcm drbg curl attr kernel-pfkey kernel-pfroute resolve socket-default stroke vici updown eap-identity eap-md5 eap-mschapv2 eap-radius eap-tls eap-ttls eap-peap xauth-generic xauth-eap xauth-pam whitelist addrblock counters
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] loaded 0 RADIUS server configurations
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] loading secrets from '/usr/local/etc/ipsec.secrets'
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] loading crls from '/usr/local/etc/ipsec.d/crls'
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] loading attribute certificates from '/usr/local/etc/ipsec.d/acerts'
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] loading ocsp signer certificates from '/usr/local/etc/ipsec.d/ocspcerts'
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] loading aa certificates from '/usr/local/etc/ipsec.d/aacerts'
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] loading ca certificates from '/usr/local/etc/ipsec.d/cacerts'
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] using '/sbin/resolvconf' to install DNS servers
2024-11-28T09:35:46-06:00 Informational charon 00[LIB] providers loaded by OpenSSL: default legacy
2024-11-28T09:35:46-06:00 Informational charon 00[DMN] Starting IKE charon daemon (strongSwan 5.9.14, FreeBSD 14.1-RELEASE-p6, amd64)

site2: ipsec logs
Code Select
2024-11-28T09:36:33-06:00 Informational charon 14[IKE] <con1|2> sending keep alive to 4.213.xx.xx[4500]
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> CHILD_SA con1{2} established with SPIs cdd9b5d2_i c18aeef8_o and TS 10.2.0.0/16 === 10.1.0.0/16
2024-11-28T09:36:01-06:00 Informational charon 12[CFG] <con1|2> selected proposal: ESP:AES_GCM_16_256/NO_EXT_SEQ
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> received ESP_TFC_PADDING_NOT_SUPPORTED, not using ESPv3 TFC padding
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> maximum IKE_SA lifetime 15499s
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> scheduling rekeying in 14059s
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> IKE_SA con1[2] established between 10.2.0.250[4.188.xx.xx]...4.213.xx.xx[4.213.xx.xx]
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> authentication of '4.213.xx.xx' with pre-shared key successful
2024-11-28T09:36:01-06:00 Informational charon 12[ENC] <con1|2> parsed IKE_AUTH response 1 [ IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr ]
2024-11-28T09:36:01-06:00 Informational charon 12[NET] <con1|2> received packet: from 4.213.xx.xx[4500] to 10.2.0.250[4500] (224 bytes)
2024-11-28T09:36:01-06:00 Informational charon 12[NET] <con1|2> sending packet: from 10.2.0.250[4500] to 4.213.xx.xx[4500] (256 bytes)
2024-11-28T09:36:01-06:00 Informational charon 12[ENC] <con1|2> generating IKE_AUTH request 1 [ IDi N(INIT_CONTACT) IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr N(MULT_AUTH) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> establishing CHILD_SA con1{2}
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> authentication of '4.188.xx.xx' (myself) with pre-shared key
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> remote host is behind NAT
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> local host is behind NAT, sending keep alives
2024-11-28T09:36:01-06:00 Informational charon 12[CFG] <con1|2> selected proposal: IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
2024-11-28T09:36:01-06:00 Informational charon 12[ENC] <con1|2> parsed IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(CHDLESS_SUP) N(MULT_AUTH) ]
2024-11-28T09:36:01-06:00 Informational charon 12[NET] <con1|2> received packet: from 4.213.xx.xx[500] to 10.2.0.250[500] (472 bytes)
2024-11-28T09:36:01-06:00 Informational charon 13[NET] <con1|2> sending packet: from 10.2.0.250[500] to 4.213.xx.xx[500] (464 bytes)
2024-11-28T09:36:01-06:00 Informational charon 13[ENC] <con1|2> generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
2024-11-28T09:36:01-06:00 Informational charon 13[IKE] <con1|2> initiating IKE_SA con1[2] to 4.213.xx.xx
2024-11-28T09:36:01-06:00 Informational charon 13[KNL] creating acquire job for policy 10.2.0.250/32 === 4.213.xx.xx/32 with reqid {1}
2024-11-28T09:36:00-06:00 Informational charon 13[CFG] installing 'con1'
2024-11-28T09:36:00-06:00 Informational charon 13[CFG] added vici connection: con1
2024-11-28T09:36:00-06:00 Informational charon 16[CFG] loaded IKE shared key with id 'ike-p1-0' for: '4.213.xx.xx'
2024-11-28T09:36:00-06:00 Informational charon 16[NET] <1> sending packet: from 10.2.0.250[500] to 4.213.xx.xx[500] (36 bytes)
2024-11-28T09:36:00-06:00 Informational charon 16[ENC] <1> generating IKE_SA_INIT response 0 [ N(NO_PROP) ]
2024-11-28T09:36:00-06:00 Informational charon 16[IKE] <1> no IKE config found for 10.2.0.250...4.213.xx.xx, sending NO_PROPOSAL_CHOSEN
2024-11-28T09:36:00-06:00 Informational charon 16[ENC] <1> parsed IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
2024-11-28T09:36:00-06:00 Informational charon 16[NET] <1> received packet: from 4.213.xx.xx[500] to 10.2.0.250[500] (464 bytes)
2024-11-28T09:35:59-06:00 Informational charon 00[JOB] spawning 16 worker threads
2024-11-28T09:35:59-06:00 Informational charon 00[LIB] loaded plugins: charon aes des blowfish rc2 sha2 sha1 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs12 pgp dnskey sshkey pem openssl pkcs8 fips-prf curve25519 xcbc cmac hmac kdf gcm drbg curl attr kernel-pfkey kernel-pfroute resolve socket-default stroke vici updown eap-identity eap-md5 eap-mschapv2 eap-radius eap-tls eap-ttls eap-peap xauth-generic xauth-eap xauth-pam whitelist addrblock counters
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] loaded 0 RADIUS server configurations
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] loading secrets from '/usr/local/etc/ipsec.secrets'
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] loading crls from '/usr/local/etc/ipsec.d/crls'
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] loading attribute certificates from '/usr/local/etc/ipsec.d/acerts'
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] loading ocsp signer certificates from '/usr/local/etc/ipsec.d/ocspcerts'
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] loading aa certificates from '/usr/local/etc/ipsec.d/aacerts'
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] loading ca certificates from '/usr/local/etc/ipsec.d/cacerts'
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] using '/sbin/resolvconf' to install DNS servers
2024-11-28T09:35:59-06:00 Informational charon 00[LIB] providers loaded by OpenSSL: default legacy
2024-11-28T09:35:59-06:00 Informational charon 00[DMN] Starting IKE charon daemon (strongSwan 5.9.14, FreeBSD 14.1-RELEASE-p6, amd64)
November 28, 2024, 04:56:51 PM #6
Seems to be a p1 issue.

You have to state the outside public IP as "My identifier" on both sites.
November 28, 2024, 05:22:09 PM #7
Quote from: viragomann on November 28, 2024, 04:56:51 PM
Seems to be a p1 issue.

You have to state the outside public IP as "My identifier" on both sites.

They are there already

November 28, 2024, 05:27:09 PM #8
Its not a Phase 1 issue if Phase 1 is established and there is a Phase 2.

I expect there are no rules that allow traffic through the tunnel.

Just create a rule in Floating allowing any any on both Firewalls to troubleshoot, if this is not production yet.
Hardware:
DEC740
November 28, 2024, 05:33:16 PM #9
Quote from: Monviech (Cedrik) on November 28, 2024, 05:27:09 PM
Its not a Phase 1 issue if Phase 1 is established and there is a Phase 2.

I expect there are no rules that allow traffic through the tunnel.

Just create a rule in Floating allowing any any on both Firewalls to troubleshoot, if this is not production yet.

Just a floating rule but still the connection is up but there is no traffic. I can't ping/ssh either anything on the other side.
November 28, 2024, 05:35:07 PM #10
Did you install a policy on both sides? (Install policy checked)
Hardware:
DEC740
November 28, 2024, 05:40:48 PM #11
Quote from: Monviech (Cedrik) on November 28, 2024, 05:35:07 PM
Did you install a policy on both sides? (Install policy checked)
yes
November 28, 2024, 07:17:48 PM #12
Then I don't know sorry. Must be either an issue with the policy not matching the traffic you send into the tunnel, or a routing issue outside the tunnel, e.g. Default Gateway.

Use packet captures to troubleshoot on WAN, LAN and ipsec (enc0) interface.
Hardware:
DEC740
December 01, 2024, 09:56:46 AM #13
I finally managed to get it to work just fine. I resorted to a fresh installation of opnsense with fresh config, and then I was able to get the IPsec tunnel up. Thank you all for your help
December 01, 2024, 10:17:55 AM #14
Good that you got it working in the end, maybe it had some sticky configuration error the first time you tried.
Hardware:
DEC740
Print
Go Up Pages1
User actions