I am testing opnsese on two VMs on Azure both live in two different virtual network and each has a single public ip address and a since nic as shown the below diagram.
I managed to get the S2S tunnel up but there is no traffic between the two opnsense servers.
(https://waynestor.blob.core.windows.net/share/vpnstatus.png)
I tried to ping a vm (or even the other opnsense) from opnsense1 server, but I get this message:
root@OPNsense:~ # ping 10.2.1.4
PING 10.2.1.4 (10.2.1.4): 56 data bytes
ping: sendto: Permission denied
ping: sendto: Permission denied
Same error I get when I try to ssh into the vm on the other side:
root@OPNsense:~ # ssh 10.2.1.4
ssh: connect to host 10.2.1.4 port 22: Permission denied
Any help is really appreciated.
(https://waynestor.blob.core.windows.net/share/opnsense.png)
Did you create firewall rules that allow traffic?
Quote from: Monviech (Cedrik) on November 28, 2024, 02:21:23 PM
Did you create firewall rules that allow traffic?
If you mean the IPSec firewall rule, yes. I was following this article https://docs.opnsense.org/manual/how-tos/ipsec-s2s.html
Is there anything else to be allowed?
Did you even allow remote access in Azure?
Quote from: viragomann on November 28, 2024, 04:24:04 PM
Did you even allow remote access in Azure?
Yes - I can ssh into each opnsense machine and I can access the https portal just fine
Here is some outputs if you can spot anything wrong:
site1:
swanctl --list-sas
no files found matching '/usr/local/etc/strongswan.opnsense.d/*.conf'
con1: #1, ESTABLISHED, IKEv2, 233b879dde1990fc_i* 7f1b33d2a0738eca_r
local '4.213.xx.xx' @ 10.1.0.250[4500]
remote '4.188.xx.xx' @ 4.188.xx.xx[4500]
AES_CBC-256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
established 1073s ago, rekeying in 12368s
con1: #2, reqid 1, INSTALLED, TUNNEL-in-UDP, ESP:AES_GCM_16-256
installed 1073s ago, rekeying in 2050s, expires in 2527s
in cb2eaa17, 0 bytes, 0 packets
out ca26585e, 0 bytes, 0 packets, 186s ago
local 10.1.0.0/16
remote 10.2.0.0/16
site2:
root@OPNsense:~ # swanctl --list-sas
no files found matching '/usr/local/etc/strongswan.opnsense.d/*.conf'
con1: #1, ESTABLISHED, IKEv2, 233b879dde1990fc_i 7f1b33d2a0738eca_r*
local '4.188.xx.xx' @ 10.2.0.250[4500]
remote '4.213.xx.xx' @ 4.213.xx.xx[4500]
AES_CBC-256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
established 1067s ago, rekeying in 12126s
con1: #2, reqid 1, INSTALLED, TUNNEL-in-UDP, ESP:AES_GCM_16-256
installed 1067s ago, rekeying in 2128s, expires in 2533s
in ca26585e, 0 bytes, 0 packets
out cb2eaa17, 0 bytes, 0 packets, 180s ago
local 10.2.0.0/16
remote 10.1.0.0/16
site1: ipsec logs:
2024-11-28T09:36:33-06:00 Informational charon 13[IKE] <con1|2> sending keep alive to 4.188.xx.xx[4500]
2024-11-28T09:36:01-06:00 Informational charon 12[NET] <con1|2> sending packet: from 10.1.0.250[4500] to 4.188.xx.xx[4500] (224 bytes)
2024-11-28T09:36:01-06:00 Informational charon 12[ENC] <con1|2> generating IKE_AUTH response 1 [ IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr ]
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> CHILD_SA con1{2} established with SPIs c18aeef8_i cdd9b5d2_o and TS 10.1.0.0/16 === 10.2.0.0/16
2024-11-28T09:36:01-06:00 Informational charon 12[CFG] <con1|2> selected proposal: ESP:AES_GCM_16_256/NO_EXT_SEQ
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> maximum IKE_SA lifetime 15691s
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> scheduling rekeying in 14251s
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> IKE_SA con1[2] established between 10.1.0.250[4.213.xx.xx]...4.188.xx.xx[4.188.xx.xx]
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> authentication of '4.213.xx.xx' (myself) with pre-shared key
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> received ESP_TFC_PADDING_NOT_SUPPORTED, not using ESPv3 TFC padding
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> authentication of '4.188.xx.xx' with pre-shared key successful
2024-11-28T09:36:01-06:00 Informational charon 12[CFG] <con1|2> selected peer config 'con1'
2024-11-28T09:36:01-06:00 Informational charon 12[CFG] <2> looking for peer configs matching 10.1.0.250[4.213.xx.xx]...4.188.xx.xx[4.188.xx.xx]
2024-11-28T09:36:01-06:00 Informational charon 12[ENC] <2> parsed IKE_AUTH request 1 [ IDi N(INIT_CONTACT) IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr N(MULT_AUTH) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
2024-11-28T09:36:01-06:00 Informational charon 12[NET] <2> received packet: from 4.188.xx.xx[4500] to 10.1.0.250[4500] (256 bytes)
2024-11-28T09:36:01-06:00 Informational charon 12[NET] <2> sending packet: from 10.1.0.250[500] to 4.188.xx.xx[500] (472 bytes)
2024-11-28T09:36:01-06:00 Informational charon 12[ENC] <2> generating IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(CHDLESS_SUP) N(MULT_AUTH) ]
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <2> remote host is behind NAT
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <2> local host is behind NAT, sending keep alives
2024-11-28T09:36:01-06:00 Informational charon 12[CFG] <2> selected proposal: IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <2> 4.188.xx.xx is initiating an IKE_SA
2024-11-28T09:36:01-06:00 Informational charon 12[ENC] <2> parsed IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
2024-11-28T09:36:01-06:00 Informational charon 12[NET] <2> received packet: from 4.188.xx.xx[500] to 10.1.0.250[500] (464 bytes)
2024-11-28T09:36:00-06:00 Informational charon 12[IKE] <con1|1> received NO_PROPOSAL_CHOSEN notify error
2024-11-28T09:36:00-06:00 Informational charon 12[ENC] <con1|1> parsed IKE_SA_INIT response 0 [ N(NO_PROP) ]
2024-11-28T09:36:00-06:00 Informational charon 12[NET] <con1|1> received packet: from 4.188.xx.xx[500] to 10.1.0.250[500] (36 bytes)
2024-11-28T09:36:00-06:00 Informational charon 13[NET] <con1|1> sending packet: from 10.1.0.250[500] to 4.188.xx.xx[500] (464 bytes)
2024-11-28T09:36:00-06:00 Informational charon 13[ENC] <con1|1> generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
2024-11-28T09:36:00-06:00 Informational charon 13[IKE] <con1|1> initiating IKE_SA con1[1] to 4.188.xx.xx
2024-11-28T09:36:00-06:00 Informational charon 13[KNL] creating acquire job for policy 10.1.0.250/32 === 4.188.xx.xx/32 with reqid {1}
2024-11-28T09:35:46-06:00 Informational charon 13[CFG] installing 'con1'
2024-11-28T09:35:46-06:00 Informational charon 13[CFG] added vici connection: con1
2024-11-28T09:35:46-06:00 Informational charon 13[CFG] loaded IKE shared key with id 'ike-p1-0' for: '4.188.xx.xx'
2024-11-28T09:35:46-06:00 Informational charon 00[JOB] spawning 16 worker threads
2024-11-28T09:35:46-06:00 Informational charon 00[LIB] loaded plugins: charon aes des blowfish rc2 sha2 sha1 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs12 pgp dnskey sshkey pem openssl pkcs8 fips-prf curve25519 xcbc cmac hmac kdf gcm drbg curl attr kernel-pfkey kernel-pfroute resolve socket-default stroke vici updown eap-identity eap-md5 eap-mschapv2 eap-radius eap-tls eap-ttls eap-peap xauth-generic xauth-eap xauth-pam whitelist addrblock counters
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] loaded 0 RADIUS server configurations
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] loading secrets from '/usr/local/etc/ipsec.secrets'
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] loading crls from '/usr/local/etc/ipsec.d/crls'
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] loading attribute certificates from '/usr/local/etc/ipsec.d/acerts'
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] loading ocsp signer certificates from '/usr/local/etc/ipsec.d/ocspcerts'
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] loading aa certificates from '/usr/local/etc/ipsec.d/aacerts'
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] loading ca certificates from '/usr/local/etc/ipsec.d/cacerts'
2024-11-28T09:35:46-06:00 Informational charon 00[CFG] using '/sbin/resolvconf' to install DNS servers
2024-11-28T09:35:46-06:00 Informational charon 00[LIB] providers loaded by OpenSSL: default legacy
2024-11-28T09:35:46-06:00 Informational charon 00[DMN] Starting IKE charon daemon (strongSwan 5.9.14, FreeBSD 14.1-RELEASE-p6, amd64)
site2: ipsec logs
2024-11-28T09:36:33-06:00 Informational charon 14[IKE] <con1|2> sending keep alive to 4.213.xx.xx[4500]
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> CHILD_SA con1{2} established with SPIs cdd9b5d2_i c18aeef8_o and TS 10.2.0.0/16 === 10.1.0.0/16
2024-11-28T09:36:01-06:00 Informational charon 12[CFG] <con1|2> selected proposal: ESP:AES_GCM_16_256/NO_EXT_SEQ
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> received ESP_TFC_PADDING_NOT_SUPPORTED, not using ESPv3 TFC padding
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> maximum IKE_SA lifetime 15499s
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> scheduling rekeying in 14059s
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> IKE_SA con1[2] established between 10.2.0.250[4.188.xx.xx]...4.213.xx.xx[4.213.xx.xx]
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> authentication of '4.213.xx.xx' with pre-shared key successful
2024-11-28T09:36:01-06:00 Informational charon 12[ENC] <con1|2> parsed IKE_AUTH response 1 [ IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr ]
2024-11-28T09:36:01-06:00 Informational charon 12[NET] <con1|2> received packet: from 4.213.xx.xx[4500] to 10.2.0.250[4500] (224 bytes)
2024-11-28T09:36:01-06:00 Informational charon 12[NET] <con1|2> sending packet: from 10.2.0.250[4500] to 4.213.xx.xx[4500] (256 bytes)
2024-11-28T09:36:01-06:00 Informational charon 12[ENC] <con1|2> generating IKE_AUTH request 1 [ IDi N(INIT_CONTACT) IDr AUTH N(ESP_TFC_PAD_N) SA TSi TSr N(MULT_AUTH) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> establishing CHILD_SA con1{2}
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> authentication of '4.188.xx.xx' (myself) with pre-shared key
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> remote host is behind NAT
2024-11-28T09:36:01-06:00 Informational charon 12[IKE] <con1|2> local host is behind NAT, sending keep alives
2024-11-28T09:36:01-06:00 Informational charon 12[CFG] <con1|2> selected proposal: IKE:AES_CBC_256/HMAC_SHA2_256_128/PRF_HMAC_SHA2_256/MODP_2048
2024-11-28T09:36:01-06:00 Informational charon 12[ENC] <con1|2> parsed IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(CHDLESS_SUP) N(MULT_AUTH) ]
2024-11-28T09:36:01-06:00 Informational charon 12[NET] <con1|2> received packet: from 4.213.xx.xx[500] to 10.2.0.250[500] (472 bytes)
2024-11-28T09:36:01-06:00 Informational charon 13[NET] <con1|2> sending packet: from 10.2.0.250[500] to 4.213.xx.xx[500] (464 bytes)
2024-11-28T09:36:01-06:00 Informational charon 13[ENC] <con1|2> generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
2024-11-28T09:36:01-06:00 Informational charon 13[IKE] <con1|2> initiating IKE_SA con1[2] to 4.213.xx.xx
2024-11-28T09:36:01-06:00 Informational charon 13[KNL] creating acquire job for policy 10.2.0.250/32 === 4.213.xx.xx/32 with reqid {1}
2024-11-28T09:36:00-06:00 Informational charon 13[CFG] installing 'con1'
2024-11-28T09:36:00-06:00 Informational charon 13[CFG] added vici connection: con1
2024-11-28T09:36:00-06:00 Informational charon 16[CFG] loaded IKE shared key with id 'ike-p1-0' for: '4.213.xx.xx'
2024-11-28T09:36:00-06:00 Informational charon 16[NET] <1> sending packet: from 10.2.0.250[500] to 4.213.xx.xx[500] (36 bytes)
2024-11-28T09:36:00-06:00 Informational charon 16[ENC] <1> generating IKE_SA_INIT response 0 [ N(NO_PROP) ]
2024-11-28T09:36:00-06:00 Informational charon 16[IKE] <1> no IKE config found for 10.2.0.250...4.213.xx.xx, sending NO_PROPOSAL_CHOSEN
2024-11-28T09:36:00-06:00 Informational charon 16[ENC] <1> parsed IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
2024-11-28T09:36:00-06:00 Informational charon 16[NET] <1> received packet: from 4.213.xx.xx[500] to 10.2.0.250[500] (464 bytes)
2024-11-28T09:35:59-06:00 Informational charon 00[JOB] spawning 16 worker threads
2024-11-28T09:35:59-06:00 Informational charon 00[LIB] loaded plugins: charon aes des blowfish rc2 sha2 sha1 md4 md5 random nonce x509 revocation constraints pubkey pkcs1 pkcs7 pkcs12 pgp dnskey sshkey pem openssl pkcs8 fips-prf curve25519 xcbc cmac hmac kdf gcm drbg curl attr kernel-pfkey kernel-pfroute resolve socket-default stroke vici updown eap-identity eap-md5 eap-mschapv2 eap-radius eap-tls eap-ttls eap-peap xauth-generic xauth-eap xauth-pam whitelist addrblock counters
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] loaded 0 RADIUS server configurations
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] loading secrets from '/usr/local/etc/ipsec.secrets'
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] loading crls from '/usr/local/etc/ipsec.d/crls'
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] loading attribute certificates from '/usr/local/etc/ipsec.d/acerts'
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] loading ocsp signer certificates from '/usr/local/etc/ipsec.d/ocspcerts'
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] loading aa certificates from '/usr/local/etc/ipsec.d/aacerts'
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] loading ca certificates from '/usr/local/etc/ipsec.d/cacerts'
2024-11-28T09:35:59-06:00 Informational charon 00[CFG] using '/sbin/resolvconf' to install DNS servers
2024-11-28T09:35:59-06:00 Informational charon 00[LIB] providers loaded by OpenSSL: default legacy
2024-11-28T09:35:59-06:00 Informational charon 00[DMN] Starting IKE charon daemon (strongSwan 5.9.14, FreeBSD 14.1-RELEASE-p6, amd64)
Seems to be a p1 issue.
You have to state the outside public IP as "My identifier" on both sites.
Quote from: viragomann on November 28, 2024, 04:56:51 PM
Seems to be a p1 issue.
You have to state the outside public IP as "My identifier" on both sites.
They are there already
(https://waynestor.blob.core.windows.net/share/p1.png)
Its not a Phase 1 issue if Phase 1 is established and there is a Phase 2.
I expect there are no rules that allow traffic through the tunnel.
Just create a rule in Floating allowing any any on both Firewalls to troubleshoot, if this is not production yet.
Quote from: Monviech (Cedrik) on November 28, 2024, 05:27:09 PM
Its not a Phase 1 issue if Phase 1 is established and there is a Phase 2.
I expect there are no rules that allow traffic through the tunnel.
Just create a rule in Floating allowing any any on both Firewalls to troubleshoot, if this is not production yet.
Just a floating rule but still the connection is up but there is no traffic. I can't ping/ssh either anything on the other side.
Did you install a policy on both sides? (Install policy checked)
Quote from: Monviech (Cedrik) on November 28, 2024, 05:35:07 PM
Did you install a policy on both sides? (Install policy checked)
yes
Then I don't know sorry. Must be either an issue with the policy not matching the traffic you send into the tunnel, or a routing issue outside the tunnel, e.g. Default Gateway.
Use packet captures to troubleshoot on WAN, LAN and ipsec (enc0) interface.
I finally managed to get it to work just fine. I resorted to a fresh installation of opnsense with fresh config, and then I was able to get the IPsec tunnel up. Thank you all for your help
Good that you got it working in the end, maybe it had some sticky configuration error the first time you tried.