IPsec Multiple Phase 2 Invalid Payload

[somnuk_s]

Currently, I'm simulate IPsec PSK Site-to-Site connection between SmallWall (1.8.3) and OPNsense (OPNsense 18.7.9-amd64) and found a strange behavior when configure multiple Phase 2 on OPNsense. If I set the mode to main on SmallWall definition, the connection will not get connected and on SmallWall machine will report "racoon: [10.3.32.59] ERROR: invalid ID payload.".

----SmallWall Log----
Dec 21 12:19:01   racoon: ERROR: phase1 negotiation failed due to time up. ca3087efc9202642:b154c91ab13d2b21
Dec 21 12:18:59   last message repeated 4 times
Dec 21 12:18:11   racoon: [10.3.32.59] ERROR: invalid ID payload.
Dec 21 12:18:11   racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02
Dec 21 12:18:11   racoon: INFO: received Vendor ID: RFC 3947
Dec 21 12:18:11   racoon: INFO: received broken Microsoft ID: FRAGMENTATION
Dec 21 12:18:11   racoon: INFO: received Vendor ID: DPD
Dec 21 12:18:11   racoon: INFO: received Vendor ID: draft-ietf-ipsra-isakmp-xauth-06.txt
Dec 21 12:18:11   racoon: INFO: begin Identity Protection mode.
Dec 21 12:18:11   racoon: INFO: respond new phase 1 negotiation: 10.3.32.60[500]<=>10.3.32.59[500]

---------OPNsense Log-----------
Dec 21 12:19:41   OPNsense charon: 03[NET] <con1-000|3> sending packet: from 10.3.32.59[500] to 10.3.32.60[500] (108 bytes)
Dec 21 12:19:41   OPNsense charon: 03[IKE] <con1-000|3> sending retransmit 5 of request message ID 0, seq 3
Dec 21 12:18:58   OPNsense charon: 03[NET] <con1-000|3> sending packet: from 10.3.32.59[500] to 10.3.32.60[500] (108 bytes)
Dec 21 12:18:58   OPNsense charon: 03[IKE] <con1-000|3> sending retransmit 4 of request message ID 0, seq 3
Dec 21 12:18:51   OPNsense charon: 03[IKE] <con1-000|3> received retransmit of response with ID 0, but next request already sent
Dec 21 12:18:51   OPNsense charon: 03[NET] <con1-000|3> received packet: from 10.3.32.60[500] to 10.3.32.59[500] (180 bytes)
Dec 21 12:18:41   OPNsense charon: 03[IKE] <con1-000|3> received retransmit of response with ID 0, but next request already sent
Dec 21 12:18:41   OPNsense charon: 03[NET] <con1-000|3> received packet: from 10.3.32.60[500] to 10.3.32.59[500] (180 bytes)
Dec 21 12:18:35   OPNsense charon: 03[NET] <con1-000|3> sending packet: from 10.3.32.59[500] to 10.3.32.60[500] (108 bytes)
Dec 21 12:18:35   OPNsense charon: 03[IKE] <con1-000|3> sending retransmit 3 of request message ID 0, seq 3
Dec 21 12:18:31   OPNsense charon: 03[IKE] <con1-000|3> received retransmit of response with ID 0, but next request already sent
Dec 21 12:18:31   OPNsense charon: 03[NET] <con1-000|3> received packet: from 10.3.32.60[500] to 10.3.32.59[500] (180 bytes)
Dec 21 12:18:22   OPNsense charon: 03[NET] <con1-000|3> sending packet: from 10.3.32.59[500] to 10.3.32.60[500] (108 bytes)
Dec 21 12:18:22   OPNsense charon: 03[IKE] <con1-000|3> sending retransmit 2 of request message ID 0, seq 3
Dec 21 12:18:21   OPNsense charon: 03[IKE] <con1-000|3> received retransmit of response with ID 0, but next request already sent
Dec 21 12:18:21   OPNsense charon: 03[NET] <con1-000|3> received packet: from 10.3.32.60[500] to 10.3.32.59[500] (180 bytes)
Dec 21 12:18:15   OPNsense charon: 03[NET] <con1-000|3> sending packet: from 10.3.32.59[500] to 10.3.32.60[500] (108 bytes)
Dec 21 12:18:15   OPNsense charon: 03[IKE] <con1-000|3> sending retransmit 1 of request message ID 0, seq 3
Dec 21 12:18:11   OPNsense charon: 13[NET] <con1-000|3> sending packet: from 10.3.32.59[500] to 10.3.32.60[500] (108 bytes)
Dec 21 12:18:11   OPNsense charon: 13[ENC] <con1-000|3> generating ID_PROT request 0 [ ID HASH N(INITIAL_CONTACT) ]
Dec 21 12:18:11   OPNsense charon: 13[ENC] <con1-000|3> parsed ID_PROT response 0 [ KE No ]
Dec 21 12:18:11   OPNsense charon: 13[NET] <con1-000|3> received packet: from 10.3.32.60[500] to 10.3.32.59[500] (180 bytes)
Dec 21 12:18:11   OPNsense charon: 13[NET] <con1-000|3> sending packet: from 10.3.32.59[500] to 10.3.32.60[500] (196 bytes)
Dec 21 12:18:11   OPNsense charon: 13[ENC] <con1-000|3> generating ID_PROT request 0 [ KE No ]
Dec 21 12:18:11   OPNsense charon: 13[CFG] <con1-000|3> selected proposal: IKE:AES_CBC_128/HMAC_SHA1_96/PRF_HMAC_SHA1/MODP_1024
Dec 21 12:18:11   OPNsense charon: 13[IKE] <con1-000|3> received FRAGMENTATION vendor ID
Dec 21 12:18:11   OPNsense charon: 13[IKE] <con1-000|3> received DPD vendor ID
Dec 21 12:18:11   OPNsense charon: 13[ENC] <con1-000|3> parsed ID_PROT response 0 [ SA V V ]
Dec 21 12:18:11   OPNsense charon: 13[NET] <con1-000|3> received packet: from 10.3.32.60[500] to 10.3.32.59[500] (128 bytes)
Dec 21 12:18:11   OPNsense charon: 06[NET] <con1-000|3> sending packet: from 10.3.32.59[500] to 10.3.32.60[500] (180 bytes)


However, if on SmallWall box, I configure one connection Phase I mode as main and the rest of connection Phase I mode as aggressive, it will connect fine. Any Idea? Why this work? It should be main mode on both two network configuration on SmallWall.


Best Regards,
Somnuk

[mimugmail]

With this explanation I'd rather search for the error on SmallWall Forums ...