reply-to on outbound rules

Started by ChristopherL, August 26, 2026, 11:44:05 PM

Previous topic - Next topic
Print
Go Down Pages1
User actions
August 26, 2026, 11:44:05 PM Last Edit: Today at 12:13:39 AM by ChristopherL Reason: fix anonymisation and adding state table entry
I am doing outbound filtering on some interfaces. And I seem to have struck a similar issue to this person.

Eg. I have this rule on an interface:                                               
                                                                               
Code Select
root@opnsense-test:~ # pfctl -vvs rules | grep -A 3 "drop out log quick on ixl0"
@195 block drop out log quick on ixl0 reply-to (ixl0 192.0.2.1) inet all label "83448bc8-ca97-401d-86f2-57d67fb16fb2"
  [ Evaluations: 211       Packets: 211       Bytes: 22618       States: 0     ]
  [ Inserted: uid 0 pid 90386 State Creations: 0     ]                         
  [ Last Active Time: Wed Aug 26 05:53:29 2026 ]                               
                                                                                                                                                               
I can allow traffic out of that interface with the following rule:             
                                                                               
Code Select
root@opnsense-test:~ # pfctl -vvs rules | grep -A 3 5443b2cb-c3ca-4e0c-81cf-88c04c85f00f
@120 pass out log quick inet from (self:10) to any flags S/SA keep state label "5443b2cb-c3ca-4e0c-81cf-88c04c85f00f"
  [ Evaluations: 11        Packets: 18        Bytes: 1946        States: 4     ]
  [ Inserted: uid 0 pid 24 State Creations: 11    ]                             
  [ Last Active Time: Wed Aug 26 05:45:02 2026 ]                                
                                                                                                                                                               
And then I can establish a TCP connection:                                     
                                                                               
Code Select
root@opnsense-test:~ # telnet 192.0.2.49 8443                                   
Trying 192.0.2.49...                                                           
Connected to 192.0.2.49.                                                       
Escape character is '^]'.                                                       
^]                                                                             
telnet> Connection closed.                                                      
                                                                               
And I can see the states in PF:                                                 
                                                                               
Code Select
root@opnsense-test:~ # pfctl -vvs states | grep -A3 192.0.2.49                 
all tcp 192.0.2.47:45666 -> 192.0.2.49:8443       ESTABLISHED:ESTABLISHED       
   [3811272169 + 16711936] wscale 7  [181588979 + 65792] wscale 7               
   age 00:00:02, expires in 23:59:58, 2:1 pkts, 112:60 bytes, rule 120         
   id: bddd8c6a00000000 creatorid: 3610e121                                     
                                                                                                                                                             
However, if I replace the above floating rule with one that matches the interface:       
                                                                               
Code Select
@148 pass out quick on ixl0 reply-to (ixl0 192.0.2.1) inet all flags S/SA keep state label "ed13d8d2-e403-4cf7-ba41-6b698f76643e"
  [ Evaluations: 36        Packets: 33        Bytes: 3751        States: 22    ]
  [ Inserted: uid 0 pid 70434 State Creations: 22    ]                         
  [ Last Active Time: Wed Aug 26 06:14:36 2026                                  
                                                                                                                                                               
I get responses, but the local firewall never completes the TCP handshake:         
                                                                               
Code Select
root@opnsense-test:~ # telnet 192.0.2.49 8443                                   
Trying 192.0.2.49...                                                           
^C                                                                             
root@opnsense-test:~ # pfctl -vvs states | grep -A 3 192.0.2.49                 
all tcp 192.0.2.47:59825 -> 192.0.2.49:8443       ESTABLISHED:SYN_SENT         
   [3105104415 + 4294902016] wscale 7  [4173125645 + 4294902016] wscale 7       
   age 00:00:10, expires in 00:00:28, 2:5 pkts, 120:300 bytes, rule 148         
   id: 07de8c6a00000000 creatorid: 3610e121 reply-to: 192.0.2.1@ixl0
Code Select
root@opnsense-test:~ # tcpdump -nevi ixl0 host 192.0.2.49   
tcpdump: listening on ixl0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
06:17:38.604709 38:68:dd:66:ac:e8 > 38:68:dd:66:7d:b0, ethertype IPv4 (0x0800), length 74: (tos 0x10, ttl
64, id 0, offset 0, flags [DF], proto TCP (6), length 60)                       
    192.0.2.47.50028 > 192.0.2.49.8443: Flags [S], cksum 0xff66 (correct), seq 4278843326, win 652
28, options [mss 1460,nop,wscale 7,sackOK,TS val 2155498607 ecr 0], length 0   
06:17:38.604815 38:68:dd:66:7d:b0 > 38:68:dd:66:ac:e8, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 6
4, id 0, offset 0, flags [DF], proto TCP (6), length 60)                       
    192.0.2.49.8443 > 192.0.2.47.50028: Flags [S.], cksum 0x5bb6 (correct), seq 1043354308, ack 42
78843327, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 2956288629 ecr 2155498607], length 0
06:17:39.628574 38:68:dd:66:ac:e8 > 38:68:dd:66:7d:b0, ethertype IPv4 (0x0800), length 74: (tos 0x10, ttl
64, id 0, offset 0, flags [DF], proto TCP (6), length 60)                       
    192.0.2.47.50028 > 192.0.2.49.8443: Flags [S], cksum 0xfb66 (correct), seq 4278843326, win 652
28, options [mss 1460,nop,wscale 7,sackOK,TS val 2155499631 ecr 0], length 0   
06:17:39.628651 38:68:dd:66:7d:b0 > 38:68:dd:66:ac:e8, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 6
4, id 0, offset 0, flags [DF], proto TCP (6), length 60)                       
    192.0.2.49.8443 > 192.0.2.47.50028: Flags [S.], cksum 0x57b6 (correct), seq 1043354308, ack 4$
78843327, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 2956289653 ecr 2155498607], length 0
06:17:39.628762 38:68:dd:66:7d:b0 > 38:68:dd:66:ac:e8, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 6
4, id 0, offset 0, flags [DF], proto TCP (6), length 60)                       
    192.0.2.49.8443 > 192.0.2.47.50028: Flags [S.], cksum 0x53b6 (correct), seq 1043354308, ack 42
78843327, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 2956289653 ecr 2155499631], length 0
06:17:40.690948 38:68:dd:66:7d:b0 > 38:68:dd:66:ac:e8, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 6
4, id 0, offset 0, flags [DF], proto TCP (6), length 60)                       
    192.0.2.49.8443 > 192.0.2.47.50028: Flags [S.], cksum 0x4f90 (correct), seq 1043354308, ack 42
78843327, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 2956290715 ecr 2155499631], length 0
This is even worse when attempting to connect to something not on the same subnet as my device, where I never even see the Syn packet:
[Edit] I successfully recreated this now.
Code Select
root@opnsense-test:~ # telnet 198.51.100.1 8443
Trying 198.51.100.1...
^C

root@opnsense-test:~ # pfctl -vvs states | grep -A 3 198.51             
No ALTQ support in kernel                                                       
ALTQ related functions disabled                                                 
all tcp 192.0.2.47:18469 -> 198.51.100.1:8443       ESTABLISHED:SYN_SENT       
   [1406893479 + 4294705408] wscale 7  [2561759781 + 4294902016] wscale 7       
   age 00:01:11, expires in 00:00:23, 7:12 pkts, 420:720 bytes, rule 146       
   id: 8de98c6a00000000 creatorid: 3610e121 reply-to: 192.0.2.1@ixl0
Code Select
root@opnsense-test:~ # tcpdump -nevi ixl0 tcp port 8443                                       
tcpdump: listening on ixl0, link-type EN10MB (Ethernet), snapshot length 262144 bytes               
21:55:52.902296 38:68:dd:66:ac:e8 > 00:00:5e:00:01:01, ethertype IPv4 (0x0800), length 74: (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    192.0.2.47.61030 > 198.51.100.1.8443: Flags [S], cksum 0xef3a (correct), seq 2484399761, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 416987146 ecr 0], length 0
21:55:52.912149 e2:39:b3:4e:bc:47 > 38:68:dd:66:ac:e8, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 59, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    198.51.100.1.8443 > 192.0.2.47.61030: Flags [S.], cksum 0xd0d9 (correct), seq 3967036082, ack 2484399762, win 64492, options [mss 1414,sackOK,TS val 3376431350 ecr 416987146,nop,wscale 7], length 0
21:55:53.902416 38:68:dd:66:ac:e8 > 00:00:5e:00:01:01, ethertype IPv4 (0x0800), length 74: (tos 0x10, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    192.0.2.47.61030 > 198.51.100.1.8443: Flags [S], cksum 0xeb52 (correct), seq 2484399761, win 65228, options [mss 1460,nop,wscale 7,sackOK,TS val 416988146 ecr 0], length 0
21:55:53.912051 e2:39:b3:4e:bc:47 > 38:68:dd:66:ac:e8, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 59, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    198.51.100.1.8443 > 192.0.2.47.61030: Flags [S.], cksum 0xccf1 (correct), seq 3967036082, ack 2484399762, win 64492, options [mss 1414,sackOK,TS val 3376432350 ecr 416987146,nop,wscale 7], length 0
21:55:54.940966 e2:39:b3:4e:bc:47 > 38:68:dd:66:ac:e8, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl 59, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    198.51.100.1.8443 > 192.0.2.47.61030: Flags [S.], cksum 0xc8ec (correct), seq 3967036082, ack 2484399762, win 64492, options [mss 1414,sackOK,TS val 3376433379 ecr 416987146,nop,wscale 7], length 0
^C Though, I have sometimes seen the same behaviour as with local addresses (Syns and Syn-Acks but nothing more), I just haven't been able to recreate it for the sake of this post.
                                                                               
The issue appears to be that any rule on an interface with a gateway configured (inbound or outbound) will have a reply-to value created automatically by OPNsense. You can disable this with the Disable reply-to check-box in the Edit-rule pop-up, which fixes the problem.                                                                           
                                                                               
But from my reading of man PF:
Code Select
reply-to
   The reply-to  option is similar to route-to, but routes packets that
   pass in the opposite direction (replies) to the specified  interface.
   Opposite  direction is only defined in the context of a state entry,
   and reply-to is useful only in rules that create state.   It  can  be
   used  on systems with multiple external connections to route all out-
   going packets of a connection through the interface the incoming con-
   nection arrived through (symmetric routing enforcement).It seems to me that the reply-to field on an outbound rule doesn't make sense. Shouldn't replies always be coming back to the firewall itself for an outbound rule? Or am I misunderstanding the purpose of the reply-to field here?

For reference:
Code Select
Type opnsense-business
Version 26.4.1p2
Architecture amd64
Print
Go Up Pages1
User actions