Ports appear to be open, but they are not

[amd.64]

I am running OPNsense 24.7.12.

I have a self housed spam server (Xeams) running on Debian 12. After being filtered mail is forwarded to Exchange Server.

I was able to receive email just fine up until about late afternoon on the 20th of January. Despite having port forward rules to forward ports 25, 80, 443 using the port checker at ping.eu shows they are not open. I ran nmap from the local side which shows all ports are open on the server itself. I ran nmap from a remote location and nmap tells me it appears to be down. I have a block of 5 static IPs. Scanning at least two of the other five produce different then scanning the one in question (IE I have .105 through 109) scanning .105 and .107 they show as being up, trace route etc -- scanning .106 nmap says it appears to be down)

Anybody have any idea how I can resolve this? Is this an issue on the ISP side?

[dseven]

Run a packet capture on your WAN interface and see if your connect attempts are arriving there?

[amd.64]

I think I have figured out what the issue is, just don't know what to do about it.

It appears that the issue is related to or caused by the one to one NAT. If I run the command curl ifconfig.me I get .105 which is the statically assigned IP address of the WAN. Instead it should provide .106 which is the virtual IP / one to one address. I have deleted and recreated the one to one NAT which did not resolve the issue. I have restarted the firewall and even restored defaults.

I also have two other publicly accessible servers running curl ifconfig.me presents me with the proper IP address of the one to one NAT / virtual IP for that server.

If it matters (I don't see how it could but ...) the OS of the server in question is Debian 12 the other two servers are Ubuntu 24.

[amd.64]

I have verified that both the public and private IP address' are the correct ones.

[amd.64]

Doing further investigation, If the one-to-one is disabled, I can perform a tracert on remote system with a different ISP (IE Verizon, I have Comcast). However, if I have the one-to-one enabled I get to the hop just before it hits my IP then it times out. IE it times out when it should hit my IP.

[amd.64]

I have done everything I can think of.

• I can take the IP in question and put it in my notebook and ping in and out.
• I have tried two different versions of OPNSense 24.7.3_1 and 24.7.12
• I have tried two different PCs, one with an Intel quad port NIC and one with two RealTek NICS
• I have restored to defaults and manually reconfigured all settings on both PCs and both versions tried
• Of the three virtual IPs i have set up and enabled with one-to-one enabled I am unable to ping any of them, with the virtual IP disabled I can ping all of them. (Yes I know that is backward.
• Running NMAP on the IP in question with the one-to-one enabled nmap tells me the system appears to be down, with the one-to-one disabled I get more "positive" results IE trace rt, port scan etc.
• I have tried a packet capture on the IP in question and the only thing I got was a "Who is" broadcast, with no response.
• MY ISP (Comcast) swears (I spoke with multiple people) there is no issue with the IP, IE has not been assigned to another customer (which makes since), no router issues, ports not blocked etc. Although I am expecting a phone call by Saturday from tier 2 support.
• Host firewall is in use but disabled for testing purposes
• Other virtual IPs although experience the same behavior with pinging, only get response with one-to-one disabled, all work just fine. IE ports forwarded and work on those IPs just as intended. It is just the one IP that is having issues.

I would ask my ISP for a new block of IPs if I thought that would work, but I don't want to go through all that work, update DNS etc if it isn't going to work. But then on the other hand if I don't try how will I know?

I am at a loss of what to do from here!

Anyone else have any ideas?

[really_lost]

I agree with dseven.  Run a packet capture on the WAN. Assuming you see them, then alter your one to one NAT rule to log. Also create a rule on the LAN side to permit logging of packets to the Debian 12 box.

You do mention that even with the one to one in place, you are getting the wrong IP for outbound packets, so it's possible the packets are making it all the way to the Debian box, but the replies to the TCP handshake come from the .105 ip instead of .106. A packet capture on the Debian 12 box could be informative too.

I too have a /29. I've only done port forwards, not 1 to 1, so I can't help with settings there.

I will say you should do more to narrow down in what point in the process are things breaking. With that info, others that have worked with 1 to 1 NATs may have some insight.

Is it possible something on the Debian firewall is permitting LAN but not non-LAN?

[amd.64]

I agree with dseven.  Run a packet capture on the WAN. Assuming you see them, then alter your one to one NAT rule to log. Also create a rule on the LAN side to permit logging of packets to the Debian 12 box.

I have run a packet capture on the WAN, and only got a "Who is" with no response. I did run it with one-to-one enable. I will also run it with it disabled.

You do mention that even with the one to one in place, you are getting the wrong IP for outbound packets, so it's possible the packets are making it all the way to the Debian box, but the replies to the TCP handshake come from the .105 ip instead of .106. A packet capture on the Debian 12 box could be informative too.

This is why I posted to the forums as well as here, I have been trying to figure this out for quit a few days and I think I might be getting tunnel vision.

But then again, 443 is open on .105 as well which goes to the Exchange Server. So wouldn't it still report as being open?

I will say you should do more to narrow down in what point in the process are things breaking. With that info, others that have worked with 1 to 1 NATs may have some insight.

Again, this why I asked for help, may times I have been sitting here looking at my screen telling my self I don't know what else to do. I did not think of this.

Is it possible something on the Debian firewall is permitting LAN but not non-LAN?

Possible? I suppose - in theory anyway. But I doubt it. I did try to access it and used the port checker with the host firewall disabled. Both attempts failed.

[amd.64]

I got it working, however, being completely honest I have no idea how.

I decided that I did not like the network scheme. It half matched the scheme that I use on my private network. I deceided I wanted my DMZ to closely match the public scheme. I changed the 2nd,3rd and 4th octets of the private IPs to match the 2nd,3rd and 4th octets of my public IPs. Now for what ever reason all port forwards, work as I want.

I am having issues with DNS. It works locally, but will not work for public sites / addresses. But I don't see how that would have affected port forwarding.