[SOLVED]: 24.7 Upgrade from 24.1.10 broke my AT&T Fiber Bypass

[adamrc]

This morning I upgraded from 24.1.10 to 24.7.  I've been successfully using the AT&T bypass which uses WPA_Supplicant along with Netgraph as seen in the instructions here:  https://github.com/owenthewizard/opnatt

Unfortunately, it hangs upon booting up.  See attached screenshot.  It no longer seems to pull the WAN IP anymore.  I can do a Control + C to continue to boot but even within the GUI, it doesn't pull an IP anymore.

Any ideas on what I could try to fix it?  It's been working great for so long prior to this update. 

[PrototypeActual]

I too have AT&T fiber but wasn't aware/using this bypass and was having issues with my AT&T and my backup Spectrum connections working. While rebooting I noticed it finally was allowing an internet connection but nothing worked after the reboot. Were you able to find a way around it or reverted back to 24.1.10 like me?

P.S.
Was setting up the bypass difficult for you or noticed any significant changes?

[adamrc]

The only fix I wanted to mess with at the time was to bring my AT&T gateway back into the mix and use that instead.  I'm curious as to if any others that have been using this AT&T gateway bypass have also experinced this issue after upgrading to 24.7.  I am really enjoying the much faster wireguard speeds on 24.7.

Setting up the bypass certainly took a bit of work but after it was all done, it has been super reliable with rock solid stability. I certainly want to get it back up and running as I don't like going through the AT&T gateway. This update is the first time any update has caused any type of disruption to my network. 

[Ben S]

I have no idea about AT&T or this bypass procedure but I had a quick read of the code in the repo you linked, and one thing I notice is your screenshot says 'Starting pfatt' but the latest code says 'Starting opnatt'.  It looks like that 'pfatt' message dates back to this commit - https://github.com/owenthewizard/opnatt/commit/134dd592d5ee459b88d94dac8a6110265ebba1a2 - in which many other things were changed, including how it runs wpa_supplicant.  I also notice your screenshot, near the bottom, it says 'wpa_supplicant running on PID', and doesn't show a PID number, suggesting wpa_supplicant is not running, which makes me think it's not calling wpa_supplicant in the right way perhaps.

So tl;dr - have you tried the latest upstream version of this script?

Also I wonder if you can find any more clues in any logs and/or try running the script from the console/SSH, to see if it gives any more diagnostics about what's going wrong.

[adamrc]

Thanks Ben S for the info.  I have not tried the latest code.  Will give that a try over the next couple of days and see what happens. 

[jay4]

I don't know why I never learn. It was a bear to setup but I got it working seamlessly, I saw the shiny upgrade button and now my bypass isn't working. I was doing the bypass with the certs, no need for the att rg.I have yet to troubleshoot with tcpdump but I'm willing to bet it's the same issue with vlan0 not being handled correctly.

[jeholliday]

This is the last time I start a major OS upgrade right before going to sleep. I was in the same boat that I've never had any other problems with updates, but this one broke my ATT bypass. I went back to the gateway until I could sort it out and was reminded why the bypass solution was so much better. I've been experiencing random connection dropouts and can't get IPv6 to work in IP pass through mode.

However, I believe I have found the solution! I ran wpa_supplicant in verbose mode and got some warnings related to TLS.

Code Select Expand

EAP: Received EAP-Request id=29 method=13 vendor=0 vendorMethod=0
EAP: EAP entering state GET_METHOD
ngeth0: CTRL-EVENT-EAP-PROPOSED-METHOD vendor=0 method=13
EAP: Status notification: accept proposed method (param=TLS)
EAP: Initialize selected EAP method: vendor 0 method 13 (TLS)
TLS: using phase1 config options
TLS: Trusted root certificate(s) loaded
OpenSSL: tls_connection_client_cert - SSL_use_certificate_file failed error:0A00018E:SSL routines::ca md too weak
TLS: Failed to set TLS connection parameters
ENGINE: engine deinit
EAP-TLS: Failed to initialize SSL.

It seems like we must have gotten a OpenSSL update or a wpa_supplicant update that makes it now reject the certificates as insecure by default. My solution is to update the wpa_supplicant.conf to include openssl_ciphers=DEFAULT@SECLEVEL=0. That makes it get further in the process. It then says that it also needs allow_unsafe_renegotiation=1 in the phase1 config. With both of those changes, I am now back in business. My complete wpa_supplicant.conf looks like this:

Code Select Expand

eapol_version=1
ap_scan=0
fast_reauth=1
openssl_ciphers=DEFAULT@SECLEVEL=0
network={
        eap=TLS
        eapol_flags=0
        key_mgmt=IEEE8021X
        phase1="allow_canned_success=1 allow_unsafe_renegotiation=1"
}

I don't know if this is the best solution, but it is working for now.

[adamrc]

I'm back up and running as well.  I scrapped everything and went to the 8311 Discord channel for bypassing and got the proper info.  Here are the files and contents that I used.  Netgraph is no longer used/needed anymore.

/usr/local/etc/rc.syshook.d/early/04-wpa (make sure to chmod +x this file)

Code Select Expand

#!/bin/sh
env OPENSSL_CONF=/conf/wpa/openssl.conf /usr/local/sbin/wpa_supplicant -Dwired -i igb0 -B -C /var/run/wpa_supplicant -c /conf/wpa/wpa_supplicant.conf

/conf/wpa/openssl.conf

Code Select Expand

openssl_conf = openssl_init

[openssl_init]
ssl_conf = ssl_sect

[ssl_sect]
system_default = system_default_sect

[system_default_sect]
Options = UnsafeLegacyRenegotiation
MinProtocol = TLSv1
CipherString = DEFAULT@SECLEVEL=0

/conf/wpa/wpa_supplicant.conf

Code Select Expand

# Generated by 802.1x Credential Extraction Tool
# Copyright (c) 2018-2019 devicelocksmith.com
# Version: 1.04 windows 386
#
# Change file names to absolute paths
ctrl_interface=DIR=/var/run/wpa_supplicant
openssl_ciphers=DEFAULT@SECLEVEL=0
eapol_version=2
ap_scan=0
fast_reauth=1
network={
        ca_cert="/conf/wpa/ca.pem"
        client_cert="/conf/wpa/client.pem"
        eap=TLS
        eapol_flags=0
        identity="REDACTED" # Internet (ONT) interface MAC address must match this value
        key_mgmt=IEEE8021X
        phase1="allow_canned_success=1"
        private_key="/conf/wpa/private.pem"
}


[effex]

Mine is still not working with the code above. The openatt.sh script seems to work just fine, however, something is not working with wpa_supplicant. I tried manually executing the script in early and it gets to "daemonize" and gives me a permission denied. Any way to further debug it?

[adamrc]

Mine is still not working with the code above. The openatt.sh script seems to work just fine, however, something is not working with wpa_supplicant. I tried manually executing the script in early and it gets to "daemonize" and gives me a permission denied. Any way to further debug it?

No need for the openatt.sh script using my method above. Just ensure certs are in the folders specified in the file.

[shermdog]

Big shout out to the folks that organized this info.  Super clean implementation and working great on 24.1 as well.

I'm back up and running as well.  I scrapped everything and went to the 8311 Discord channel for bypassing and got the proper info.  Here are the files and contents that I used.  Netgraph is no longer used/needed anymore.

/usr/local/etc/rc.syshook.d/early/04-wpa (make sure to chmod +x this file)

Code Select Expand

#!/bin/sh
env OPENSSL_CONF=/conf/wpa/openssl.conf /usr/local/sbin/wpa_supplicant -Dwired -i igb0 -B -C /var/run/wpa_supplicant -c /conf/wpa/wpa_supplicant.conf

/conf/wpa/openssl.conf

Code Select Expand

openssl_conf = openssl_init

[openssl_init]
ssl_conf = ssl_sect

[ssl_sect]
system_default = system_default_sect

[system_default_sect]
Options = UnsafeLegacyRenegotiation
MinProtocol = TLSv1
CipherString = DEFAULT@SECLEVEL=0

/conf/wpa/wpa_supplicant.conf

Code Select Expand

# Generated by 802.1x Credential Extraction Tool
# Copyright (c) 2018-2019 devicelocksmith.com
# Version: 1.04 windows 386
#
# Change file names to absolute paths
ctrl_interface=DIR=/var/run/wpa_supplicant
openssl_ciphers=DEFAULT@SECLEVEL=0
eapol_version=2
ap_scan=0
fast_reauth=1
network={
        ca_cert="/conf/wpa/ca.pem"
        client_cert="/conf/wpa/client.pem"
        eap=TLS
        eapol_flags=0
        identity="REDACTED" # Internet (ONT) interface MAC address must match this value
        key_mgmt=IEEE8021X
        phase1="allow_canned_success=1"
        private_key="/conf/wpa/private.pem"
}


[franco]

If someone wants this integrated in the GUI I'd appreciate a GitHub ticket more than these discussions after random releases breaking this stuff we never had integrated. ;)


Cheers,
Franco

[effex]

Mine is still not working with the code above. The openatt.sh script seems to work just fine, however, something is not working with wpa_supplicant. I tried manually executing the script in early and it gets to "daemonize" and gives me a permission denied. Any way to further debug it?

No need for the openatt.sh script using my method above. Just ensure certs are in the folders specified in the file.

On the identity piece, is that the RG modem's former MAC address like the same as the opnaatt.sh script or are you saying my WAN mac address on my opnsense box?

[effex]

If someone wants this integrated in the GUI I'd appreciate a GitHub ticket more than these discussions after random releases breaking this stuff we never had integrated. ;)


Cheers,
Franco

Do I just open a request to have this functionality added? I'm happy to do so if that helps.

[franco]

Yep.