OPNsense Forum

Archive => 20.7 Legacy Series => Topic started by: lrosenman on August 01, 2020, 09:02:57 am

Title: pfatt and 20.7 don't seem to work
Post by: lrosenman on August 01, 2020, 09:02:57 am
I have 20.1 working great with 20.1.  When I upgrade to 20.7, I don't get DHCP from ATT.

Is there something(TM) changed between the HardenedBSD releases that possibly breaks NetGraph?

I made the grave mistake of NOT having a off machine backup of my config and had to rebuild it from scratch.

I don't want to go through this again trying to get current.
Title: Re: pfatt and 20.7 don't seem to work
Post by: franco on August 02, 2020, 11:48:28 am
TBH, we do not patch netgraph unless we have a panic report (which we had to be solved in time for 20.7).

The Beta and RC1 were available for testing and reports were acted upon if there are FreeBSD patches that solve the issue (like the LTE panic).


Cheers,
Franco
Title: Re: pfatt and 20.7 don't seem to work
Post by: fraggle on August 02, 2020, 09:26:22 pm
I can confirm the same behavior.
Auto upgraded, and broke the ATT Fiber (pfatt/opnatt) functionality.

Complete reinstall, and reconfigured pfatt, with the same results.

I can see EAP traffic headed to the ONT and a DHCP request, but no reply.  I'm assuming a EAP failure.

<edit - logs>
14:22:46.441955 xx:xx:xx:xx:xx:x0 (oui Unknown) > xx:xx:xx:xx:xx:x3 (oui Unknown), ethertype EAPOL (0x888e), length 60: EAPOL start (1) v2, len 0
14:22:46.443955 00:90:d0:63:ff:01 (oui Unknown) > xx:xx:xx:xx:xx:x0 (oui Unknown), ethertype 802.1Q (0x8100), length 68: vlan 0, p 7, ethertype EAPOL, EAP packet (0) v1, len 15
14:22:59.562188 xx:xx:xx:xx:xx:x0 (oui Unknown) > Broadcast, ethertype 802.1Q (0x8100), length 346: vlan 0, p 0, ethertype IPv4, OPNsense.localdomain.bootpc > 255.255.255.255.bootps: BOOTP/DHCP, Request from xx:xx:xx:xx:xx:x0 (oui Unknown), length 300
Title: Re: pfatt and 20.7 don't seem to work
Post by: mrancier on August 02, 2020, 11:32:27 pm
I successfully upgraded to r1 and then to release, with no issues.  I am using pfatt in traditional mode, not with eap certs, though.
Title: Re: pfatt and 20.7 don't seem to work
Post by: fraggle on August 03, 2020, 03:43:49 am
What do you mean by "traditional mode"?

All I have done is set opnatt.sh as 99-opnatt.sh (after the appropriate .sh editing for the OF_INT and MAC.

What is the "traditional", so I can work backwards to figure out where my cfg is breaking?
Title: Re: pfatt and 20.7 don't seem to work
Post by: harshw on August 03, 2020, 03:44:34 am
I have 20.1 working great with 20.1.  When I upgrade to 20.7, I don't get DHCP from ATT.

Is there something(TM) changed between the HardenedBSD releases that possibly breaks NetGraph?

I made the grave mistake of NOT having a off machine backup of my config and had to rebuild it from scratch.

I don't want to go through this again trying to get current.

Are you using the original vanilla pfatt from aus or one of the many forks that now have WPA supplicant and certificates etc?

Thinking of upgrading, need to figure out how to do it without breaking my internet ...
Title: Re: pfatt and 20.7 don't seem to work
Post by: harshw on August 03, 2020, 05:47:13 am
What do you mean by "traditional mode"?

All I have done is set opnatt.sh as 99-opnatt.sh (after the appropriate .sh editing for the OF_INT and MAC.

What is the "traditional", so I can work backwards to figure out where my cfg is breaking?

The original "traditional" pfatt script from aus was forked and now there's a couple of scripts that can use certificates from the ATT devices etc to authenticate (so you don't even need a ATT device). I believe this is what he's referring to. It's also noteworthy that the poster said he upgraded to RC1 and then to GA. So perhaps something changed between RC1 and GA that is causing the script to fail?
Title: Re: pfatt and 20.7 don't seem to work
Post by: fraggle on August 03, 2020, 04:22:50 pm
Ahh.

In that case, I'm using a more "traditional" configuration:
https://github.com/MonkWho/pfatt

NetGraph bridging of 802.1x traffic.
Title: Re: pfatt and 20.7 don't seem to work
Post by: lrosenman on August 03, 2020, 05:21:41 pm
Same here, using the monkwho opnatt.sh script bridging EAPOL. 
I did NOT install RC1 either.

Title: Re: pfatt and 20.7 don't seem to work
Post by: mrancier on August 05, 2020, 02:00:01 am
Sorry about the delay.  Life kinda got in the way.  In any case, I meant traditional in the sense that I am not using the extracted certificates from the gateway, which I do have, to do a full bypass, ie., ont to opnsense.
The only other thing I might point to is that at some point within the last upgrade the devs stopped loading the netgraph modules by default, so you have to ensure that you are adding them to your loader.conf or loader.conf.local . 
Title: Re: pfatt and 20.7 don't seem to work
Post by: lrosenman on August 05, 2020, 04:16:27 am
The current opnatt.sh script loads the appropriate netgraph modules.

A fresh install of 20.1.9_1 works, upgrading to 20.7 breaks the DHCP stuff (no address, no connectivity).

Title: Re: pfatt and 20.7 don't seem to work
Post by: harshw on August 05, 2020, 07:20:58 am
Sorry about the delay.  Life kinda got in the way.  In any case, I meant traditional in the sense that I am not using the extracted certificates from the gateway, which I do have, to do a full bypass, ie., ont to opnsense.
The only other thing I might point to is that at some point within the last upgrade the devs stopped loading the netgraph modules by default, so you have to ensure that you are adding them to your loader.conf or loader.conf.local .

Are you using the original script by aus or the one from MonkWho or others? If you're using the original script and it is working for you = then it's just a matter of seeing why the newer ones aren't working (changed locations for files and binaries perhaps?)
Title: Re: pfatt and 20.7 don't seem to work
Post by: lrosenman on August 05, 2020, 08:38:13 pm
In my case I'm using the new OPNatt.sh from MonkWho on 20.1 (I was using the original aus one when this started, and 20.7 broke it), and the current opnatt.sh script from MonkWho works on 20.1 and does NOT work on 20.7.

I can't afford to test 20.7 again as it breaks all my connectivity.

Title: Re: pfatt and 20.7 don't seem to work
Post by: mrancier on August 05, 2020, 11:08:10 pm
Sorry about the delay.  Life kinda got in the way.  In any case, I meant traditional in the sense that I am not using the extracted certificates from the gateway, which I do have, to do a full bypass, ie., ont to opnsense.
The only other thing I might point to is that at some point within the last upgrade the devs stopped loading the netgraph modules by default, so you have to ensure that you are adding them to your loader.conf or loader.conf.local .

Are you using the original script by aus or the one from MonkWho or others? If you're using the original script and it is working for you = then it's just a matter of seeing why the newer ones aren't working (changed locations for files and binaries perhaps?)

I am using the original script, or rather the last one I downloaded directly from aus's github before it evaporated.
Title: Re: pfatt and 20.7 don't seem to work
Post by: samsonmcnulty on August 20, 2020, 01:32:49 am
Sorry about the delay.  Life kinda got in the way.  In any case, I meant traditional in the sense that I am not using the extracted certificates from the gateway, which I do have, to do a full bypass, ie., ont to opnsense.
The only other thing I might point to is that at some point within the last upgrade the devs stopped loading the netgraph modules by default, so you have to ensure that you are adding them to your loader.conf or loader.conf.local .

Are you using the original script by aus or the one from MonkWho or others? If you're using the original script and it is working for you = then it's just a matter of seeing why the newer ones aren't working (changed locations for files and binaries perhaps?)

I am using the original script, or rather the last one I downloaded directly from aus's github before it evaporated.

Could you possibly provide a copy of that script so I can compare the Monkwho opnatt script and see what might be different?
Title: Re: pfatt and 20.7 don't seem to work
Post by: lrosenman on August 20, 2020, 01:37:38 am
Last week even my 20.1 system stopped getting DHCP from ATT, so I took the OPNatt stuff out and am NOT bypassing the RG at the moment.  If we find a way to get it to work on 20.7 (I upgraded since opnatt wasn't working anyway).

I did lose IPv6 as I can't seem to convince dhcp6c et al to get the PD from the NVG599.

I'll attach the opnatt.sh script I was running on 20.1.9 before ATT (AFAIK) broke it.

let me know what else I can supply.
Title: Re: pfatt and 20.7 don't seem to work
Post by: harshw on August 21, 2020, 05:42:40 pm
Here's the original pfatt.sh script (not the newer one) if anyone wants to try this with 20.7

As usual you have to provide values for ONT_IF, RG_IF and RG_ETHER_ADDR should be set to the ATT RG mac

Code: [Select]
#!/bin/sh
set -e

ONT_IF=
RG_IF=
RG_ETHER_ADDR=
OPNSENSE='yes'
LOG=/var/log/pfatt.log

getTimestamp(){
    echo `date "+%Y-%m-%d %H:%M:%S :: [pfatt.sh] ::"`
}

{
    echo "$(getTimestamp) pfSense + AT&T U-verse Residential Gateway for true bridge mode"
    echo "$(getTimestamp) Configuration: "
    echo "$(getTimestamp)        ONT_IF: $ONT_IF"
    echo "$(getTimestamp)         RG_IF: $RG_IF"
    echo "$(getTimestamp) RG_ETHER_ADDR: $RG_ETHER_ADDR"
    echo "$(getTimestamp)      OPNSENSE: $OPNSENSE"

    echo -n "$(getTimestamp) loading netgraph kernel modules... "
    /sbin/kldload -nq ng_etf
    echo "OK!"

    if [ ${OPNSENSE} != 'yes' ]; then
        echo -n "$(getTimestamp) attaching interfaces to ng_ether... "
        /usr/local/bin/php -r "pfSense_ngctl_attach('.', '$ONT_IF');"
        /usr/local/bin/php -r "pfSense_ngctl_attach('.', '$RG_IF');"
        echo "OK!"
    fi

    echo "$(getTimestamp) building netgraph nodes..."

    echo -n "$(getTimestamp)   creating ng_one2many... "
    /usr/sbin/ngctl mkpeer $ONT_IF: one2many lower one
    /usr/sbin/ngctl name $ONT_IF:lower o2m
    echo "OK!"

    echo -n "$(getTimestamp)   creating vlan node and interface... "
    /usr/sbin/ngctl mkpeer o2m: vlan many0 downstream
    /usr/sbin/ngctl name o2m:many0 vlan0
    /usr/sbin/ngctl mkpeer vlan0: eiface vlan0 ether

    /usr/sbin/ngctl msg vlan0: 'addfilter { vlan=0 hook="vlan0" }'
    /usr/sbin/ngctl msg ngeth0: set $RG_ETHER_ADDR
    echo "OK!"

    echo -n "$(getTimestamp)   defining etf for $ONT_IF (ONT)... "
    /usr/sbin/ngctl mkpeer o2m: etf many1 downstream
    /usr/sbin/ngctl name o2m:many1 waneapfilter
    /usr/sbin/ngctl connect waneapfilter: $ONT_IF: nomatch upper
    echo "OK!"

    echo -n "$(getTimestamp)   defining etf for $RG_IF (RG)... "
    /usr/sbin/ngctl mkpeer $RG_IF: etf lower downstream
    /usr/sbin/ngctl name $RG_IF:lower laneapfilter
    /usr/sbin/ngctl connect laneapfilter: $RG_IF: nomatch upper
    echo "OK!"

    echo -n "$(getTimestamp)   bridging etf for $ONT_IF <-> $RG_IF... "
    /usr/sbin/ngctl connect waneapfilter: laneapfilter: eapout eapout
    echo "OK!"

    echo -n "$(getTimestamp)   defining filters for EAP traffic... "
    /usr/sbin/ngctl msg waneapfilter: 'setfilter { matchhook="eapout" ethertype=0x888e }'
    /usr/sbin/ngctl msg laneapfilter: 'setfilter { matchhook="eapout" ethertype=0x888e }'
    echo "OK!"

    echo -n "$(getTimestamp)   enabling one2many links... "
    /usr/sbin/ngctl msg o2m: setconfig "{ xmitAlg=2 failAlg=1 enabledLinks=[ 1 1 ] }"
    echo "OK!"

    echo -n "$(getTimestamp)   removing waneapfilter:nomatch hook... "
    /usr/sbin/ngctl rmhook waneapfilter: nomatch
    echo "OK!"

    echo -n "$(getTimestamp) enabling $RG_IF interface... "
    /sbin/ifconfig $RG_IF up
    echo "OK!"

    echo -n "$(getTimestamp) enabling $ONT_IF interface... "
    /sbin/ifconfig $ONT_IF up
    echo "OK!"

    echo -n "$(getTimestamp) enabling promiscuous mode on $RG_IF... "
    /sbin/ifconfig $RG_IF promisc
    echo "OK!"

    echo -n "$(getTimestamp) enabling promiscuous mode on $ONT_IF... "
    /sbin/ifconfig $ONT_IF promisc
    echo "OK!"

    echo "$(getTimestamp) ngeth0 should now be available to configure as your pfSense WAN"
    echo "$(getTimestamp) done!"
} >> $LOG
Title: Re: pfatt and 20.7 don't seem to work
Post by: cwied on August 22, 2020, 08:30:26 pm
For what it's worth, I just upgraded to 20.7, and I still seem to be able to get DHCP from ATT using the traditional bypass. I've compared my pfatt.sh to opnatt.sh and they're mostly identical.

I do have a few extra parameters on the ifconfigs to make the interfaces promiscuous, but I wouldn't have thought that they should matter:
Code: [Select]
/sbin/ifconfig $RG_IF promisc -tso4 -tso6 -vlanhwtso
/sbin/ifconfig $ONT_IF promisc -tso4 -tso6 -vlanhwtso

Title: Re: pfatt and 20.7 don't seem to work
Post by: cwied on August 22, 2020, 11:24:38 pm
A quick addition: one thing that broke for me recently on reboot was IPS. I was able to get IPS running some time ago, but after a power outage a week ago, my system would no longer maintain connectivity. It would come up just fine, but lose connectivity after a short time (less than a minute). I think that the netmap configuration for IPS somehow conflicts with the netgraph configuration for the bypass. Turning off intrusion detection fixed the problem. I haven't looked for a solution to have both work.
Title: Re: pfatt and 20.7 don't seem to work
Post by: harshw on August 23, 2020, 02:57:41 am
For what it's worth, I just upgraded to 20.7, and I still seem to be able to get DHCP from ATT using the traditional bypass. I've compared my pfatt.sh to opnatt.sh and they're mostly identical.

I do have a few extra parameters on the ifconfigs to make the interfaces promiscuous, but I wouldn't have thought that they should matter:
Code: [Select]
/sbin/ifconfig $RG_IF promisc -tso4 -tso6 -vlanhwtso
/sbin/ifconfig $ONT_IF promisc -tso4 -tso6 -vlanhwtso

The original script also has promiscuous mode enabled on the RG and ONT

Code: [Select]
    echo -n "$(getTimestamp) enabling promiscuous mode on $RG_IF... "
    /sbin/ifconfig $RG_IF promisc
    echo "OK!"

    echo -n "$(getTimestamp) enabling promiscuous mode on $ONT_IF... "
    /sbin/ifconfig $ONT_IF promisc
    echo "OK!"

The only additional parameters are for TSO. The -tso4/-tso6 disables TSO for IPv4 and v6. And the -vlanhwtso disables TSO on VLAN. That really shouldn't affect anything ...

Title: Re: pfatt and 20.7 don't seem to work
Post by: samsonmcnulty on September 18, 2020, 04:59:55 am
Has anyone that had their setup break with the update been able to confirm a workaround or cause for this?
Title: Re: pfatt and 20.7 don't seem to work
Post by: BeepDog on September 27, 2020, 04:29:43 am
I just upgraded from 20.1 to 20.7.

I'm using the original pfatt.sh from aus. So far, so good. I don't know if that's because I'm already authed, but I haven't had any troubles yet. If the internet is still up tomorrow, I can assume it works fine?

Everything came up, the interfaces remain the same, I have ipv4, but not ipv6. Hopefully that's not the start of something bad...