How to Fix: FRR BGP peer with MD5 password not connecting to peer

[kumba]

I spent a few days hating life trying to figure out why my defined BGP Peer with an MD5 password would not connect but a peer without a password would connect just fine. The short version is even though you can type in the MD5 password into the Neighbor definition it does not actually setup the TCP-MD5 connection within the kernel. This is crucial because the TCP-MD5 connection happens at layer 3 which is below where BGP operates at. The MD5 Password field combined with the Local Initiator IP in the neighbor definition just tells the FRR back-end what kind of interface/MD5 pair to hook to. You'll get log errors about zebra_client_message() failing and VNC encountered an error and exited. Basically nothing that says "Hey, I can't find this interface/MD5 pair". Joy...

The solution is to manually load the MD5 key into the kernel and have a script that runs on reboot to reload the keys.

BTW, getting to this point involved a lot of frustration and swearing. This information REALLY should be noted in the OPNsense documentation somewhere for the FRR/BGP neighbors using a MD5 password. I'm posting it here for the next guy who doesn't feel like digging through a dozen or so bug reports going back to 2017 while pouring over Wireshark/PCAPs and using a Cisco sandbox router to get relevant error logs to figure it out. For comparison Cisco says "MD5_DIGEST_MISSING:Dropping packets" when this happens. Useful logging is useful. However, lets get on with the fix.

First lets define the basic network between OPNSense/FRR and a password-protected BGP neighbor:

• BGP1: 172.16.30.1/28
• BGP1 Password: password54321
• OPNSense IP: 172.16.30.2/28

When defining the BGP Neighbor with an MD5 password in the Web GUI here's what you would type in:

• Create or Edit your BGP Neighbor and fill out all the other stuff you need
• Enable 'Advanced Mode' in the upper left corner
• Type the password into the BGP MD5 Password field, I.E. 'password54321'
• Enter the IP address facing the neighbor under the Local Initiator IP field, I.E. '172.16.30.2'
• Save your changes and apply settings/reload config

I assume if you're reading this that most of you have gotten to this point. You'll notice the neighbor status is stuck at "Active" and you'll get log errors about zebra_client_message() failed along with other weird errors. Nothing very descriptive of what's going on at all.

Now what you need to do is load the keys into the kernel itself and setup the TCP-MD5 connection. Everything below here requires SSH/shell access to the OPNSense box and basic shell knowledge. You can look up how to enable SSH and get to a shell through the normal OPNSense docs website. The first thing we're going to do is create a /root/setkey-bgpmd5.conf config file for 'setkey' to load:

Code: [Select]

#!/sbin/setkey -f
# Flush out old keys to be clean if you want, might break IPSEC things
#flush;
#
### BGP1 Neighbor
# Setup TCP connection with MD5 digest key
add 172.16.30.1 172.16.30.2 tcp 0x1000 -A tcp-md5 "password54321" ;
add 172.16.30.2 172.16.30.1 tcp 0x1000 -A tcp-md5 "password54321" ;

Once you have created the above file you can run it manually by typing 'setkey -f /root/setkey-bgpmd5.conf'. You'll now be able to connect to the BGP neighbor with an MD5 password no problem. It's beautiful.

The last thing to do is make sure that the keys are reloaded upon reboot. To do that I utilized the @reboot functionality of cron. I added the following to the end of the /etc/crontab file:

Code: [Select]

#
# Load the TCP-MD5 keys needed for BGP peers on reboot
@reboot root (/sbin/setkey -f /root/setkey-bgpmd5.conf) > /dev/null

You can now reboot and verify that you are still able to reconnect to your BGP neighbor. Hopefully this post helps the next guy save a few days of their time. If nothing else, it's something for me to find next time I do this and forget how. You will also need to check that the above crontab entry still exists after updates. Sometimes the updates will clear out the old /etc/crontab.

Let me know if there's a different/better approach to this. I'm all ears.

[mimugmail]

This is already implemented and works great, verified by some other people.

[JarJarBinks]

Okay so just in case someone else finds this thread after struggling for hours.

Yes, BGP MD5 authentication works perfectly out of the box. But you must configure “local initiator IP” for it to work.