BGP4 over Multilink Frame-Relay

[4DeanB]

Is anyone using MFR with BGP? This is my first MFR installation and I'm having problems locating info. It's so new, it's not it my books. I had (2) 2651XMs multi-homed running BGP4 over 1 T1 per provider per router. We are migrating to a dual MFR, multi-homed, BGP4 configuration. I just upgraded one router to the MFR (Which the Telco tells me is two "bonded" T1s) and left the other router as is while we wait for the second provider to install their second T1. Everything seems OK (3meg bandwidth on MFR router) with the exception of the BGP routing functionality. Previous to the MFR upgrade, we could unplug one of the provider's T1s and watch the BGP auto-failover to the other running T1 within 15 seconds. Presently, that is not the case. If we unplug one provider, the respective router is never re-discovered via an alternate BGP route. Both Telcos tell me that they verified that they are advertising each others routes. No BGP configs were altered during the upgrade. Are there any known issues or considerations? Thanks for reading my post and your help is appreciated.

 

[tschouten]

Ok, here is my thinking on this, correct if I am wrong.

You have one router bundled with two T1 links using MFR.
You unplug one and the routing doesn't change. This would be because the MFR bundle still sees an active link (the other T1 bundled with it). If you unplugged both links in the bundle the link would timeout (after a few seconds)be concidered down, the route would be concidereed bad, BGP would then perform it's function.

It is basically like an ether-channel on a switch, kill one link in the ether-channel the channel still exists. Kill all links in the channel and the channel no longer exists.

If I read your post wrong let me know. Because that sounds like what is happening to me. You aren't killing the bundle only dropping one link in the bundle.

 

[tschouten]

Ohh, should mention the default MFR hello time is 10 seconds. Wait for Response to Hello is 4 senonds before link is considered down. You can modify this time if needed. Both links would need to go through that time before the bundle would be considered down. Then you would need to wait a few seconds for the route to be knocked off the list. BGP recalculates and runs through the other route.

 

[4DeanB]

Thanks for your response tschouten. I hear what you're saying and I considered the same issue. Each T1, including the dual T1 bundled MFR, is utilizing an external Adtran CSU/DSU. During the outage simulation, I powered off both Adtrans associated with the MFR so that both links would drop. The route never recovered. I need to dig deeper.

 

[tschouten]

Gak,

I have tried looking everywhere under BGP designs. The only thing I could find was MPLS support, that just isn't going to cut it. I'm not sure BGP works with MRF? JimmyZ may know a little more than myself. I've only read about MRF never had the oportunity to use it yet. As for trying to use it with BGP. You are right I can't find a thing about it's use with BGP.

 

[jimmyzz]

4DeanB,
Taking the MFR out of it, are you configuring this environment primarily for load balancing or redundancy? Can you confirm your toplogy.

ISP1 ISP2
|| |
[routerA] <---> [routerB]
|
[routerC]
Internal LAN

If for redundancy, then you can configure BGP to your ISPs and HSRP between routerA and routerB. Then &quot;track&quot; the status of T1 interface of each router, and have routerC use the HSRP standby ip as its default route. If you are not planning to use HSRP, then iBGP needs to be configure amongst the router A B and C. Can you post your BGP config for these routers.

JimmyZ

 

[4DeanB]

JimmyZ,
Thanks for the reply. Primary concern is redundancy; zero down time. If one Provider goes to lunch, the other Provider will pick up the routes to our class C block inside both 2651XMs. ISP1 is &quot;renting&quot; us a Class C block. ISP1 also gave us a /26 block (as well as a /30 for the gateway) when we ordered their T1 which we don't use, at least not by us-does BGP need it? ISP2 gave us a /28 block when we ordered their T1 which we don't use (as well as a /30 for the gateway). Both ISPs announce the Class C block. RouterA LAN, RouterB LAN, Firewall-1 WAN, Firewall-2 WAN all use addresses from the Class C BGP block. If I kill ISP1, I can't ping routerA LAN (all pings are from outside) or firewall-1 WAN. If I trace route to these two, route dies at ISP2 gateway address. If I kill ISP2, I can't ping routerB LAN or firewall-2 WAN. Trace route to these two dies at ISP1 gateway address. I really appreciate your time and effort in looking at this. Let me know if I can elaborate on anything else.


ISP1 ISP2 (MFR)
| | |
[routerA] [routerB]
| |
|---switch--|
| |
firewall-1 firewall-2
| |
|-switch-|


Current configuration : 1463 bytes
!
version 12.2
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
!
hostname Router A

enable secret
!
ip subnet-zero
!
!
!
!
!
!
interface FastEthernet0/0
description ISP1 /26 block we don't use.
ip address xxx.xxx.xxx.xxx 255.255.255.192
duplex auto
speed auto
!
interface Serial0/0
description ISP1 T1 gateway
ip address 12.xxx.xxx.xxx 255.255.255.252
encapsulation ppp
no fair-queue
!
interface FastEthernet0/1
description Class C BGP4 Block address
ip address 12.xxx.xxx.1 255.255.255.0
duplex auto
speed auto
!
interface Serial0/1
description Soon to be connected to ISP1 MFR bundle
no ip address
shutdown
!
router bgp 26559
no synchronization
bgp log-neighbor-changes
network 12.xxx.xxx.0 mask 255.255.255.0(ISP1 class C block)
network 12.xxx.xxx.xxx mask 255.255.255.192(ISP1 block not in use)
network 66.xxx.xxx.xxx mask 255.255.255.240(ISP2 block not in use)
neighbor 12.xxx.xxx.xxx remote-as 7018(ISP1 next hop address)
neighbor 12.xxx.xxx.xxx send-community(ISP1 next hop address)

neighbor 12.xxx.xxx.xxx route-map stamp1 out(ISP1 next hop address)

neighbor 66.xxx.xxx.xxx remote-as 26559 (ISP2 block not in use)
neighbor 66.xxx.xxx.xxx next-hop-self (ISP2 block not in use)

no auto-summary
!
ip classless
ip route 0.0.0.0 0.0.0.0 12.xxx.xxx.xxx (ISP1 next hop)
ip route 66.xxx.xxx.xxx 255.255.255.240 FastEthernet0/0 (ISP2 block not in use)
ip http server
ip bgp-community new-format
ip pim bidir-enable
!
!
access-list 10 permit 66.xxx.xxx.xxx(ISP2 block not in use)
access-list 10 permit 12.xxx.xxx.xxx(ISP1 block not in use)
access-list 10 permit 12.xxx.xxx.0(ISP1 Class C)
route-map stamp1 permit 10
match ip address 10
set community 7018:20
!
route-map stamp1 deny 20
!
!
line con 0
line aux 0
line vty 0 4
password xxxx
login
!
!
end

-------------------------------------------
Building configuration...

Current configuration : 2312 bytes
!
version 12.2
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
!
hostname Router B
!
logging queue-limit 100
enable secret 5
!
ip subnet-zero
!
!
!
!
!
!
!
!
!
!
!
!
no voice hpi capture buffer
no voice hpi capture destination
!
!
mta receive maximum-recipients 0
!
!
!
!
interface MFR0
description ISP2 Multilink frame-relay
no ip address
encapsulation frame-relay IETF
load-interval 30
frame-relay multilink bid lucent110.61.lportAAA
frame-relay lmi-type ansi
!
interface MFR0.1 point-to-point
ip address 66.xxx.xxx.xxx 255.255.255.252(ISP2 Gateway)
frame-relay interface-dlci XXX
!
interface FastEthernet0/0
description ISP2 /28 block we don't use
ip address 66.xxx.xxx.xxx 255.255.255.240
duplex auto
speed auto
!
interface Serial0/0
no ip address
encapsulation frame-relay MFR0
no ip route-cache
no ip mroute-cache
no arp frame-relay
frame-relay multilink lid lucent110.61.lportBBB
!
interface FastEthernet0/1
description Class C BGP4 address
ip address 12.xxx.xxx.2 255.255.255.0
duplex auto
speed auto
!
interface Serial0/1
no ip address
encapsulation frame-relay MFR0
no ip route-cache
no ip mroute-cache
no arp frame-relay
frame-relay multilink lid lucent110.61.lport785
!
router bgp 26559
no synchronization
bgp log-neighbor-changes
network 12.xxx.xxx.0 mask 255.255.255.0 (ISP1 Class C)
network 12.xxx.xxx.xxx mask 255.255.255.192 (ISP1 /26)
network 66.xxx.xxx.xxx mask 255.255.255.240 (ISP2 /28)
neighbor 12.xxx.xxx.xxx remote-as 26559 (ISP1 /26 RouterA e0 address)
neighbor 12.xxx.xxx.xxx next-hop-self (ISP1 /26 RouterA e0 address)
neighbor 12.xxx.xxx.xxx soft-reconfiguration inbound (ISP1 /26 Router A e0 address)

neighbor 66.xxx.xxx.xxx remote-as 14900 (ISP2 GW address)
neighbor 66.xxx.xxx.xxx soft-reconfiguration inbound(ISP2 GW address)
neighbor 66.xxx.xxx.xxx route-map stamp2 out(ISP2 GW address)

no auto-summary
!
ip http server
ip classless
ip route 0.0.0.0 0.0.0.0 66.xxx.xxx.xxx (ISP2 gateway)
ip route 12.xxx.xxx.0 255.255.255.0 FastEthernet0/0
ip route 12.xxx.xxx.xxx 255.255.255.192 FastEthernet0/0 (ISP1 /26)
!
!
!
access-list 11 permit 66.xxx.xxx.xxx (ISP2 /28)
access-list 11 permit 12.xxx.xxx.xxx (ISP1 /26)
access-list 11 permit 12.xxx.xxx.0 (ISP1 BGP class C)
!
route-map stamp2 permit 10
match ip address 11
set as-path prepend 26559 26559
!
route-map stamp2 deny 20
!
call rsvp-sync
!
!
mgcp profile default
!
!
!
dial-peer cor custom
!
!
!
!
line con 0
line aux 0
line vty 0 4
session-timeout 30 output
exec-timeout 30 0
password #####
login
!
!
end

 

[tschouten]

Hey Jimmy, I cannot see anything wrong in his config, can you? I have to be missing something. In all honesty this looks like it should work.

 

[rainman]

On your ROUTERB, is it possible that BGP is not functioning properly because your peering address is 1 hop behind the physical peering address? We ran into a problem similar in the past, and the solution was:

router bgp 26559
neighbor [neighbor IP] update-source MFR0.1 point-to-point


Not sure if this is in the right direction, but it's worth a shot.

-Rainman

 

[jimmyzz]

4DeanB,
I can see a few problems with your config and design. On RouterA you've got a neighbor statement to ISP2 (AS26559) but its not in the route table of RouterA. Vice versa, on RouterB, you've got a neighbor statement to ISP1 (AS7018). Again RouterB will not have a route for ISP1. What you need to do is run iBGP between RouterA and RouterB and use HSRP for redundancy. Your topology should look like this:

(AS7018) (AS26559)
ISP1 ISP2
| ||
| (AS100) ||
[routerA] <----> [routerB]
(HSRP)
|
(HSRP Virtual GW IP)
[switch]
| |
[FW1] [FW2]
| |
Internal LAN

Basic config for this design:

RouterA:
router BGP 100
no sync
neighbor <routerB IP> remote-as 100
neighbor <routerB IP> next-hop-self
neighbor <ISP1 IP> remote-as 7018

interface FastEthernet0/0
ip address xxx.xxx.xxx.xxx (connection to RouterB)
standby 1 ip <Virtual HSRP address>
standby 1 preempt
standby 1 track Serial0/0

RouterB:
router BGP 100
no sync
neighbor <routerA IP> remote-as 100
neighbor <routerA IP> next-hop-self
neighbor <ISP2 IP> remote-as 26559

interface FastEthernet0/0
ip address xxx.xxx.xxx.xxx (connection to RouterA)
standby 1 ip <Virtual HSRP address>
standby 1 preempt
standby 1 priority 110 (make this the active router)
standby 1 track MFR0.1

You point the default gateway of your Firewalls to the HSRP virtual GW address (I'm not sure why you've got dual FWs but you can explain in your next post). Setting the priority 105 (default is 100) on routerB makes it the active/preferred HSRP router. The &quot;tracking&quot; of the MFR0.1 interface tells the HSRP router to change its priority level when this interface goes down, and in effect, routerA should take over as the active router for the HSRP group.

Each router knows about the BGP status to their ISP, but also knows about the other router via their iBGP through AS100. Give it a try and let us know how you go.

JimmyZ

 

[tschouten]

Scratches head, why didn't I catch that?