OSPF Route Selection Issue 1

[SweetRevelation]

Hello all,

I have an OSPF issue with a certain route being selected over another despite them being the same type (external type 1) and the route being selected having a HIGHER cost.

Our network has a main office which is on area 0 and a colocation facility running area 1 let's say. There are two connections between the main office and the Colocation facility. One is a DS3 and the other is a VPN tunnel. The VPN tunnel is for use when the DS3 goes down, and both links are running on area 1.

So at our main office there are two area border routers each with area 0 and area 1, the area 1 interface being the VPN and the DS3, these are different devices too by the way.

OSPF network commands have been set up very specifically to only run ospf on links between routers so ospf is only running on our /30 subnets and then we are redistributing connected subnets into ospf as external type 1 so that it will pay attention to cost unlike type 2...

The cost of the interfaces for the VPN has been set to 100.

When the VPN is down ospf converges and when I am connected to a router at the colo in area 1 and I look at the routing table trying to get to a network living on the VPN router at the main office and I see the route in the table with a metric of say 50. This looks good.

I then bring up the VPN between the office and the colo facility and ospf reconverges and I check the routing table for the same route and now I have a route, pointing through the VPN, with a metric of 110. So, the connected routes are redistributed into OSPF, then the cost of the VPN is added bringing it to 130, way above the 50 that this route has replaced.

Both routes appear in the routing table as external type 1, but the one with the higher cost (physically closer but higher cost) is chosen and put in the routing table instead of the one with the lower cost.

I have re-created this issue in GNS3 using cisco 2691s, it behaves exactly the same way. Here is a debug of WHEN the router decides to do this, the question is WHY?

*Mar 1 00:59:17.075: OSPF: Start processing Type 5 External LSA 172.20.210.0, mask 255.255.255.0, adv 172.20.200.73, age 1034, seq 0x80000001, metric 20, metric-type 1
*Mar 1 00:59:17.075: Add better path to LSA ID 172.20.210.0, gateway 172.25.200.1, dist 130
*Mar 1 00:59:17.075: Add path: next-hop 172.25.200.1, interface FastEthernet0/1
*Mar 1 00:59:17.079: network update dest_addr 172.20.210.0 mask 255.255.255.0 gateway 172.25.200.1
*Mar 1 00:59:17.079: Add External Route to 172.20.210.0. Metric: 130, Next Hop: 172.25.200.1
*Mar 1 00:59:17.079: OSPF: Path left undeleted to 172.20.210.0



Here is the routing table with the VPN up:

R5#sho ip route 172.20.210.0
Routing entry for 172.20.210.0/24
Known via "ospf 1", distance 110, metric 130, type extern 1
Last update from 172.25.200.1 on FastEthernet0/1, 00:36:38 ago
Routing Descriptor Blocks:
* 172.25.200.1, from 172.20.200.73, 00:36:38 ago, via FastEthernet0/1
Route metric is 130, traffic share count is 1


Here is the routing table with the VPN down:

R5#sho ip route 172.20.210.0
Routing entry for 172.20.210.0/24
Known via "ospf 1", distance 110, metric 50, type extern 1
Last update from 172.18.0.1 on FastEthernet0/0, 00:00:19 ago
Routing Descriptor Blocks:
* 172.18.0.1, from 172.20.200.73, 00:00:19 ago, via FastEthernet0/0
Route metric is 50, traffic share count is 1


Any OSPF gurus out here who can help me out? What's going on with this route selection?

I appreciate any help anyone can provide.

 

[Quadratic]

Interesting. From R5, can you do a "show ip ospf interface" for Fa0/0 and Fa0/1?

CCNP, CCDP, CCIP

 

[Quadratic]

My reason for asking that is because I want to be absolutely sure that both links are in the same area. There aren't many thinks that can trump cost for OSPF's decision-making, but those links being in different areas (0 and 1) would do it.

CCNP, CCDP, CCIP

 

[SweetRevelation]

Thanks for your reply and interest, Quadratic.

R5#sho ip ospf interface fastEthernet 0/0
FastEthernet0/0 is up, line protocol is up
Internet Address 172.18.0.4/29, Area 0.3.0.0
Process ID 1, Router ID 172.25.200.2, Network Type BROADCAST, Cost: 10
Transmit Delay is 1 sec, State DR, Priority 1
Designated Router (ID) 172.25.200.2, Interface address 172.18.0.4
Backup Designated router (ID) 172.18.0.1, Interface address 172.18.0.1
Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5
oob-resync timeout 40
Hello due in 00:00:03
Supports Link-local Signaling (LLS)
Index 1/1, flood queue length 0
Next 0x0(0)/0x0(0)
Last flood scan length is 0, maximum is 7
Last flood scan time is 0 msec, maximum is 4 msec
Neighbor Count is 1, Adjacent neighbor count is 1
Adjacent with neighbor 172.18.0.1 (Backup Designated Router)
Suppress hello for 0 neighbor(s)
R5#

R5#sho ip ospf interface fastEthernet 0/1
FastEthernet0/1 is up, line protocol is up
Internet Address 172.25.200.2/29, Area 0.3.0.0
Process ID 1, Router ID 172.25.200.2, Network Type BROADCAST, Cost: 10
Transmit Delay is 1 sec, State DR, Priority 1
Designated Router (ID) 172.25.200.2, Interface address 172.25.200.2
Backup Designated router (ID) 172.18.0.10, Interface address 172.25.200.1
Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5
oob-resync timeout 40
Hello due in 00:00:02
Supports Link-local Signaling (LLS)
Index 2/2, flood queue length 0
Next 0x0(0)/0x0(0)
Last flood scan length is 3, maximum is 4
Last flood scan time is 0 msec, maximum is 4 msec
Neighbor Count is 1, Adjacent neighbor count is 1
Adjacent with neighbor 172.18.0.10 (Backup Designated Router)
Suppress hello for 0 neighbor(s)
R5#


I've also got a TAC case open with this issue at the moment, if I find out anything interesting from them I will post it here for anyone curious.

 

[SweetRevelation]

I've discovered the reason for what I'm seeing in OSPF and while not many people will probably have a setup like we did, I think it's good information to put out there.

First off in our design we do not have network commands on each area, we are redistributing connected routes into the OSPF process as external routes. We work at a company with a lot of technical users that are always trying different attack vectors and disabling OSPF on most interfaces seemed the easiest way to go. So already this is not a normal setup.

The routes were being injected into OSPF as external type 1 routes which will actually take cost into effect for processing, this was important since I have a redundant connection between sites and only want one of them to be used if the other is down, so I increased the metric across the link by applying a cost on the interface on both sides.

When the VPN link was down I would see a route in the table with a low metric and traffic would be taking the path I prefer, when the VPN link is brought up the low metric route would disappear form the table and be replaced by one with a higher cost.

Upon researching this I found more about the way OSPF handles these LSAs. When you redistribute anything (connected, static, or another protocol) into OSPF your router becomes an ASBR (Autonomous System Boundary Router) and creates a type 5 LSA for this route. If you have a static route or a BGP route you would have a next hop involved in the process. When the type 5 LSA is created the next hop would be put in the LSA as the forward address. When you redistribute a connected route there is no next hop(obviously) so the forward address is written as 0.0.0.0

According to the RFC for OSPFv2 the logic states that if there is a forward address that is where you should send your packets. This makes sense, when there is a forward address of 0.0.0.0 however:

"In the above example, the forwarding address field has been set to 0.0.0.0, indicating that packets for the external destination should be forwarded to the advertising OSPF router."

So I wanted to look into more detail on this tidbit. I found:

Else, this LSA describes an AS external path to destination N. Examine the forwarding address specified in the AS-external-LSA. This indicates the IP address to which packets for the destination should be forwarded. If the forwarding address is set to 0.0.0.0, packets should be sent to the ASBR itself. Among the multiple routing table entries for the ASBR, select the preferred entry as follows. If RFC1583Compatibility is set to "disabled", prune the set of routing table entries for the ASBR as described in Section 16.4.1.

This portion of the RFC basically states "If the forwarding address in the Type 5 LSA is 0.0.0.0 then you are to send packets to the router who originated the LSA, look up the router id, and find it in the routing table, send packets this way.

This made sense to me so I looked in my routing table for 172.20.200.73 which is the router-id of the originating router for that segment. I had a route over the DS3 because of it's lower cost so I was again stumped, this should work.

Cisco TAC pointed out a flaw in the RFC that is not immediately obvious, or at least wasn't to me. In OSPF you can set your router-id to whatever you feel like. It could be 1.1.1.1 or 2.2.2.2 or whatever, it doesn't have to exist in your nework or to be routable. So, if the ASBR's router-id may not appear in the routing table, does it make sense to look there? Ultimately it really doesn't, so they had to look elsewhere. They look in the ospf database for the list of border-routers, it will obviously be in here by it's router-id.

So, looking in there I see the router-id sending over the higher cost path because IN THIS CASE the path to the ASBR can either go straight through 0.3.0.0 the area this router is in, or 0.0.0.0 in which case it would have to cross an area, the inter-area patch is chosen over the intra and it ignores the cost.

SO, even though my original route is a type E1 from both directions and therefore not subject to intra vs inter rules and should be routed according to cost, the forwarding address of 0.0.0.0 causes the router to check another table, where the intra vs inter rules DO apply and my packets take a less than ideal path.

Hope I explained that well, and this helps someone in the future.