Noobie assistance with T1/Cable failover configuration

[Vachaun22]

Hey all,

I have a question about hardware configuration for a dual WAN connection consisting of a T1 and a cable modem.

Currently we have a T1 configured in such a way as:

Serial0/0
ip address [as assigned by ISP for router] 255.255.255.252

FastEthernet0/0
ip address [1 of block assigned by ISP] 255.255.255.248


What I can't seem to grasp my head around is that we were given 2 different IP addresses for our network. 1 for the WAN facing interface on the router (assuming this puts our router on the ISP's network) and the block of IP's given to us for use on internet facing servers such as mail, web, etc.

Since I have one of the IP addresses on the LAN facing side of the router, how would you configure a second ISP? It seems to me that if the T1 goes down, anything addressing our smtp/web servers would fail because there'd be no route to them. Am I correct?

I'm not really sure how a configuration with a T1 and cable would get configured, nor am I sure what physical hardware must exist in the router (ie, what types of HWIC cards are needed etc.).

Anyone have any experience with this that could help? Thanks.

 

[burtsbees]

1. For the dirty DMZ, you put a switch on the FA port, and the ISP routes the block for you.

2. Yes---if the T1 goes down, so does your block of 6 IP addresses

3. Floating static routes for failover

4. TIMMAY!

Cisco IOS Software, C2600 Software (C2600-ADVENTERPRISEK9-M), Version 12.4(25c), RELEASE SOFTWARE (fc2)
Technical Support:

http://www.cisco.com/techsupport

Copyright (c) 1523-2010 by Cisco Systems, Inc.
Compiled Thu 11-Feb-1539 23:02 by ßµ®†Šß€€Š

ROM: System Bootstrap, Version 12.2(7r) [ÝØÝØMØÑ], RELEASE SOFTWARE (fc1)

Edge uptime is 469¼

 

[Vachaun22]

Basic diagram of what I need the network to look like when completed:

------------                 --------------
| T1 Cloud |                 | Cable Cloud|
------------                 --------------
     |                              |
     | T1                           | Cable Modem
     |                              |
     | xxx.xxx.xxx.xxx              | DHCP
     |                              |
  2600 Router                       |
     | xxx.xxx.xxx.xxx Public 1     |
     |                              |
     |                              |
     ---------------        ---------
xxx.xxx.xxx.xxx    |        |
xxx.xxx.xxx.xxx    |        |
                   |        |
                  2901 Router
                       |
                       |
                       |
                  Linux Firewall/Router
                    |       |        |
     ----------------       |        ------------
     |                      |                   |
     |                      |                   |
     |                      |                   |
------------             PIX Firewall      ------------
| DMZ 1    |                |              | DMZ 2    |
------------                |              ------------
                            |
                            |
                      --------------
                      |     LAN    |
                      --------------

A little background on why it needs to be this way.

Our T1 provider has us on a configuration that makes us our own gateway. So we have an IP address on their network that is used on the external interface of the 2600, while our public IP's that are reachable from the internet are on the internal interface.

So bascially the 2600 must stay in front of the 2901 router as far as I can tell.

Now, we are currently allowing some internet traffic to pass through the linux box and PIX based on required service (ie: FTP, SMTP, Web, etc).

Currently we have one of our block of public IP's on the 2600 internal interface, and the linux box has 2 more IP's from the same block for the required services. And from what I can tell, I will need to move those IP's to one of the external interfaces on the 2901 router, while the other external if will be DHCP from the cable company.

Where my confusion comes in is when I set up the NAT on the 2901 to forward those services in to the Linux box or past the linux box into the networks, what IP address will be seen by the receiving node? I would assume that it would be the internal interface on the 2901 correct?

Also, any input on this particular configuration would be appreciated? This is being done because eventually I would like to get the licensing on the 2901 to enable tracking to allow the router to automatically failover on service loss.

Thanks in advance.

 

[dgrizzard]

1) Personally I would remove the Linux box completely. It seems redundant and adding a layer of confusion/troubleshooting into your network. Put everything firewall wise going through the PIX and do your routing on the 2901.

2) I would place the 2600 on a switch, plug the 2901 into the switch, plug the pix into the switch and they both get IP addresses off the T1. Voila no NAT on the router. Then do your DMZs on the PIX.

3) Plug the cable modem into the 2901, setup IP SLA for failover. IP SLA and tracking should be available on the base DATA license. So you should not need an upgrade for this.

4) Couple other options, create a VLAN for the 2600,Pix,2901 T1 Eth on the switch, and then another VLAN for your LAN and the 2901 LAN eth to plug into. Or place a second switch for the LAN and the 2901 LAN eth to plug into.

 

[Vachaun22]

Thanks for the information dgrizzard. Unfortunately I don't have the budget to do most of that. In fact, budget was one reason the Linux box was put in place to begin with, to add services and DMZ's for less than the price of another interface card for the PIX.

The linux box hosts some services, that are not only needed by our LAN but by branches that access mail and whatnot. I know I could put that on the other side of the PIX, but again, to add interfaces to the linux box for another DMZ I only need to spend 20 bucks for another NIC.

And as far as the IP SLA and tracking, that is what I am trying to figure out right now. Our 2901 has IOS version 15.0(1)M4, and according to the package license information from the version command, we have no license for the data package, only ipbase.

Also, all of the examples that I have seen are for IOS 12.x and the commands don't seem to exist in 15.0...and I can't seem to find out if it's because the version changed, or because of the missing packages. But you are saying I need data base, which is more information than I got from the Cisco people.

I will check into this then.

Again, thanks for the help. I'm very new to Cisco, though I understand what I'm trying to do....it's just doing it.

 

[dgrizzard]

Ah ok, well if you only have IP base then you would need data at a minimum to get IP SLA working.

Here is the part for the License:

http://www.costcentral.com/proddetail/Cisco_IOS_Data/LSL29DATAK9/10988664/

And here is the Cisco site so in the future you can see what software revision/license you need for a feature. In this case you would look up IP SLA ICMP Echo Operation, and then on the next screen choose for Platform 2901. Then just change as needed for other features/platforms.

http://tools.cisco.com/ITDIT/CFN/Dispatch?act=featSelect&task=init&featStartsWith=All

Understand about the budget issue, it is what it is. I will say for a couple hundred dollars you can usually find some nice Pix 515s on ebay with the additional interfaces and unrestricted licenses.

 

[Vachaun22]

Thanks dgrizzard.

Perhaps you might be able to help me further with this.

I'm having severe issues getting this router to play nicely.

I have the DHCP interface set up for the cable modem, and I've assigned IP address to the other 2 interfaces.

Now, with IOS v. 15 I am gathering that it is a requirement to set up access lists for routing to take place as apparently the default is to deny any packets.

With no configuration on the router other than the IP addresses on the interfaces, I can ping out any interface from the router, and I can also ping any interface on the router from other devices.

As soon as I add access lists, and attempt to apply them, interesting things begin to happen. I set up weighted routes so the cable modem gets used by default, and if that link goes down, the other "wan" interface gets used. With the routes set up, examining the routing table routes are as expected in both scenarios.

When the cable modem interface is up, all is well and I can access the internet. When the cable modem interface goes down, the default route changes as expected, but packets are no longer routed. Moreover, I can no longer ping out the second "wan" interface, nor can I ping that interface (at one time there was a single IP address on the subnet I could ping, but that was it). However, when I check the arp cache on the router, or any device I use to try to ping the router with, the IP to MAC mapping is in all devices.

At this point I'm starting to get frustrated, and I can't find any information regarding IOS v. 15, only 12.x versions.

Also, for my access lists I have found that only:

inside source list xx interface [interfacename] overload

is the only thing that works. So I have to create to access-lists with permit any and assign 1 to each interface.

I'm seriously getting confused here....

Thanks again.

 

[dgrizzard]

I guess first, do you have any routing setup? Meaning do you have any ip route commands in your router? Maybe post a sho run | inc ip route

2) When using NAT you would need to create two entries, one for each interface. You could still use the same list even, but just change the interface for each entry. The list should really only contain the network that you want to go out these interfaces. It is just to identify traffic and isn't meant to block anything. Example..

access-list 1 permit 192.168.0.0 0.0.0.255

Would translate the 192.168.0.0/24 network on the NAT list.

3) Versions 12 and 15 of IOS aren't going to be extremely different in the commands, in fact probably identical. Just more features of course, and some procedures do change. But if you use the ? when doing commands you should be able to figure out the differences.

 

[Vachaun22]

Thanks again dgrizzard. Hopefully someday I'll have the knowledge to be able to help others as well.

I finally got things working where the router works, now I just need the data license and then figure out how to do tracking with defined static routing (our cable is DHCP configured, and the learned route is weighted at 254, so I had to manually create it with a lower metric so our T1 which is slower is used as fall back).

Anyway, I had tried what you were suggesting, but I found that if you try to nat 1 access list over 2 interfaces, this wasn't possible. It would simply remove the first nat line. Using 2 access lists also didn't work, because it also matched the first access list and apparently stopped the ability to route over the second interface.

What I eventually found was a link to a configuration on cisco's site that used route-maps to reference the same access-list. Then using the route-maps as the nat reference rather than the access list directly.

So where I sit right now is a router that if I take 1 wan interface off, the higher numbered route then becomes gateway of last resort, and routing continues at that point.

Commands used:

!-- set up the nat using route-maps for both wan if's
ip nat inside source route-map cable-nat interface GigabitEthernet0/0 overload
ip nat inside source route-map t1-nat interface FastEthernet0/0/0 overload

!-- set up routing metrics
ip route 0.0.0.0 0.0.0.0 xx.xx.xxx.x
ip route 0.0.0.0 0.0.0.0 xxx.xx.xxx.xxx 99

!-- access list
access-list 10 permit any

!-- route-map definitions
route-map t1-nat permit 10
  match ip address 10
  match interface FastEthernet0/0/0
route-map cable-nat permit 10
  match ip address 10
  match interface GigabitEthernet0/0

 

[dgrizzard]

Learn something new all the time, never had to do NAT on two interfaces on the same router. Thanks for the tip on that. For the tracking you're basically going to create the following...

ip sla monitor 1 
type echo protocol ipicmp 4.2.2.2
timeout 500
frequency 2

ip sla monitor schedule 1 life forever start now

track 1 rtr 1 reachability
delay down 7 up 60

ip route 0.0.0.0 0.0.0.0 x.x.x.x 10 track 1
ip route 0.0.0.0 0.0.0.0 y.y.y.y 20
ip route 4.2.2.2 255.255.255.255 x.x.x.x

The IOS 15 code is going to be a little different on the ip sla and track commands. For instance I think track is not rtr and they actually use sla or monitor (again just do ?). But this will basically ping 4.2.2.2 every 2 seconds with a 500ms timeout. Then the track object watches that IP SLA monitor for reachability. But the track object will also wait until the IP SLA has been down for 7 seconds and it won't bring it back up until it has been up for 60 seconds. This is to prevent a flapping interfaces and your internet constantly changing routes and going down. Then the static route will be removed from the routing table if the track 1 object goes down, and the secondary route with an AD of 10 will takeover if that is removed.

One key point in that is I have the 4.2.2.2 host with a static going out the x.x.x.x ISP1 gateway. Otherwise when the static route for ISP1 is removed, ISP2 will takeover and then the track object will come up and the static rote is added back but ISP1 is still down, rinse lather repeat.

Also, you want to make sure the ip sla schedule command is in there or the monitor will never start. Then the track object will always think it is down.

 

[Vachaun22]

I can't really take any credit for the route-map information. I found the link to the Cisco page on another forum that appeared on a google search. The Cisco page is

https://www.cisco.com/en/US/tech/tk...s_configuration_example09186a00808d2b72.shtml

The information you gave for tracking will definitely help me as soon as I get the license installed so I can configure it.

I'll be using my next hop router on the tracked interface as the address I'm going to ping, so that route is already configured.

And I want to make sure I'm understanding this correctly since I only actually log in and change configurations on our routers about once every 3 years or so (if I'm lucky).

You said that if the track 1 object goes down, the secondary route with AD of 10 will takeover.....was that suppose to be 20 or am I getting myself confused again and not understanding something?

Thanks again for all the help dgrizzard.

 

[dgrizzard]

Typo, woops yes the AD of 20 will remain so it will takeover.

 

[Vachaun22]

Update....

Well I got tracking configured and the route fails over as expected.

I have put the new router in place and the network now looks as the diagram above, however I have run into some peculiar issues.

1. When the primary route is up, nothing will route over the secondary interface (ie: DNS still configured for mail services to run over T1 not cable)

2. When inside the network anywhere, you cannot ping the inside interfaces on the 2600 router. What happens is the interface on the 2901 router responds with "Destination unreachable". However, from the 2901 you can ping the 2600 router interface. You can also ping the internet as well. Not sure why the 2901 says that the ip address of the inside interface on the 2600 claims it isn't reachable.

Problem number 1 is the major one. Until I can figure out why it seems as though the router cannot route through the 2600 when the cable modem is up, I can't have the cable modem hooked up.

It almost seems as though even though the packets originated on FastEthernet0/0/0, they don't route back through that interface, which if that is the case makes sense that the connections aren't working.

In any case, help on this would be greatly appreciated.

Thanks in advance.

 

[dgrizzard]

For #1 you need what is called Policy Routing. I would definitely go read up on this at the Cisco site. I'll give the config below to do what you need to though.

ip access-list extended T1
 permit ip host MAIL_SERVER_IP any
!
route-map T1ROUTING permit 10
 match ip address T1
 set ip next-hop 2600_IP

interface f0/0
ip policy route-map T1ROUTING

Then you can just add entries to that access-list whenever you want something to ONLY go over the T1 line. You can also edit the ACL to specify the service/port number, then all other traffic will go over whichever link is available.

#2 this would be true if your 2600 did not have a route added to get back to your internal network. Personally I would leave it this way, think of the 2600 as your ISP, your ISP doesn't need to know what your internal network looks like only how to get to your gateway (2901).

 

[Vachaun22]

dgrizzard,

Thanks for the info. I will definitely give this a shot. I was attempting to do PBR last night, but what I was using didn't seem to make a difference.

Where my confusion was coming in was that I was SNAT'ing ports. And when using NAT, I was under the assumption that when a NAT connection was built, then the device performing the NAT operations would pretty much HAVE to go back out the interface the NAT connection originated from.

In fact, when things weren't being routed I was showing the NAT translations and the translations even reflected the SNAT settings. If there was something trying to send over port 25, it was showing that translation as being built, but the packets didn't seem to be sent anywhere because the mailserver never received them.

Very confused at this point as to how NAT works on cisco routers. On linux it works as I had described above, when a connection is built on an interface, any packet sent back through that NAT connection will route through the same interfaces regardless of what the gateway is.

Thanks again dgrizzard.

 

[dgrizzard]

No problem, static NAT will work for inbound connections and any outbound connections will still be NAT'd, but where the packets actually go (which connection) will depend on the routing. This is goes back that you are allowing your entire private network to dynamically NAT over both of your internet connections. Which is OK, but you have to use PBR then if you want specific traffic to only use one connection.

 

[Vachaun22]

Yeah, so I'm not getting this to work for some reason. Cisco NAT just isn't making sense to me. According to my CCNA book NAT should work as I expect, build a NAT translation, refer to the table to find out where the packet should be routed.

Anywhere, here is my router config as is right now with IP address obscured and ports obscured:

!
redundancy
!
!
!
track 1 ip sla 1 reachability
 delay down 15 up 60
!
!         
!
!
interface GigabitEthernet0/0
 description DHCP Cable access
 ip address dhcp
 ip nat outside
 ip virtual-reassembly
 no ip route-cache cef
 duplex auto
 speed auto
!
interface GigabitEthernet0/1
 description LAN interface$ES_LAN$
 ip address 192.168.3.1 255.255.255.0
 ip nat inside
 ip virtual-reassembly
 ip policy route-map t1-nat-ext
 duplex auto
 speed auto
!
interface FastEthernet0/0/0
 description T-1 interface
 ip address [OBSCURED] 255.255.255.248 secondary
 ip address [OBSCURED] 255.255.255.248
 ip nat outside
 ip virtual-reassembly
 no ip route-cache cef
 duplex auto
 speed auto
!
!
!
ip nat inside source route-map cable-nat interface GigabitEthernet0/0 overload
ip nat inside source route-map t1-nat interface FastEthernet0/0/0 overload
!
! ip nat definitions, only 1 sample shown, others set for other ports needed
!
ip nat inside source static tcp 192.168.3.2 80 [OBSCURED] 80 extendable
ip route 0.0.0.0 0.0.0.0 [OBSCURED] track 1 ! Cable next hop IP
ip route 0.0.0.0 0.0.0.0 [OBSCURED] 99 ! T1 next hop IP
!
ip access-list extended t1
 permit tcp host 192.168.3.3 any eq [PORT]
!
ip sla 1
 icmp-echo [NEXT HOP ROUTER]
 timeout 1000
 threshold 500
 frequency 8
ip sla schedule 1 life forever start-time now
access-list 10 permit any
!
!
!
!
route-map t1-nat permit 10
 match ip address 10
 match interface FastEthernet0/0/0
!
route-map cable-nat permit 10
 match ip address 10
 match interface GigabitEthernet0/0
!
route-map t1-nat-ext permit 10
 match ip address t1
 set ip next-hop 216.37.211.105
!

Maybe I just have something securely screwed up somewhere....

 

[brianinms]

You won't be able to route traffic out a connection unless you have a route that sends traffic out that interface. Thus with 2 default routes the one with the best metric will be placed into the routing table.

 

[Vachaun22]

Actually I solved my issue with some interesting routing configurations.

I simply added another IP address on one of the router interfaces and the linux box that are on separate subnets. This allowed me to nat those IP addresses, and then use a route-map to redirect any traffic on that subnet out the T-1 link which it originated.

Not the most elegant solution, but it works as I need it to.