A Tale of Two Protocols, Two Switches and MLS

[SoyGalesa]

Ok, this one takes a bit of explaining so here goes.

My company is in the middle of a Token Ring/Ethernet Migration and as part of this we are converting our servers to Ethernet. Each server currently has a Token Ring card as it's primary and an Ethernet card to connect to our backup devices on a seperate LAN. No problem so far everything is working.

We are upgrading by installing another Ethernet card for a primary connection, and changing the binding order in NT and NOVELL to make the Token Ring card a secondary. Still no problems.

Despite my warnings ("What to you know you're just the network guy&quot that we needed to remove the token ring cards (or at least disable them) I was overruled by the server team who said that it would not cause an issue and so the servers end up with one primary connection to Ethernet, one secondary to Token Ring and the backup LAN Ethernet card. The server guys do not always remove the Gateway IP address for the Token Ring card hence we end up with two gateways and two routes out of the server, which is a problem compounded by the following.

We are also taking the opportunity to upgrade our core 5500 switch to a brand new 6500. The 5500 has an RSM and the 6500 has an MSFC.The 5500 has two Token Ring Blades and the 5500 has two 48 port FastEthernet Blades the uncoverted servers are in the 5500 on the Token Ring blade and as the servers are migrated they are patched into the 6500.

The original plan was to run the two in parallel but to start using the MSFC for all the routing tasks and just using the 5500 as another switch. During the course of this swap over we had a problem with the MSFC system routing IPX (I believe it was an IOS version problem - which we have now fixed) and ended up using the RSM as the main router and the 6500 as a switch into which all our converted servers are patched.

Ok, so the RSM is running at about 90%-95% utilisation (Whoa!!!) and I decide to implemet MLS on the 5500 switch to try and drop this down a bit. Everything seems OK except, and this is where the problem is.

A server (WinNT 4.0 & MS SQL 6.5 in case its relevant) was converted to Ethernet and the server guy left the gateway IP address in as usual. I get in to work the next day and find that people cannot access the server properley. I initially deduce that the server is routing through the backup VLAN (since this has a gateway address in as well?????) and since this does not have a router associated with it the traffic is going nowhere. I correct this by removing the gateway address and all seems OK. People within the same VLAN as the server can access everything OK, but people on other VLANS cannot access it. This is mainly our development team who reside on a different VLAN and use SQL Enterprise Manager as to connect to the DBS server.

After a lot of investigating I decide that the cause of the issue is that traffic is routing into the server via Ethernet but routing out via Token Ring (remember what I said about the server guys not removing the gateway addresses). Well I remove the gateway address from the Token Ring card and am confident that it will solve the problem - it doesn't. People outside the VLAN cannot access the server.

This morning I turned off MLS on this VLAN, and everyone can work OK.

Questions..

1) Why was MLS failing to function correctly? I have an idea it was down to the server connections but all help would be happily received.

2) Are all server people such arrogant so-and-so's?

 

[SF18C]

1. Not really sure

2. In my exprience YESSS!!!!!

Glad you worked it out though! SF18C
CCNA, MCSE, A+, N+ & HPCC

"Tis better to die on your feet than live on your knees!"

 

[SoyGalesa]

Not quite solved, as we still need to migrate the servers and turn on MLS in the short term.

 

[baddos]

Why are your server admins so persantant on wanting to keep the token ring network? Ethernet is way faster

If you feel that the server is routing down the token ring network instead of the ethernet, try this out.

Open up a dos box "cmd.exe"

type "route print"

This should show you all the routes in the NT server's routing table. What you want to look for is the default gateway routes "0.0.0.0" and their respective metrics.

NT will use the lowest metric route. I.e. If you have two 0.0.0.0 routes, the one w/ the lowest metric will win. If they both have the same metric, Windows will do a coin toss and see who wins and use that as the default route. NT will NOT do loadbalancing/sharing. Also, if the primary default gateway becomes unreachable, NT will use the other default gateway. When the primary comes back online, NT will still stay with the secondary gateway (it won't switch back).

Basically, if your server admins are adiment about keeping the legacy token ring network in place, you can do this.

Set the default gateway on the token ring NIC to use a higher metric than 1. Make sure the ethernet's default gateway is set to 1.

I don't remember how to do this on NT4.0, but on 2000 you click on the advanced tab in the TCP/IP properties and pick a different metric.

 

[SoyGalesa]

OK, without wishing to add too much complexity to this issue already - here we go.

Historically the network has always been Token Ring and it has only started to be upgraded since I joined the company. We have an IBM AS400 server in our office and overnight certain processing is done to files extracted from the SQL servers and uploaded to the AS400. This is done via "gateway servers" running OS/2.

The problem is that the person who set up the system left the company ages ago, and there is very little documentation. Now the "gateway servers" all have Token Ring cards in and as far as I am aware are only running NetBeui as their protocol. So when I said that the Token Ring cards should be removed I knew that since NetBeui is a broadcast protocol then there should be no issues with taking the Token Cards out of these servers and NetBeui would still find the servers (after all the names are not changing, only the protocol).

Because of the lack of documentation (and the fact we have no test facilities to try these things out on) all the server guys made a big noise about the AS400/Gateway Servers/SQL Servers not working and the overnight processing (which is business critcal) failing.

As a sop to me (the cable guy) they agreed to demote the Token Ring card in the binding order on the SQL boxes, but refused to take out the gateway addresses. I have tried the metric trick and manipulating the routes via ROUTE ADD, and I've even (secretly) removed some of the Token Ring Gateway addresses etc., but the only thing that has solved it was removing MLS.

The problem is that removing MLS from this VLAN means the RSM traffic is going to be higher, and we are not quite ready to go live with the MSFC on the 6500. So we need MLS on this VLAN. We also have another five SQL boxes to comvert over the next few months, so if we have to turn MLS off to make these work there is a danger the RSM will peak over 100%.

I know it seems incredibly complicated, but just think how I feel having to support it.

 

[baddos]

If this is all on the same vlan, then just configure the switches for the same vlan. Netbui isn't routeable, so routing isn't affected by it.

Here's what I propose:

Directly connect the 5500 and the 6500 on the same vlan or on a trunk port if you are running multiple vlans.

5500 ------ 6500

Assign the OS/2, sql, and AS/400 server to the same vlan on the ethernet subnet.

Test communication. At this point, it should just be layer2 communication and no layer3. Your 5500 should work fine.

 

[jgercken]

The 6000's can only use an MSM or MSFC as their MLS RP where the 5000's can use any router that supports MLS. It sounds as if you had some routing loops (asyncronous traffic using the ethernet & token nics would do that)

I'm assuming that you are now using the 6500 as the core switch. You'll probably want to place some access-lists to filter unneeded broadcasts between vlans. Also I recommend using the protocolfilter feature. It helps a lot if you have mixed ip/ipx/decnet/appletalk protocols running.
-Jeff ----------------------------------------
Wasabi Pop Tarts! Write Kellogs today!