frame relay problems 2

[IXICHostage]

We have a frame relay network that consists of two remote circuits and one host circuit. For the last three months we have been experiencing drop-outs in service. Our host site has a Citrix MetaFrame, and about four or five times per day, our remote users are disconnected from the MetaFrame. When I ping the remote sites during the drop-outs, there are noticeable delays in packet delivery, and occaisionally, the packets are not returned.

The network equipment, which consists of new Cisco routers and Paradyne FrameSavers, is leased from our network service provider. Traffic consists of IPX and TCP/IP packets. The provider owns the equipment and the boxes are locked down (so to speak).

I have reported the traffic delays to our provider's technical support hotline countless times and have not yet received a satisfactory explanation as to why the problems are occurring. Also, my managers do not consider an oral response from tech. support to be a sufficient basis for taking corrective action. They want to see something on paper before committing to new and/or additional service.

Our provider was supposed to provide us access to a product called Viewspan last September. They told us that Viewspan was supposed to enable us to monitor the frame relay network. We do not yet have access to Viewspan. I have been asking for this for months. (I have also heard that their SLA is supposed to be one of the betters ones on the market.)

Does anyone know if it is possible to monitor the frame relay traffic without disrupting the router and CSU/DSU? I can't reconfigure the router or CSU since I don't have the passwords. Also, I'm an application developer and don't know much about routers and CSU/DSUs.

Any ideas about how to handle this situation would be greatly appreciated! The users are going to lynch me soon, and I'm looking for some good options. Thanks!

 

[wybnormal]

First thing.. PING works with ICMP and if the router is loaded, ICMP is one of the last things answered. I have found traceroute to much more reliable of an indicator to how well the circuit is working. There are several settings in the routers you can look at along with SNMP statistics that you can build some pretty solid information from.

Solarwinds.net offers a 30 day trial of their suite of products, one of which builds a database of response times across links. It uses a form of traceroute and will build a graphic that details the results over time. The tool is Network Monitor. They also have a monitor for interface utilization but it does not log the data.. maybe next release I'm told.

Mike S
"Diplomacy; the art of saying 'nice doggie' till you can find a rock" Wynn Catlin

 

[talisker]

I agree with Mike - check out the Solarwinds tools. In addition to Network Monitor & Bandwidth Monitor, I find Extended Ping extremely useful (although I think they could have come up with a better name!). EP is also easier to use than NM.

VisualRoute is also handy for getting a real-time view of response time & dropped packets.

 

[IXICHostage]

Thanks for the tips, Mike and talisker. The utilities will be a big help!

 

[pavan1]

The packet droputs as I have experienced is generaly due to fluctuating leased lines. You can down load a 30 days trial of WHATSUPGOLD WHICH which will give you a graphical report of the link. Try it out.

 

[Guest_imported]

ViewSPAN does provide Frame Relay & ATM support for all of ICIXs Virtual Circuitry. Contact your sales rep or send an e-mail to vspan.help@intermedia.com requesting more information. Also a product description is available at

http://www.intermedia.com.

 

[clue]

Maybe if you went to the Intermedia home page you would find information about the product ?

try

http://www.intermedia.com/products/data/view_span.html

 

[IXICHostage]

Thank you, IXICHELPDESK and Clue. We signed up for Viewspan last September, and IXIC granted access to us in late April. That's a bit of a gap there. Also, don't ask me what the end-users said to me. Trust me: It doesn't pay to be the messenger in this situation.

Viewspan is a great product, and the reports are indicate that we are using only a small percentage of our bandwidth on an hourly basis. However, the information that we are looking for is probably not contained in the reports. The summaries are hourly, and what we need is a minute-by-minute summary. Are we exceeding bandwidth for a very brief interval? Is this our problem? If I could get an answer to that question, we'd probably sign up for more bandwidth, which is something that IXIC would want us to do.

One other option that we are going to pursue is switching to one protocol, TCP/IP. Nuking IPX/SPX may alleviate some of the symptoms.

Thanks again for your responses!

 

[ITProf]

One thing you may do, if you have read access to the cisco routers:
Look at the following command:
show int <int name number> ex. show int s0.1
This will show you the amount of traffic crossing that interface. Look at the input and output queues and see if they are registering any drops. If they are, then your connection may be getting saturated at certain points.
Also, look at the show frame-relay lmi (not sure if this is the exact command, but you can do a &quot;show ?&quot; to see all of the possible commands.)
This will also show any drops in the LMI protocol (keepalives).

Chris

 

[Poider]

A Cisco router will shut down a frame relay PVC for three reasons, firstly if the signalling on the serial interface goes down (eg. the cable being pulled out), if it stops seeing LMI packets (eg. something wrong with the telco's frame switch) or LMI reporting the DLCI for that PVC is down (somwthing wrong beyond telco's frame switch).

First, do a &quot;debug frame-relay lmi&quot;. If you see LMI going in and out every 10 seconds even when your have an 'outage', then your frame is okay.

Next, do an extended ping and increase the ping timeout to something big (eg. 30 seconds). If you do get pings back but they take a while (such as 3-5 seconds or more), call you telco and tell them off for delivering a bad connection. It usually happens when something inside the telco's network blows up, and they re-route your PVC around the world three times, making your response times unusable. The problem is that their network is doing exactly what it is supposed to do (ie. re-routing) hence they don't see it as a fault.

If you don't get pings back, call the telco and report it as a fault. I've seen problems where the DLCI is up according to LMI, but traffic goes down a black hole. Since the link stays up, your backup interfaces won't necessarily come up.

Also, ping and traceroute and both ICMP so unless you're running Quality-Of-Service and assigning different service levels to ping and traceroute, they will be queued with the same priorities.

Mental Note: Always get an service-level-agreement that includes maximum response times, otherwise you can't complain when you have unusable response times of 5 seconds.

 

[wybnormal]

Correction is needed here.. PING and TRACEROUTE are not BOTH based on ICMP. PING is.. uses ICMP both directions.. Traceroute on the other hand uses UDP on the transmit side and then uses the returned ICMP packet to determine the information about timing etc. The UDP is what will provide the data pathing info and timing. ICMP is just the messenger to return the &quot;error&quot; code which is parsed by traceroute. Traceroute has proven to be a much more reliable tool on a busy network then plain ol' ping.

MikeS
&quot;Diplomacy; the art of saying 'nice doggie' till you can find a rock&quot; Wynn Catlin

 

[Guest_imported]

Since you are using the ViewSPAN application. You should look into the tabular statistics reports. There is a round trip delay calculation in millisecinds, that is based on Layers 1-3 of the IETF Frame Relay protocol. This calculation is performed every 15min by the switching fabric. It is a true real time value for your PVC that is based on the broadband service you purchase.

PING and Traceroute are based on all seven layers of the IP v4 stack and can give you false information about your latency within a service provider's broadband network.

For example - overloaded routers that are bottlenecks, overloaded LAN/MAN segments, defective stacks, stack shiming, IP rings, broadcast storms, and Denial of Service attacks all impact PING and Traceroute results.

 

[jhouser79]

TRACERT is a command in which allows you to follow the path your ping sweep has went through, it will allow you to follow each hop that your ping has to go through, as long as you are able to individualize each hop then you will be okay...

 

[Firestorm]

Check the port utilization. You did not specify who the carrier was but I noticed Sprints FR used to crude out when the demand on the ports exceeded 60%. This was usaully corrected by having Sprint check the circuit.

From what I understand Citrix Metaframe is a bandwidth hog by itself so I would suspect lack of bandwidth for the job.
This is only an uneducated guess since you did not state the size of the pipes or the number of users.

You will need someone to check the ports on the Cisco during a slow down to catch this or have history enabled to verify slowdowns against bandwidth demand.

 

[Guest_imported]

We watch the Denial of Service traffic from code red and other hacker attemps on our circuits with viewspan. Picking the lport where the IP route was breached is like taking candy from a baby.

It is a great tool !

 

[Guest_imported]

We are experiencing the same problem that you were with our Frame Relay. Did you ever get a resolution to your problem?

Dave Schallick
Cincinnati, Ohio

 

[IXICHostage]

Dave, do you have a MetaFrame which is configured for both IPX and TCP/IP and which operates over your frame relay?

 

[Guest_imported]

It must be a Lucent Network. That's why you are having problems.