Karrillion
IS-IT--Management
I have a strange issue which I've done a lot of research and troubleshooting on but so far to no avail.
Certain websites like yahoo.com, ajc.com, and wikipedia.org either come up very slowly or not at all. Yahoo comes up very slowly except there are no pictures or content, only some links. ajc.com (the local Atlanta newpaper) comes up very slowly with pictures and content. wikipedia.org does not come up at all.
The vast majority of websites come up fine, but there are certain websites critical to our business that will not come up or they act like the previous 3 sites I mentioned.
I can traceroute yahoo and ajc.com. When I traceroute wikipedia I get the IP but it times out after a few hops. I cannot ping the IP. So it appears DNS response is fine. I wonder if Yahoo and the AJC are pulling content from servers we cannot connect to.
I can telnet on port 80 to yahoo and ajc but not to wikipedia.
The AT&T Metro Ethernet connection is 20mbps.
Our network consists of a Canoga Perkins 9145 Fiber box (fiber comes in one port and comes out as ethernet on another) connected to a Cisco 2901 router on the Gigabit0/0 interface. A Cisco ASA5510 firewall is connected to the Gigabit0/1 interface on the 2901 router. Cisco stackable switches are connected to the inside interface of the firewall and then the PC's are connected to the switches. We use AT&T Metro E with XO Communications as our ISP.
I took a laptop, assigned the IP info for the firewall to its adapter, and connected it directly to the Gigabit0/1 interface on the router and was still unable to connect properly to those sites.
I then connected the same laptop directly via crossover cable to the Canoga Perkins 9145's ethernet port, assigned the IP address for the router to the laptop's adapter, and tested. All websites came up perfectly.
I contacted Cisco. The support rep and I repeated my tests. Again while connected directly to the fiber box all websites work fine. While connected directly to the inside interface of the router, those websites would not work correctly.
We tried adjusting MTU and the ip tcp mss adjust command. Neither seemed to yield results. We changed the Duplex and Speed to auto/auto on both interfaces, then tried setting them to Full/100 without success.
The router is not using any security features, nor is it doing content filtering. It only bridges the external WAN IP's to the internal public IP's.
Cisco sent me another router. The tech and I configured it and tested but the problem is still there, so it's apparently not a defective router.
One other note of interest: Two weeks ago, everything suddenly worked fine. A couple days later, it went back to having problems. A day later it worked fine again, but then the next day, those sites were again giving us issues. I had done nothing to the router at all.
It seems to point to some strange glitch in the router. I have even changed the cable from the fiber box to the router and saw no improvement. Rebooting or power cycling the router has no effect.
In summary, gentlemen, I am stumped. The Cisco rep is doing some research and will get back to me soon. Unless you or I or he come up with anything I can only determine that the router must have some defect on the quantum level that hates Yahoo and those other websites.
Here's the router config:
PAGEINC#sh run
Building configuration...
Current configuration : 3085 bytes
!
! Last configuration change at 16:41:45 UTC Tue Mar 22 2011 by server
!
version 15.0
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname PAGEINC
!
boot-start-marker
boot-end-marker
!
logging buffered 51200 warnings
!
no aaa new-model
!
!
!
!
no ipv6 cef
ip source-route
ip cef
!
!
!
!
ip domain name yourdomain.com
!
multilink bundle-name authenticated
!
!
!
crypto pki trustpoint TP-self-signed-1030121226
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-1030121226
revocation-check none
rsakeypair TP-self-signed-1030121226
!
!
crypto pki certificate chain TP-self-signed-1030121226
certificate self-signed 01
3082024E 308201B7 A0030201 02020101 300D0609 2A864886 F70D0101 04050030
31312F30 2D060355 04031326 494F532D 53656C66 2D536967 6E65642D 43657274
69666963 6174652D 31303330 31323132 3236301E 170D3131 30323039 30303338
31345A17 0D323030 31303130 30303030 305A3031 312F302D 06035504 03132649
4F532D53 656C662D 5369676E 65642D43 65727469 66696361 74652D31 30333031
32313232 3630819F 300D0609 2A864886 F70D0101 01050003 818D0030 81890281
8100B51C B866B24C B2C1FCB3 61BB5E7F 03590B1D ECC5BD3B 6409BD14 64A01388
9903FC96 4AA68B71 34A1F5B6 0F7843D0 D22C4AD9 07404F88 30F37551 F463FB68
0693AB5A 2D6E9040 76F54BDB CA542868 541D6F7E C1520CA6 D1C22AEB A20E9963
0508E9EF 90107326 99455937 9D9AE8D6 BF865BA4 CC707721 FFB7534B A1D57E90
710F0203 010001A3 76307430 0F060355 1D130101 FF040530 030101FF 30210603
551D1104 1A301882 16504147 45494E43 2E796F75 72646F6D 61696E2E 636F6D30
1F060355 1D230418 30168014 2E037EA4 339861AE 6AB49183 C14110FC B3081DCF
301D0603 551D0E04 1604142E 037EA433 9861AE6A B49183C1 4110FCB3 081DCF30
0D06092A 864886F7 0D010104 05000381 81009FC3 CF867ED9 26C3C8A6 84344E64
68531E81 C7D25F33 35EA927E DB92F7BC 502FDD40 3DEFD52F A8B85313 0068CA72
F03D9A51 BE689633 0D70955B 84563336 7B0E055C 2E79A0C8 83176EA7 8B668E22
DDF20E0E 482F4432 46F8EDFC 240DD581 5FEA3D85 CE222262 D1FA9567 6929450B
F6BFC130 19C39893 62FEFDB3 B2282EE7 0ADD
quit
license udi pid CISCO2901/K9 sn FTX145100M7
!
!
username redacted
!
redundancy
!
!
!
!
!
!
!
!
!
interface GigabitEthernet0/0
description $ETH-LAN$$ETH-SW-LAUNCH$$INTF-INFO-GE 0/0$
ip address 65.46.217.166 255.255.255.252
duplex auto
speed auto
!
!
interface GigabitEthernet0/1
ip address 66.239.221.129 255.255.255.224
duplex auto
speed auto
!
!
ip forward-protocol nd
!
ip http server
ip http access-class 23
ip http authentication local
ip http secure-server
ip http timeout-policy idle 60 life 86400 requests 10000
!
ip route 0.0.0.0 0.0.0.0 65.46.217.165
!
access-list 23 permit 10.10.10.0 0.0.0.7
access-list 23 permit 66.239.221.128 0.0.0.31
!
!
!
!
!
!
control-plane
!
!
!
line con 0
login local
line aux 0
line vty 0 4
access-class 23 in
privilege level 15
login local
transport input telnet ssh
line vty 5 15
access-class 23 in
privilege level 15
login local
transport input telnet ssh
!
scheduler allocate 20000 1000
end
Any ideas?
Certain websites like yahoo.com, ajc.com, and wikipedia.org either come up very slowly or not at all. Yahoo comes up very slowly except there are no pictures or content, only some links. ajc.com (the local Atlanta newpaper) comes up very slowly with pictures and content. wikipedia.org does not come up at all.
The vast majority of websites come up fine, but there are certain websites critical to our business that will not come up or they act like the previous 3 sites I mentioned.
I can traceroute yahoo and ajc.com. When I traceroute wikipedia I get the IP but it times out after a few hops. I cannot ping the IP. So it appears DNS response is fine. I wonder if Yahoo and the AJC are pulling content from servers we cannot connect to.
I can telnet on port 80 to yahoo and ajc but not to wikipedia.
The AT&T Metro Ethernet connection is 20mbps.
Our network consists of a Canoga Perkins 9145 Fiber box (fiber comes in one port and comes out as ethernet on another) connected to a Cisco 2901 router on the Gigabit0/0 interface. A Cisco ASA5510 firewall is connected to the Gigabit0/1 interface on the 2901 router. Cisco stackable switches are connected to the inside interface of the firewall and then the PC's are connected to the switches. We use AT&T Metro E with XO Communications as our ISP.
I took a laptop, assigned the IP info for the firewall to its adapter, and connected it directly to the Gigabit0/1 interface on the router and was still unable to connect properly to those sites.
I then connected the same laptop directly via crossover cable to the Canoga Perkins 9145's ethernet port, assigned the IP address for the router to the laptop's adapter, and tested. All websites came up perfectly.
I contacted Cisco. The support rep and I repeated my tests. Again while connected directly to the fiber box all websites work fine. While connected directly to the inside interface of the router, those websites would not work correctly.
We tried adjusting MTU and the ip tcp mss adjust command. Neither seemed to yield results. We changed the Duplex and Speed to auto/auto on both interfaces, then tried setting them to Full/100 without success.
The router is not using any security features, nor is it doing content filtering. It only bridges the external WAN IP's to the internal public IP's.
Cisco sent me another router. The tech and I configured it and tested but the problem is still there, so it's apparently not a defective router.
One other note of interest: Two weeks ago, everything suddenly worked fine. A couple days later, it went back to having problems. A day later it worked fine again, but then the next day, those sites were again giving us issues. I had done nothing to the router at all.
It seems to point to some strange glitch in the router. I have even changed the cable from the fiber box to the router and saw no improvement. Rebooting or power cycling the router has no effect.
In summary, gentlemen, I am stumped. The Cisco rep is doing some research and will get back to me soon. Unless you or I or he come up with anything I can only determine that the router must have some defect on the quantum level that hates Yahoo and those other websites.
Here's the router config:
PAGEINC#sh run
Building configuration...
Current configuration : 3085 bytes
!
! Last configuration change at 16:41:45 UTC Tue Mar 22 2011 by server
!
version 15.0
service timestamps debug datetime msec
service timestamps log datetime msec
no service password-encryption
!
hostname PAGEINC
!
boot-start-marker
boot-end-marker
!
logging buffered 51200 warnings
!
no aaa new-model
!
!
!
!
no ipv6 cef
ip source-route
ip cef
!
!
!
!
ip domain name yourdomain.com
!
multilink bundle-name authenticated
!
!
!
crypto pki trustpoint TP-self-signed-1030121226
enrollment selfsigned
subject-name cn=IOS-Self-Signed-Certificate-1030121226
revocation-check none
rsakeypair TP-self-signed-1030121226
!
!
crypto pki certificate chain TP-self-signed-1030121226
certificate self-signed 01
3082024E 308201B7 A0030201 02020101 300D0609 2A864886 F70D0101 04050030
31312F30 2D060355 04031326 494F532D 53656C66 2D536967 6E65642D 43657274
69666963 6174652D 31303330 31323132 3236301E 170D3131 30323039 30303338
31345A17 0D323030 31303130 30303030 305A3031 312F302D 06035504 03132649
4F532D53 656C662D 5369676E 65642D43 65727469 66696361 74652D31 30333031
32313232 3630819F 300D0609 2A864886 F70D0101 01050003 818D0030 81890281
8100B51C B866B24C B2C1FCB3 61BB5E7F 03590B1D ECC5BD3B 6409BD14 64A01388
9903FC96 4AA68B71 34A1F5B6 0F7843D0 D22C4AD9 07404F88 30F37551 F463FB68
0693AB5A 2D6E9040 76F54BDB CA542868 541D6F7E C1520CA6 D1C22AEB A20E9963
0508E9EF 90107326 99455937 9D9AE8D6 BF865BA4 CC707721 FFB7534B A1D57E90
710F0203 010001A3 76307430 0F060355 1D130101 FF040530 030101FF 30210603
551D1104 1A301882 16504147 45494E43 2E796F75 72646F6D 61696E2E 636F6D30
1F060355 1D230418 30168014 2E037EA4 339861AE 6AB49183 C14110FC B3081DCF
301D0603 551D0E04 1604142E 037EA433 9861AE6A B49183C1 4110FCB3 081DCF30
0D06092A 864886F7 0D010104 05000381 81009FC3 CF867ED9 26C3C8A6 84344E64
68531E81 C7D25F33 35EA927E DB92F7BC 502FDD40 3DEFD52F A8B85313 0068CA72
F03D9A51 BE689633 0D70955B 84563336 7B0E055C 2E79A0C8 83176EA7 8B668E22
DDF20E0E 482F4432 46F8EDFC 240DD581 5FEA3D85 CE222262 D1FA9567 6929450B
F6BFC130 19C39893 62FEFDB3 B2282EE7 0ADD
quit
license udi pid CISCO2901/K9 sn FTX145100M7
!
!
username redacted
!
redundancy
!
!
!
!
!
!
!
!
!
interface GigabitEthernet0/0
description $ETH-LAN$$ETH-SW-LAUNCH$$INTF-INFO-GE 0/0$
ip address 65.46.217.166 255.255.255.252
duplex auto
speed auto
!
!
interface GigabitEthernet0/1
ip address 66.239.221.129 255.255.255.224
duplex auto
speed auto
!
!
ip forward-protocol nd
!
ip http server
ip http access-class 23
ip http authentication local
ip http secure-server
ip http timeout-policy idle 60 life 86400 requests 10000
!
ip route 0.0.0.0 0.0.0.0 65.46.217.165
!
access-list 23 permit 10.10.10.0 0.0.0.7
access-list 23 permit 66.239.221.128 0.0.0.31
!
!
!
!
!
!
control-plane
!
!
!
line con 0
login local
line aux 0
line vty 0 4
access-class 23 in
privilege level 15
login local
transport input telnet ssh
line vty 5 15
access-class 23 in
privilege level 15
login local
transport input telnet ssh
!
scheduler allocate 20000 1000
end
Any ideas?