1811 Downloads Fail, RST inside window, stray segments?

Alterac · Oct 27, 2009

Im having an issue with alot of my sites with RST inside current window and Stray Segment errors
It is making all applications send alot of retrys and preventing downloads from finishing.
Seems to have something to do with the CBAC of http and https tcp packets.

I have tried various things, and adjusting tcp-mss and such, nothing seems to fix the problem.

Below is all info:

Platform: 1811 Router
IOS: c181x-advipservicesk9-mz.124-15.T9.bin 12.4(15)T9
I have also tried 12.4(22)T, and 12.4(24)T2 both have the same issue.

Config:

Code:

version 12.4
service timestamps debug uptime
service timestamps log datetime
service password-encryption
!
hostname test
!
boot-start-marker
boot-end-marker
!
logging buffered 51200 warnings
enable secret <supersecret>
!
no aaa new-model
!
!
dot11 syslog
!
!
ip cef
no ip dhcp use vrf connected
ip dhcp excluded-address 192.168.38.1 192.168.38.10
ip dhcp excluded-address 192.168.38.240 192.168.38.255
!
ip dhcp pool Office
   network 111.168.38.0 255.255.255.0
   default-router 111.111.38.1
   domain-name test.local
   dns-server 111.111.38.1
!
!
ip domain name BumsteadC.local
ip host Bumstead-S01.BumsteadC.local 10.50.7.100
ip host Bumstead-S02.BumsteadC.local 10.50.7.101
!
ip name-server 208.67.222.222
ip name-server 208.67.220.220
!
no ip port-map ymsgr port tcp 5050 description Yahoo! Instant Messenger
ip port-map user-kaseya port tcp 5721 description Kaseya Data
ip port-map user-yahoo port tcp 5050 description Yahoo IM
!
ip inspect log drop-pkt
ip inspect one-minute high 4000
ip inspect one-minute low 3000
ip inspect tcp reassembly queue length 64
ip inspect tcp reassembly timeout 30
ip inspect name SDM_LOW dns
ip inspect name SDM_LOW ftp
ip inspect name SDM_LOW h323
ip inspect name SDM_LOW https
ip inspect name SDM_LOW icmp
ip inspect name SDM_LOW http
ip inspect name SDM_LOW pop3
ip inspect name SDM_LOW pptp
ip inspect name SDM_LOW esmtp
ip inspect name SDM_LOW ica
ip inspect name SDM_LOW icabrowser
ip inspect name SDM_LOW user-kaseya
ip inspect name SDM_LOW user-yahoo
ip ips notify SDEE
!
multilink bundle-name authenticated
!
!
!
!
crypto isakmp policy 100
 encr aes 256
 authentication pre-share
 group 5
crypto isakmp key <key> address 0.0.0.0 0.0.0.0
crypto isakmp keepalive 30 3
!
!
crypto ipsec transform-set aes256-sha-trans esp-aes 256 esp-sha-hmac
 mode transport
!
crypto ipsec profile plains-ipsec-profile-1
 set transform-set aes256-sha-trans
!
!
archive
 log config
  hidekeys
!
!
ip ssh authentication-retries 2
!
class-map match-any sdm_p2p_kazaa
 match protocol fasttrack
class-map match-any sdm_p2p_edonkey
 match protocol edonkey
class-map match-any sdm_p2p_gnutella
 match protocol gnutella
class-map match-any sdm_p2p_bittorrent
 match protocol bittorrent
!
!
policy-map sdmappfwp2p_SDM_HIGH
 class sdm_p2p_gnutella
   drop
 class sdm_p2p_bittorrent
   drop
 class sdm_p2p_edonkey
   drop
 class sdm_p2p_kazaa
   drop
!
!
!
!
interface Tunnel0
 description DMVPN
 bandwidth 1000
 ip address 172.31.1.38 255.255.255.0
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 ip mtu 1400
 ip nhrp authentication DMVPN_NW
 ip nhrp map 172.31.1.1 <DMVPN Hub1>
 ip nhrp map 172.31.1.2 <DMVPN Hub2>
 ip nhrp map multicast <DMVPN Hub1>
 ip nhrp map multicast <DMVPN Hub2>
 ip nhrp network-id 100000
 ip nhrp holdtime 360
 ip nhrp nhs 172.31.1.1
 ip nhrp nhs 172.31.1.2
 ip tcp adjust-mss 1360
 delay 1000
 tunnel source FastEthernet0
 tunnel mode gre multipoint
 tunnel key <key>
 tunnel protection ipsec profile plains-ipsec-profile-1
!
interface FastEthernet0
 description HIGHSPEED_INET
 no ip dhcp client request dns-nameserver
 ip address x.x.x.110 255.255.255.252
 ip access-group inet_acl in
 ip nat outside
 ip inspect SDM_LOW out
 ip virtual-reassembly
 duplex auto
 speed auto
 service-policy input sdmappfwp2p_SDM_HIGH
 service-policy output sdmappfwp2p_SDM_HIGH
!
interface FastEthernet1
  shutdown
!
interface FastEthernet2
 description OPERATOR_LAN
 spanning-tree portfast
!
interface FastEthernet3
 description OPERATOR_LAN
 spanning-tree portfast
!
interface FastEthernet4
 description OPERATOR_LAN
 spanning-tree portfast
!
interface FastEthernet5
 description OPERATOR_LAN
 spanning-tree portfast
!
interface FastEthernet6
 description OPERATOR_LAN
 spanning-tree portfast
!
interface FastEthernet7
 description OPERATOR_LAN
 spanning-tree portfast
!
interface FastEthernet8
 description OPERATOR_LAN
 spanning-tree portfast
!
interface FastEthernet9
 description OPERATOR_LAN
 spanning-tree portfast
!
interface Vlan1
 ip address 192.168.38.1 255.255.255.0
 no ip redirects
 no ip unreachables
 no ip proxy-arp
 ip nat inside
 ip virtual-reassembly
 ip route-cache flow
!
interface Async1
 no ip address
 encapsulation slip
!
router eigrp 12345
 network 172.31.1.0 0.0.0.255
 network 192.168.38.0
 no auto-summary
!
router rip
 network 10.0.0.0
 network 192.168.38.0
!
ip forward-protocol nd
ip route 0.0.0.0 0.0.0.0 x.x.x.109
!
!
no ip http server
no ip http secure-server
!
ip dns server
ip nat inside source list 150 interface FastEthernet0 overload
ip nat inside source static tcp 10.50.7.100 3389 interface FastEthernet0 3390
!
ip access-list standard snmp
 permit 172.16.164.0 0.0.0.255
!
ip access-list extended inet_acl
 permit gre any any
 permit udp any any eq isakmp
 permit esp any any
 permit tcp 209.148.223.192 0.0.0.7 any eq 3390
 permit tcp host 208.118.96.132 any eq 3390
 permit tcp any any eq 22
 permit udp host 208.67.222.222 eq domain any
 permit udp host 208.67.220.220 eq domain any
 permit icmp any any echo
 permit icmp any any echo-reply
 permit udp any eq bootps any eq bootpc
 deny   ip any any
!
logging trap debugging
logging 172.16.164.10
logging 172.16.164.12
access-list 150 remark No-NAT ACL
access-list 150 deny   ip 192.168.38.0 0.0.0.255 192.168.0.0 0.0.255.255
access-list 150 deny   ip 192.168.38.0 0.0.0.255 10.0.0.0 0.255.255.255
access-list 150 deny   ip 192.168.38.0 0.0.0.255 172.16.0.0 0.0.15.255
access-list 150 permit ip 192.168.38.0 0.0.0.255 any
snmp-server community test RO snmp
snmp-server community test RW snmp
no cdp run
!
!
!
!
!
!
control-plane
!
banner login ^C
-----------------------------------------------------------------------
No Unauthorized Access!
-----------------------------------------------------------------------

^C
!
line con 0
 login local
line 1
 modem InOut
 stopbits 1
 speed 115200
 flowcontrol hardware
line aux 0
line vty 0 4
 privilege level 15
 login local
 transport input ssh
!
end

LOG:

Code:

*Oct 27 16:47:52: %FW-6-DROP_PKT: Dropping Other session 209.148.219.193:443 192.168.38.11:1323  due to  RST inside current window with ip ident 25800 tcpflags 0x5014 seq.no 288975356 ack 3363647737
*Oct 27 16:48:31: %FW-6-DROP_PKT: Dropping Other session 209.148.219.193:443 192.168.38.12:2020  due to  Stray Segment with ip ident 1472 tcpflags 0x5011 seq.no 2686554796 ack 4275837539
*Oct 27 16:49:18: %FW-6-DROP_PKT: Dropping Other session 216.224.252.110:1033 209.148.219.193:443  due to  RST inside current window with ip ident 229 tcpflags 0x5014 seq.no 2387206851 ack

----------------------------------
Bill

North323 · Oct 27, 2009

is this over a vpn? have you looked at pmtu?

http://www.netheaven.com/pmtu.html

Alterac · Oct 27, 2009

Sorry, its traffic that is not over the VPN.

From inside to the internet.

I will remove the dm-vpn completley from my config.

----------------------------------
Bill

North323 · Oct 27, 2009

did this just start happening? when did it start? are you seeing any errors on your router? like ISP issues?

Alterac · Oct 27, 2009

This just started as I was rolling out a new IOS in order to bring up our DMVPN setup. The contractor we had helping us update our core infrastructre said we needed at least 12.4(15)T9 to have the DMVPN be reliable (due to some bug him and cisco figured out in prior versions).

The devices we have that do no display this issue are running: 12.4(6)T9

----------------------------------
Bill

Alterac · Oct 27, 2009

No ISP Issues on the routers, this is happening at multiple sites, including my test lab (should have tested this better.. I guess you live your learn)

Code:

FastEthernet0 is up, line protocol is up
  Hardware is PQ3_TSEC, address is 0027.0d36.ebc8 (bia 0027.0d36.ebc8)
  Description: HIGHSPEED_INET
  Internet address is x.x.x.110/30
  MTU 1500 bytes, BW 100000 Kbit/sec, DLY 100 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation ARPA, loopback not set
  Keepalive set (10 sec)
  Full-duplex, 100Mb/s, 100BaseTX/FX
  ARP type: ARPA, ARP Timeout 04:00:00
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of "show interface" counters never
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: fifo
  Output queue: 0/40 (size/max)
  5 minute input rate 3000 bits/sec, 3 packets/sec
  5 minute output rate 3000 bits/sec, 3 packets/sec
     145058 packets input, 180353887 bytes
     Received 564 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored
     0 watchdog
     0 input packets with dribble condition detected
     92967 packets output, 10005661 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 unknown protocol drops
     0 babbles, 0 late collision, 0 deferred
     0 lost carrier, 0 no carrier
     0 output buffer failures, 0 output buffers swapped out

Since the problem occurs on my test lab router, I can tinker around with it at anytime to try to fix things.

----------------------------------
Bill

Alterac · Oct 28, 2009

Still have not been able to fix this

Anymore ideas?

----------------------------------
Bill

burtsbees · Oct 28, 2009

If you're downloading from Edonkey, Gnutella, bittorrent or Kazaa, then the FW is instructed to drop the packets. The FW is definitely dropping packets.

/

tim@tim-laptop ~ $ sudo apt-get install windows
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Couldn't find package windows...Thank Goodness!

Alterac · Oct 28, 2009

Sorry, should have mentioned.

Downloading virus updates/etc from nai.com and updates from microsoft. Along with files from our HQ.

So standard http/https downloads.

----------------------------------
Bill

Alterac · Oct 29, 2009

Alright, I think i have this solved.

The downloads stalling had nothing to do with the RST or the Stray Segment errors (which are normal with the cbac on, its just doing its job).

The problem with file downloads was the service-policy for blocking on p2p apps, it seemed to be overzealus and break legitimate http downloads, so i removed it for now with plans to troubleshoot more.

Lines Removed:

Code:

service-policy input sdmappfwp2p_SDM_HIGH
service-policy output sdmappfwp2p_SDM_HIGH

I will definatly update this post once I find the exact policy causing the issues.

This brings up a good point to log dropped stuff, would have shown me alot more useful information.

----------------------------------
Bill

Alterac · Oct 30, 2009

So it turns out the blocking of the edonkey protocol was the culprit.

Who woulda known.

Remove the match protocol edonkey and the http downloads worked again.

----------------------------------
Bill

burtsbees · Oct 30, 2009

Your appfw was not even built...

/

tim@tim-laptop ~ $ sudo apt-get install windows
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Couldn't find package windows...Thank Goodness!

Alterac · Oct 30, 2009

Thats what I thought, but aparently just these folowing lines stops most P2P applications by applying the policy map to the external interface.

The appfw isnt even used

class-map match-any sdm_p2p_kazaa
match protocol fasttrack
match protocol kazaa2
class-map match-any sdm_p2p_edonkey
match protocol edonkey
class-map match-any sdm_p2p_gnutella
match protocol gnutella
class-map match-any sdm_p2p_bittorrent
match protocol bittorrent
!
!
policy-map sdmappfwp2p_SDM_HIGH
class sdm_p2p_gnutella
drop
class sdm_p2p_bittorrent
drop
class sdm_p2p_edonkey
drop
class sdm_p2p_kazaa
drop
!
!
interface FastEthernet0
service-policy input sdmappfwp2p_SDM_HIGH
service-policy output sdmappfwp2p_SDM_HIGH
!

----------------------------------
Bill

burtsbees · Oct 30, 2009

Crap---did not see the name of the pm

Sorry.

The name of the policy map applied to the outside interface is appfwblablabla

//

tim@tim-laptop ~ $ sudo apt-get install windows
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Couldn't find package windows...Thank Goodness!

Alterac · Oct 30, 2009

Yea, good point, the naming convention sucks, stupid SDM from along time ago did that one for me.

Since i am removing the edonkey controls, ive been grouping all of them into one p2p_apps class and making a simple p2p_block policy to make it clearer.

Bill

burtsbees · Oct 30, 2009

Google appfw

http://ccietobe.blogspot.com/2009/01/cbac-with-appfw.html

and

http://www.cisco.com/en/US/docs/ios/12_4t/12_4t4/ht_fw_im.html

That is what is confusing---I thought that at one time, there was one of these built, and since deleted, but still applied to the interface. But here, it is a policy map. With the right image, you can create zones and apply the map to a zone rather than just an interface.

http://www.cisco.com/en/US/products/sw/secursw/ps1018/products_tech_note09186a00808bc994.shtml

/

tim@tim-laptop ~ $ sudo apt-get install windows
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Couldn't find package windows...Thank Goodness!

Alterac · Oct 30, 2009

I have used the appfw before to block instant messaging and found that it generated 500-700mb of dns traffic a day!. (this was a while ago, i need to lab it to see wtf is going on there).

Any tips on getting my head wrapped around Zone based instead of CBAC? or is there any "real" reason to use the new fangled stuff?

Bill

burtsbees · Oct 30, 2009

You can get funky and more granular with ZBF. If you understand PBR and QoS, then you're more than 1/2 way there.

Hell, you have a ZBF, without the zones!

/

tim@tim-laptop ~ $ sudo apt-get install windows
Reading package lists... Done
Building dependency tree
Reading state information... Done
E: Couldn't find package windows...Thank Goodness!

edsonc · Nov 23, 2009

Alterac,

If you are still having this issue, please add the following line to your ACL:

tcp permit any any established

Alterac · Nov 23, 2009

The Rsts, and Stray segments proved to just be crappy traffic.

The http downloads were failing due to the edonkey p2p filter we had been using.

You should never need the permit established when you use the CBAC/Zone firewalls. (Unless im wrong here, hah)

Bill

Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

1811 Downloads Fail, RST inside window, stray segments?

Technical User

Technical User

Technical User

Technical User

Technical User

Technical User

Technical User

Programmer

Technical User

Technical User

Technical User

Programmer

Technical User

Programmer

Technical User

Programmer

Technical User

Programmer

MIS

Technical User

Similar threads

Log in

Part and Inventory Search

Sponsor