Conn thru PIX 501 with static()s with real IPs on both sides breaks

[stoyang]

Hi all,

This is gonna be a long message

We have a PIX 501 used as a firewall to protect three servers. Because there are no technically-skilled persons near the PIX and the servers, we want in case the PIX fails the servers to be reachable again by simply physically removing the PIX from the setup. This imposes the servers having real IPs.

With the setup below, connection to the servers is lost every 4 (four) hours, and can be reestablished remotely by a rather tricky procedure I will also describe below.

I'm not good at ascii art, but:

[gateway]--[pix outside](PIX)[pix inside]
| | |
s1 s2 s3

We have a /28 network, let's say 1.2.3.0/28 . The gateway is 1.2.3.1 . The servers have IPs 1.2.3.2/28, 1.2.3.3/28, 1.2.3.4/28;

PIX's outside interface gets 1.2.3.5 255.255.255.240
PIX's inside interface gets 1.2.3.1 255.255.255.255 (yes, I duplicate the IP of the ISP's gateway)

The default route of the servers is 1.2.3.1, the default route of the PIX is "outside 0 0 1.2.3.1".

The PIX has static mappings:
static (inside,outside) 1.2.3.2 1.2.3.2 netmask 255.255.255.255
static (inside,outside) 1.2.3.3 1.2.3.3 netmask 255.255.255.255
static (inside,outside) 1.2.3.4 1.2.3.4 netmask 255.255.255.255

The access-list is, for the purpose of testing, any any:
access-list acl_out permit tcp any any
access-list acl_out permit udp any any
access-list acl_out permit icmp any any
access-group acl_out in interface outside

Now, the part that is driving me crazy: set up like this, all runs for approx. 4 hours, after which connection to servers behind the PIX is lost. The procedure that brings up all again in fact injects in the ARP cache of the ISP the MAC of the PIX's outside interface for all the addresses of the servers on the inside interface: I log to the PIX through ssh, enable, conf t, ip address outside 1.2.3.2 255.255.255.255 (the connection drops after this, of course), open a new connection, log to the PIX on the new IP address, then switch back: ip address outside 1.2.3.5 255.255.255.255; after this, the first server is visible again, I log to it, then log to the PIX through the serial cable I have attached to server 1, then ip address outside 1.2.3.3 255.255.255.255, ping 1.2.3.3 from outside, then ip address outside 1.2.3.4 255.255.255.255, ping again, then restore PIX's original outside ip. All the servers are visible for another 4 hours, and after that the scenario has to be replayed (no good sleeping in 3 hours steps, at all! :-/ )

I have tried: adding explicitly routes to the inside machines, playing with masks (i.e. the outside /28, the inside gets /32 and vice versa), disabling proxyarp (although I don't know whether the PIX proxies ARP between interfaces, it should not), running pings from the servers to the world and from the world to the servers every two seconds, clearing ARP cache,... all this to no avail.

My theory (I can't prove or reject it, because I can't sniff the traffic between the PIX's outside interface and the gateway of the ISP) is that the ARP cache of the gateway's interface expires, and then the PIX doesn't answer with its MAC address to the ARP queries of the gateway for the IPs for which the former has static() mappings.

Any comments, thoughts and/or solutions will be highly appretiated!

Best Regards,
--sdg

The PIX setup follows:
: Saved
:
PIX Version 6.1(4)
nameif ethernet0 outside security0
nameif ethernet1 inside security100
enable password Phic8nXWfCLR/.bt encrypted
passwd hqASA3FtNZfzgVXS encrypted
hostname pixfirewall
domain-name fakeddomain.com
fixup protocol ftp 21
fixup protocol http 80
fixup protocol h323 1720
fixup protocol rsh 514
fixup protocol rtsp 554
fixup protocol sqlnet 1521
fixup protocol sip 5060
fixup protocol skinny 2000
no fixup protocol smtp 25
names
access-list acl_out permit tcp any any
access-list acl_out permit udp any any
access-list acl_out permit icmp any any
pager lines 25
logging on
logging buffered errors
logging trap errors
logging host inside 1.2.3.2
interface ethernet0 10baset
interface ethernet1 10full
mtu outside 1500
mtu inside 1500
ip address outside 1.2.3.5 255.255.255.240
ip address inside 1.2.3.1 255.255.255.255
ip audit info action alarm
ip audit attack action alarm
pdm logging informational 100
pdm history enable
arp timeout 14400
static (inside,outside) 1.2.3.3 1.2.3.3 netmask 255.255.255.255 0 0
static (inside,outside) 1.2.3.4 1.2.3.4 netmask 255.255.255.255 0 0
static (inside,outside) 1.2.3.2 1.2.3.2 netmask 255.255.255.255 0 0
access-group acl_out in interface outside
route outside 0.0.0.0 0.0.0.0 1.2.3.1 1
route inside 1.2.3.2 255.255.255.255 1.2.3.2 1
route inside 1.2.3.3 255.255.255.255 1.2.3.3 1
route inside 1.2.3.4 255.255.255.255 1.2.3.4 1
timeout xlate 03:00:00
timeout conn 1:00:00 half-closed 0:10:00 udp 0:02:00 rpc 0:10:00 h323 0:05:00 si
p 0:30:00 sip_media 0:02:00
timeout uauth 0:05:00 absolute
aaa-server TACACS+ protocol tacacs+
aaa-server RADIUS protocol radius
http server enable
http 192.168.1.0 255.255.255.0 inside
no snmp-server location
no snmp-server contact
snmp-server community public
no snmp-server enable traps
floodguard enable
no sysopt route dnat
sysopt noproxyarp inside
sysopt noproxyarp outside
telnet timeout 5
ssh 0.0.0.0 0.0.0.0 outside
ssh 0.0.0.0 0.0.0.0 inside
ssh timeout 45
terminal width 80
Cryptochecksum:822c2b66553813a784d31aa6462ccad5
: end

 

[yizhar]

HI.

> sysopt noproxyarp outside
This is probably the cause of the ARP problem.

Someone else here wrote about the KISS rule - Keep It Stupid Simple. I think that is should be applied here.

Use a standard configuration without those tricks, and purchase additional pix501 that will be a cold standby waiting in the box for any problem. Once there is such a problem, the users will need to place the standby pix and reboot the ISP router.

Such a complex configuration that you have is/will cause more problems then it can prevent.

Bye
Yizhar Hurwitz

http://come.to/yizhar

http://teachers.sivan.co.il/yizhar

 

[stoyang]

I know KISS is good, but can't obey the rule in this case :-/ And I doubt there'll be a cold standby PIX available...

Unfortunately enough, when the problem first started (and repeated itself twice) the proxyarp was enabled
(it was ``no sysopt noproxyarp'' for both interfaces).

Regards,
--sdg