Unicast ARP requests

[jdavis1]

Recently a new W2K3 file server was deployed on a LAN that I manage. Since the deployment, users on the LAN have complained of poor performance when using services on the new server. I mirrored the port and discovered that the new server was sending a massive number of ARP requests Periodically (every 10 seconds or so) the server will flood the network with 1000 - 5000 ARP requests. Furthermore, these are unicast ARP requests, not broadcasts as you would expect. The ARPs seem to addressed at the L2 (MAC) level to the device that is assigned the IP address contained in the request.

Has anyone encountered anything like this? I suspect that it is some setting in the OS, or some wierd MS "management" tool, but it could also be malicious software I suppose. I have no access to the server. The effect is that fully 25% of the ethernet frames on this highly active ethernet port are ARP frames - not the fraction of a percentage that I would normally expect.

Thanks,
-Jeff

 

[gdvissch]

I'm a little puzzled here but could you send a capture so we can have a look at it?

I'm thinking of two things some sort of discovery process (e.g. the server looking for printers, which would be linked to a management software)
or some sort of software that tries to poison the arp caches on a single machine or all the machines of your network in order to be able to sniff the traffic.
Arp cache poisoning is commonly used in a switched environment where you normally see only broadcast traffic or packets destined to you. Using arp cache poisoning, you tell all the hosts on your network that certain IP's (the targets your sniffing) have your MAC address effectively re-routing the traffic to the attacking machine. This way it can also see unicast traffic from the target host. After receiving the packet, the attacker bridges it to the real host so no-one notices a difference ...
Another way of achieving this is flooding all CAM tables in the switches by sending ARP floods. When the switch is effectively flooded with ARPs it falls back to another mode where it starts to forward all packets to all ports and thus becomiong a hub. Again the point is to be able to sniff traffic destined to other hosts in a switched environment.
Anyhow, I would check the process list of the server and identify all unusual process...

G.

 

[jdavis1]

G,

Thanks for the reply.

This doesn't appear to be any sort of arp cache poisoning technique. The unicast arp packets contain the correct source MAC (both in the header and arp payload) and IP. As I said, the destination MAC corresponds to the machine that is assigned the IP address contained in the request. I too suspect that this is related to some sort of management software, or some option in the W2K3 IP stack. A long-term capture (several hours) revealed that arps comprised around 25% of the total # of packets seen on the interface that this fairly busy file server is connected to. Unfortuanately, I don't have access to the machine to check the processes or event logs. I have sent the traces to the group that manages the server and their networking staff, and they are convinced that this is normal arp traffic. Go figure.

I'm not sure how I post a capture to this board, but here are a few of the arp packets from the trace. The addresses have been changed to protect the innocent. Detail on the first frame (frame 4) follows the summary.

Frame Source Destination Summary Bytes Delta Time
----------------------------------------------------------------------------------------------------------------------
4 00145EB41234 IBM F2206C ARP: C PA=[192.168.10.84] PRO=IP 64 0.295.578
5 00145EB41234 001438886462 ARP: C PA=[192.168.15.108] PRO=IP 64 0.000.051
6 00145EB41234 JapanC0D4EF4 ARP: C PA=[192.168.15.109] PRO=IP 64 0.000.005
7 00145EB41234 IBM 16FE78 ARP: C PA=[192.168.10.117] PRO=IP 64 0.000.008
8 IBM F2206C 00145EB41234 ARP: R PA=[192.168.10.84] HA=IBM F2206C PRO=IP 60 0.000.070
9 IBM 16FE78 00145EB41234 ARP: R PA=[192.168.10.117] HA=IBM 16FE78 PRO=IP 60 0.000.041
10 001438886462 00145EB41234 ARP: R PA=[192.168.15.108] HA=001438886462 PRO=IP 64 0.000.077
11 00145EB41234 001125BB300E ARP: C PA=[192.168.11.140] PRO=IP 64 0.000.005
12 00145EB41234 3Com B8903D ARP: C PA=[192.168.11.141] PRO=IP 64 0.000.007
13 001125BB300E 00145EB41234 ARP: R PA=[192.168.11.140] HA=001125BB300E PRO=IP 60 0.000.091
14 3Com B8903D 00145EB41234 ARP: R PA=[192.168.11.141] HA=3Com B8903D PRO=IP 60 0.000.005
15 00145EB41234 0014385C3B6A ARP: C PA=[192.168.15.165] PRO=IP 64 0.000.043
16 00145EB41234 001321C3BDAF ARP: C PA=[192.168.15.166] PRO=IP 64 0.000.005
17 JapanC0D4EF4 00145EB41234 ARP: R PA=[192.168.15.109] HA=JapanC0D4EF4 PRO=IP 60 0.000.005
18 00145EB41234 001185D82934 ARP: C PA=[192.168.15.177] PRO=IP 64 0.000.063

- - - - - - - - - - - - - - - - - - - - Frame 4 - - - - - - - - - - - - - - - - -
DLC: ----- DLC Header -----
DLC:
DLC: Frame 4 arrived at 11:08:30.5528; frame size is 64 (0040 hex) bytes.
DLC: Destination = Station IBM F2206C
DLC: Source = Station 00145EB41234
DLC: Ethertype = 0806 (ARP)
DLC:
ARP: ----- ARP/RARP frame -----
ARP:
ARP: Hardware type = 1 (10Mb Ethernet)
ARP: Protocol type = 0800 (IP)
ARP: Length of hardware address = 6 bytes
ARP: Length of protocol address = 4 bytes
ARP: Opcode 1 (ARP request)
ARP: Sender's hardware address = 00145EB41234
ARP: Sender's protocol address = [192.168.8.18]
ARP: Target hardware address = 00096BF2206C
ARP: Target protocol address = [192.168.10.84]
ARP:
ARP:
DLC: Frame padding= 22 bytes

 

[jdavis1]

Additional information.

I did a little further analysis on the capture. First, I checked the time between burst of unicast arps from the server. They seem to be somewhat random, but definately seem to be controlled by a timer. Times between bursts are 2.5 seconds, 5 seconds, 9 seconds, 10 seconds, etc. Not 5.1 seconds, or 9.3 seconds.

Further, I checked to see if there was any host sending arps with a protocol address equal to the server's IP address, thinking that maybe the arp bursts were an attempt to guard against a duplicate IP. I found none of these. I see no IP packets originating from a host other than the server with a source address equal to that of the server's address.

Finally, I checked the destination hosts to see if there was any commonality. Destination devices are a heterogeneous group of printers, workstations, etc.