Apache webserver suddenly stops answering, doesn't crash

[sitecrusher]

Hi,

I'm doing some tests on Apache webserver (both 1.3 and 2.0 suffer from this problem) on Linux Debian 2.4 with enough CPU and enough RAM.
The load linear raises upto 8000 transactions/sec (this is 250 Mbps) and then suddenly falls down to 10 transactions/sec. At the time the server gets 8000 transactions/sec the CPU is almost 100% used (50% in kernel and 50% in user space), it then starts to spawn lots of child-processes and the CPU becomes idle !
It seems like there's no work done anymore, there are more than maxSpareServers not working, but they aren't cleaned up until the load disappears, no more requests and then one-by-one the child-processes are killed...
I don't get any logs that can help me out here.
Anyone familiar with this problem ?

Thanks a lot in advance

 

[gchen]

It sounds like your box got trajon ( you might want to re-boot the box and see if it goes away. then ask some tarjon pro to help. Good luck!

 

[sitecrusher]

Hi

I made a trace with tcpdump and here are my conclusions:
Because of the CPU going to 100% busy and there are a lot of new users/sec, the server can't answer all those incoming requests and goes into a state of just receiving the new SYN's, but not answering them ==> SYN flood attack.
I think i'll have to look into the linux kernel parameters to find some prevention for DoS attacks and tuning those parameters...

 

[danomac]

If it is a SYN flood attack, enabling syncookies in the kernel will help.

 

[Deleted]

Adjust the compile time parameters (ex: maxusers)

% sysctl -a | grep maxusers

http://unixjunky.com