I'm using wget to perform a daily incremental backup of the files on a remote webserver. Recently, those backups have been causing the remote server's firewall to block my IP. The hosting provider thinks the block is occuring because the firewall only allows three concurrent connections from an IP and blocks the IP when more than three are attempted.
I don't think wget creates multiple concurrent connections, so my working theory is that a connection error occurs during the wget session, and wget attempts to reconnect to the server while the server is still holding the previous, now defunct, connection open waiting for it to timeout. From the server's point of view, I now have two connections open to my IP, and if another connection error occurs before the first session times out on the server, I've got three connections open and have triggered the IP block. I could be completely wrong, but that's the only theory I can come up with at the moment.
Does anyone have any suggestions how I might avoid this situation? I've checked the wget docs for a "wait before reconnect" switch, hoping I could make wget wait a couple minutes between connection attempts, but have found nothing.
Any ideas or suggestions would be much appreciated.
I don't think wget creates multiple concurrent connections, so my working theory is that a connection error occurs during the wget session, and wget attempts to reconnect to the server while the server is still holding the previous, now defunct, connection open waiting for it to timeout. From the server's point of view, I now have two connections open to my IP, and if another connection error occurs before the first session times out on the server, I've got three connections open and have triggered the IP block. I could be completely wrong, but that's the only theory I can come up with at the moment.
Does anyone have any suggestions how I might avoid this situation? I've checked the wget docs for a "wait before reconnect" switch, hoping I could make wget wait a couple minutes between connection attempts, but have found nothing.
Any ideas or suggestions would be much appreciated.