high data transfer rate problem 1

[BigDaz]

hello,

i am working on a LAN with a low traffic rate and i wish to be able to send large amounts of data almost constantly over a tcp/ip connection (between 200k and 800k per second), but only in one direction.
i have writen a basic program (well two) to do this, but i have a problem.
each time data is ready to be sent the "client" program sends the amount of bytes that are to be sent then loops round that many times send 1000 bytes x amount of times (so for 200k, 200 loops) while the "host" loops x amount of time receiving the data. after all the data is read in the "host" sends back an "ok to send" message when the "client" receives this the process starts again.

this works ok, but when left for a time the sending or receiving slows down and each send or receive takes seconds to complete, with the inclusion of a delay (of up to 750 ms) the program runs without fault.

what is wrong with the method of data transfer? what causes the slowing down? is there a better way of doing this?

thanks (for just reading that), BigDaz.

 

[gbiello]

I'll take a stab.

If you're sending data faster than the dest computer can process it, the dest computer could be sending source-quench signals to the sending machine. This would slow down the transfer rate. Try some packet sniffing to confirm this.

-gbiello

 

[pansophic]

Are you closing the socket and opening a new one between transfers? If not, that may solve some of your issues.

In addition, it may be a logic issue in your code. For instance, you may be waiting on a disk write to flush the buffers (which can take a considerable amount of time compared with the data transfer rate).

You may have some success posting a code snippet to one of the programming forums as well. It may turn out that this is not an TCP/IP issue at all.

pansophic

 

[BigDaz]

the "host" waits for a message and the "client", when ready, sends a message containing the about of data it is going to send. the "client" sends the data in 1000 byte "sends", ie for 200,000 bytes it would send 1000 bytes 200 times, the "host" would receive 1000 bytes 200 times, both in a loop statement.

if i were to close the socket and open a new one between transfers, where would the best place to do this be?

the program is quite basic and opens one socket between host and client and does keep it open for both the programs durations.

thanks, bigdaz

 

[pansophic]

That depends on what your goal is. Why are you doing this transfer? It sounds as if it is to create a stress on the network, not to actually transfer a block of data.

But I was thinking that when you finish a series of loops, rather than send an ack and starting a new series of loops that you would send the ack and close the socket. Then establish a new socket for the next series of loops.

I would also ensure that you clear any arrays and buffers, and close any open files so that you don't consume too much memory.

pansophic

 

[BigDaz]

at the moment my client program has a listening thread which waits for a message from the host, when the host asks for some data the client send the required amount back (via the above method [a loop of sends]), the data that the client sends is just random stuff at the moment (but eventually will be important data). i have a thread in the host program that calls the getdata function, waits for it to finish then calls it again.

i've tried closing the socket and establishing a new one between send loops but still, without a delay, the transfers slow down. it is like there is a delay in each send loop, but there isn't

thnaks, bigdaz

 

[pansophic]

What are you doing with the data when you receive it? Are you writing it to a byte array or string array? Or are you writing it to file? If it is one of the above, try to destroy the object (or file stream) and then create a new one. When the arrays get large, they are definitely much slower to write to.

If you are still having trouble with it, try putting in some debugging statements that will give you an idea what is happening when. Don't forget to put a time in debug, preferrably with at least millisecond resolution. It might help you determine where the delay is occurring.

pansophic

 

[BigDaz]

well at the data is writen to a byte array, after each recv the data is rewriten, i don't anything with it yet, but i will do.
the program displays on screen the amount of data that has been sent and ACKed and the time it took to sent it and receive the ACK.

for one loop of send/recv's the program works fine, it's when a series of loops is initiated the sending or receiving slows down unless there is a small delay between each one, even if the socket is closed and a new one established between each loop.

thanks, bigdaz.

 

[pansophic]

Try running a sniffer if you believe that it really a network problem. Maybe we can exchange the dump via email.

It sounds as if it is a problem in the application, rather than a TCP problem, but maybe a capture will answer some of those questions.

You can download Ethereal from

http://www.ethereal.com

if you don't have a sniffer already.

pansophic