8 data streams into one

[BigDaz]

hello,

i have a problem where i have data coming in from 8 eight serial ports and i want to get that data and put into one data stream to be sent via tcp/ip over ethernet.
please could suggest/outline a possible solution for this?
each serial port and the tcp/ip link are handled in their own thread.

cheers, BigDaz

 

[obislavu]

Where is data put from the 8 threads ?
Do you transform the received data before sending over ethernet ?
Add a new thread that will consume the data received by the 8 threads e.g reading/sending over ethernet.
-obislavu-

 

[teriviret]

BigDaz, what you describe is basically a muliplexer or serializer -- something that takes input from multiple input streams and combines them together into one output stream.

Start by having one dedicated thread that is the only one allowed to access the tcp/ip data stream (I think this is what obislavu is refering to). It should wait for any input from the 8 serial ports, and if any data comes in, it should tag that data with some kind of token or identifier, so you know which serial port it came from. Then ship the data back out to the tcp/ip stream.

One good way for the tcp/ip stream thread to wait on the multiple input streams is to use the function WaitforMultipleObjects(). You'll have to represent each serial port with a waitable object -- a handle to an event, thread, or maybe just the serial port file handle itself.

Since you have other threads already handling the serial ports, synchronize them with the tcp/ip stream thread. Have a mutex or semaphore which each thread must acquire before passing data to the tcp/ip stream thread. Pass the data by copying it into a buffer that the tcp/ip stream thread can also access. You might want to have a different buffer for each serial port. Then set some kind of waitable object associated with that buffer. This could be an event object for example. The event objects for each of the 8 serial ports would be what the tcp/ip stream thread waits on in its call to WaitForMultipleObjects().

The token added by the tcp/ip stream thread to the data from each serial port thread is used by the other end of the tcp/ip stream to separate the chunks of data coming across, to reconstruct the 8 different original input streams.

The token plus the data it goes with is a "packet" since the token goes with just that data and not the data that comes before or after it. However, since TCP is a byte-oriented stream, you need some way to delimit your packets so the other side of the stream can locate where the tokens are, and know how to match up the tokens with the data. To do this you could add a "length" byte or word to each packet.

So here's an example showing how data coming in from 3 different serial ports would get multiplexed together and how you'd add length and token fields to do the multiplexing:

[tt]
input serial port 1: 55 AA 55 00 78
input serial port 2: 00 00 07
input serial port 3: 88 99 AA BB

output tcp/ip stream:

01 05 55 AA 55 00 78 02 03 00 00 07 03 04 88 99 AA BB

Let's break that apart to see what's what:

01 <-- token indicating next chunk of data goes with serial port 1
05 <-- length of this chunk is 5 bytes
55 AA 55 00 78 <-- data for this chunk

Next packet:

02 <-- token indicating next chunk of data goes with serial port 2
03 <-- length of this chunk is 3 bytes
00 00 07 <-- data for this chunk

Next packet:

03 <-- token indicating next chunk of data goes with serial port 3
04 <-- length of this chunk is 3 bytes
88 99 AA BB <-- data for this chunk
[/tt]

 

[BigDaz]

hello,

thank you for the suggestions, the data has it's own header and i'd like to keep these together for transmission. what i'd like to is have like a job queue in the tcp/ip transmission thread which the 8 serial threads add to as data comes in with a shared buffer between the tcp/ip and serial thread (for each of the 8) but i'm unsure how to implement it. the data comes in fast up to 6 MBaud accross the ports so maybe i need a machanism to drop the least important data jobs if things get busy.

cheers, bigdaz