Tricky WINHttp question (maybe not tricky for some)

[tedsmith]

My customer has a setup for downloading (via Web) a small amount every 15 seconds in three different batches of 40kb each.
Currently I am using WinHTTP in sync mode.
This waits until the web servers have done it's connection (varies between 1 and 3 secs each batch) a maximum of 9 seconds in total. The delay is in the internet connecting not the data speed because the size of the file makes hardly any difference.

The reason I used sync was I don't want to have the rest of the program running because I want all three to be fully received before I proceed to process the data before the next batches are received.
Its the equivalent to sending first names in the first batch, second in the second and addresses in the third. I need them all every time to match up

I am trying to cut down the overall download time.

My customers advises his server will handle all three requests without delay (as it is also servicing a number of other people) and so cut down the wait by one third so -

If I sent the first 2 as async and the third as sync, if the second one took longer the end of it could be missed.

If I used async for all three batches and they arrived 'together' how could I tell that all had been fully received before I started to process them. The time of each bus is not related to it's size but random 'happenings' in the internet beyond out control.

If the downloads do happen virtually at once how would I know batch which was which and put them into their correct 'slot' for processing?

 

[strongm]

>how could I tell that all had been fully received

They each raise an OnResponseFinished event

 

[strongm]

Resolved?

 

[tedsmith]

Not yet. I have gone with the sync approach as it is more convenient to the rest of the app to know when each batch has been received (even if it does take longer)

 

[strongm]

Er ... the point is you CAN know when the entire batch has been received asynchronously.

 

[strongm]

Decided to stick with the 'sync' approach then?

 

[tedsmith]

Yes.
The sync turns out to be the most convenient because the further processing of each batch really doesn't want to start until the whole of each batch is received.
If I used async and the processing every approx 15 seconds started halfway through a download I would have to wait another 14.99 or so seconds and the batch data would be that much later still.

I didn't want to use data received to start the processing like an interrupt driven routine in case data was temporarily interrupted for any external reason.
It has to happen every 15 seconds and uses previous data if the data is old.

The system is a passenger information system for buses feeding 72 remote bus stops throughout our city every 15 seconds.
The data is GPS sightings of buses but the scheduled timetable has to be used if GPS is not available for any approaching bus. Some bus stops have Led Signs and others have computers with multiple screens showing graphics or movies as well.

I do it all with Recordsets from 3 joined tables, one the scheduled times, another the Destination names and another a temporary table made every 15 secs from the GPS data - quite involved as you can imagine! Luckily all queries only take about 5 milliseconds to execute.

 

[strongm]

>until the whole of each batch is received.

Fair enough - but you still seem to be missing my point, which is that you can raise an event when the entire batch has been received as per your original question ("could I tell that all had been fully received before I started to process them?") negating any download interruption problem.

Frankly what I'd do is set a (probably timer-driven) loop calling for data from the buses to work continuously in the background, and use the the reception event for each complete dataset to populate a 'last known position' variable for each bus.

Then I'd have a separate timer triggering every 15 seconds that used the 'last known position' data that we'd already retrieved.

Still, as long as you've got a solution that you are happy with (even though it seems odd that you are using a procedural, sequential solution for what is clearly an event-based problem)

 

[tedsmith]

Thanks for your interest but it's not actually event based by the time the data gets to me.

I have no control of the acquisition and conversion of the GPS data. This is derived from a smartcard ticketing system operated by a private company (based in the USA). The GPS is primarily used for adjusting the fare cost with distance a passenger travels and debits it to his smartcard when the passenger disembarks the bus.

What I have to do is request the estimated departure time from each bus stop in 'batches' of max 40 bus stops of my choosing. These are when any bus is due within 15 minutes at any of the 72 bus stops.

So the 'events' have to be initiated by me, not as a result of the bus moving from one location to another.
I have no way of knowing when a new location has been sensed so I think all I can do is regularly poll the data to see if there has been any change.

The reason for my original question in this thread was, because my query has to go from Australia to USA to be processed and back it was taking up to 5 seconds to receive it every 15 seconds so I had to wait before I could send it out to any station.

I then realised I couldn't do anything anyway in this time so I may as well wait and be sure I got everything!

The main delay is not in the internet but rather in the time the USA super computer returns the info requested as it is also driving a massive smartphone info system for the public.

Ironically the system is working very well and much more accurate than any other bus passenger info systems I have seen in other cities around the world. Another success story for Tek-Tips (and strongm!)