Random vs Sequential read - time diffs

[pentegr]

Anyone got an idea of the typical time difference between a cobol sequential or a random read of a single record. Prefereably a referance in generally accepted documentation

I'm looking for a ratio rather than an absolute time which would obviously vary from platform to platform.

I'd like some evidence to support the type of file access to use based on the hit rate on a particular file. I've always just used a thumb-suck 30% value in the past but would like to have some hard facts to support decisions

Thanks in advance

 

[webrabbit]

It makes a big difference on what kind of file you're talking about.

 

[tcsbiz]

30% of what? Is a sequential read 30% faster than a random read?

It depends. If the data is in physical order on the disk by a key value and you read the file sequentially, that could be faster if the record you will pick is close to where you start reading. A sequential read can be faster if you read a large number of records and they are in proximity to each other.

A random read of a record is faster if the data isn't physically arranged by key.

The number of records plays into it. If we're talking 10,000 records versus 1,000,000,000 records, the physical storage and access method makes a big difference.

In general, if you want just one record, access it randomly. If you need to retrieve a large number of records at a time, arrange the data properly and read sequentially.

I doubt you'll find any written documentation stating one way or the other.

 

[pentegr]

Thanks for your replies so far guys, maybe I hadn’t phrased my question clearly enough so a brief example may help clear up what I’m looking for

Say I have a file of one million records. It is a VSAM file sorted in key sequence. I don’t think the storage medium makes that much difference in this case, as no matter how the file is accessed, the medium will be the same, but for the sake of clarity let’s just say the file is stored on DASD. The file will always be pretty big (in the order of 100’s of thousands of records).

At what point would it make more sense (from an efficiency perspective – hence my reference to hit rate) to read only the necessary records randomly vs reading through the entire file sequentially?

Obviously blocking factors have an influence, however…

…a single random read takes longer than a single sequential read … so for the specific program if the hit rate on the file was 50% would the program run faster if random access was used, or sequential access? At what point does one make this distinction?

So far I haven’t been able to come across any benchmarking or documentation comparing the two access methods other than the “30% hit rate rule-of-thumb” mentioned above.

 

[tcsbiz]

Say you're reading 100,000 records out of 1,000,000. Further say that those records are in physical sequence in the order that you need. Setting the file at the first record you need and reading sequentially would be much faster than reading each one randomly.

Now say that the records you need are stored physically in a random order. Reading sequentially would require you to potentially read the 1,000,000 records 100,000 times to get each record. Reading the records randomly would be faster.

I think you have to run some benchmark tests in your environment to find the real differences.



Good luck. This is one of those issues that have been debated forever and i don't think there is one correct answer.

 

[pentegr]

Tx tcsbiz

!!!! I was hoping there'd be a short answer to this ... I guess there never really is though

 

[tcsbiz]

You're welcome.

The short answer is "It Depends". I think I've used those two words more than any other in my career.