A better way how to combine content of two files?

[AlesN]

Hi,

I am new to programming and Cobol in particular. I have two files:
1. file defining tag names and order of tags in the final document
2. file containing tag names and values associated with them

These files contain tens of thousands of records identified by ID common for both files. I want to combine the files to get the resulting file containing tagged values from file 2, with tags shown in an order defined in file 1. I came up with this solution:
* filter out tags of interest (based on ID) from file 1 and sort them based on their names into a temporary file (3)
* filter out values from file 2 (based on ID) and sort them based on their names into another file (4)
* merge records from the temporary files (3,4) into yet another file (5)
* sort this last file (5) according to order taken over from file 1 (copied to file 3 to file 5) - the final file (6)

Is there some other method, how to do it, without creating three temporary files? Thanks in advance for suggestions.

 

[mikrom]

Are the files not database files so you can't use Embedded SQL?

 

[nclouston]

If you are running on a mainframe you can use the JOINKEYS function of your sort program. No need to touch COBOL at all.


Nic

 

[AlesN]

Thank you for suggestions!

mikrom - these are CSV files. I thought I could somehow bypass database creation and use original files.

nclouston - unfortunately, I do not have access to mainframe. It runs on Windows.

 

[nclouston]

Sample data for both inputs and expected output from samples woud make things a lot clearer.


Nic

 

[mikrom]

If the data are not in specific COBOL file format and you don't want to load the CSV-data into database, then why do you only want to use COBOL for processing?

CSV files are only simple text files, which could be processed with simpler tools than COBOL.

 

[k5tm]

faq209-5343

Tom Morrison
Consultant

 

[AlesN]

nclouston:
This is an example of what I do:
file 1 - input data (CSV):
[pre]
ID tag output line
1123 Description 3
1123 Something else 2
6745 Other Description 3
1123 Yet another desc 1
[/pre]

file 2 - input data (CSV):
[pre]
ID tag value
1123 Description 55
1123 Something else 356
6745 Other Description 333
1123 Yet another desc 54
[/pre]

file 6 - final output (now COBOL sequential):
[pre]
ID tag output line value
1123 Yet another desc 1 54
1123 Something else 2 356
1123 Description 3 55
[/pre]

I have this behavior already. I filter out and sort wanted records from file 1 to one temporary file (3). I do the same with records from file 2 into another file (4) Then I read files 3 and 4 to create and write records constituting results into another file (5). Finally, I sort this file (5) according to value of the output line field. It works but I am not sure if the approach I chose is optimal. Creating 3 temporary files for each ID in the process seems to me I could do it better.
The final file (6) will be further used by other COBOL programs.

 

[nclouston]

Is there a 1-to-1 relationship of the records between files 1 and 2? If so, are the records in the same sequence ie the first record for id 1123 in file 1 matches the first record for id 1123 in the other file? If so to both questions then it is a simple 2-file match - no need to discard unwanted data. Sort the output into the sequence that you want.


Nic

 

[AlesN]

Nic (ncoulston): no, the sequence in files 1 and 2 is not necessarily in the same order.

 

[nclouston]

Then you do need 2 sorts followed by an extract followed by sort into final sequence. Temporary files are not a problem. after ll, they are deleted when no longer needed i.e. at the end of the total process.


Nic

 

[mikrom]

First i would sort file-1 into file-3 on the ascending 2-field-key=(ID, OUTPUT-LINE) i.e.:
file-3

ID          tag                 output-line
1123        Yet another desc    1
1123        Something else      2
1123        Description         3
6745        Other Description   3

then sort file-2 into file 4 on the ascending 2-field-key=(ID, TAG)
file-4

ID          tag                  value
1123        Description          55
1123        Something else       356
1123        Yet another desc     54
6745        Other Description    333

After that, i would read file-3 record by record and for every record read the value from file-4 using START statement with key=(ID, TAG) and write the resulting record (ID, TAG, OUTPUT-LINE, VALUE) into file-5. Then the resulting file-5 should be already sorted according to (ID, OUTPUT-LINE):

ID          tag                  output line  value
1123        Yet another desc     1            54
1123        Something else       2            356
1123        Description          3            55
6745        Other Description    3            333

 

[AlesN]

nclouston:
Ramdisk! A ramdisk is a suitable solution/workaround for the problem with temporary files. I moved them there. My concern with creating not any more usable files on a permanent device is gone. Thank you all for suggestions.

 

[fredericofonseca]

as you looking to do this in COBOL you should be looking into using its functionality.
more specifically SORT and MERGE

for your case you would need 2 SORT statements and 1 MERGE statement

by using input/output procedures of the above commands you can filter out records that you don't want if that is the case.

and main thing is that it would be doing all ID's in a single pass, not 1 ID at the time

have a look at the following links

old style merging of 2 sorted files - before MERGE was available on COBOL

http://www.mainframegurukul.com/ibmmainframeforums/viewtopic.php?t=6940

sites with some examples for both SORT and MERGE

https://riptutorial.com/cobol/example/24010/merge-regional-data-into-master

following - mainly chapter 9

http://h30266.www3.hpe.com/odl/unix/progtool/cobv28ua/6297/6297_profile_contents.html

Regards

Frederico Fonseca
SysSoft Integrated Ltd

http://www.syssoft-int.com

FAQ219-2884
FAQ181-2886

 

[AlesN]

Frederico, thank you for the documentation links, very valuable. I am aware of the need to do 2 sorts. I was not sure if the approach I chose is correct. It seems it is a valid one. It works as expected.