Processing multiple files at the same time

[vrs1008]

Hello,

I have a question about processing multiple files at the same time.

If the source folder contains

4532_201_271.dat
4532_202_271.dat
4532_203_271.dat

4533_201_271.dat
4533_202_271.dat

I want to create one 271 files for all the files start with 4532 and one for all those start with 4533. How can I do that??

The number generation is dynamic & is generated by othr application. What I want is to club the file with the same batch number (starting number in the file name).



Regards,
Vibhav1008

 

[eyetry]

will you always have a fixed number of files? or will it vary?

Are you just concatenating the data from each file type or does file '_202_' relate to file '_201_' in some way?

Problem you'll have is that you can't have 3 occurances of the same map writing to the same output file at the same time. One of the occurances will have locked the output file.

If you just need to concatenate the input files to a single output file, one option is, change your output card to 'append'.

As far as separating 4532 from 4533 output setup two events triggering off the the different prefixes, appending to related outputs.

 

[janhes]

You can get the file name of the input file and use the relevant part to create the output file appending data if it already exists.

 

[BocaBurger]

Problem might be in the order the output is created. There is no way to tell which input file will be triggered first. You might want to write the data to a database, then use a sort query to correctly order your information.



BocaBurger
<===========================||////////////////|0
The pen is mightier than the sword, but the sword hurts more!

 

[vrs1008]

eyetry (Programmer),

No fixed number of files...It is dynamically generated by other application & it can be any number. I have to take all the files with the same starting number in the file name.

I just want to append these file to create one 271 file with multiple interchange.

Making output card as append won't solve the problem as I want to create another file if the first number in the file name changes.

I am not too sure about the numbers as well so creating 2 event will also not help me.

BocaBurger,

I thought of that approch but I want to make thing simpler as I am receving multiple 271 from the system for one 270. So I want to recreate one 271 in the same order that of 270 out of these multiple 271. I am looking for the alternate approch to dumping all 271 in database & recreating one 271 from the database. I am already storing 270 information in the database for cross referencing purpose.

Can you please suggest the simpler one???




Regards,
Vibhav1008

 

[eyetry]

In the Integration Flow Designer your map should have two events.

Setup first event...

map1(4532):
input-trigger1 = 4532*271.dat
output-target1 = file_name_x (append)

copy/paste event 1 and change your trigger & target criteria.

map1(4532) (copy):
input-trigger1 = 4533*271.dat
output-target1 = file_name_x (append)

Once copied you can also change the component name so its specific to the event, map1(4533)

This will not allow you to multi-process 4532 files or 4533 files but you can process 4532 and 4533 files at the same time and still concatenate your files.

As, Boca stated this is FIFO processing. Should process inthe order the files arrive but not necessarily true.

 

[vrs1008]

evetry,

Thx for the reply but as I explained, the number generation is handled by external application & it is not constant.

today it is
4532*271.dat and 4533*271.dat

but that might be 4534*271.dat / 4535*271.dat / 4540*271.dat....

So I can not set them up in the Integration flow designer.

Regards,
Vibhav1008

 

[eyetry]

sorry, thought you were refering to the middle value

Could you use a control map and overwrite the output file name of the runmap based on the prefix 4532/3/4/5 ?