Hi,
I have a DS problem I am hoping someone might be able to shed some light on.
BACKGROUND:
I have a DS job (A) which calls 2 other DS jobs (B,C). B and C
are jobs which read the contents of 2 designated (and seperate) folders (folders "1" and "2"
on the network. Both jobs include a sequential file stage, one transform and 1 output link to an oracle table. The contents of folder "1" are comma-delimited text files with 5 columns, whereas the contents of folder "2" are comma-delimited text files and csv's with 8 columns.
The PROBLEM:
The problem I'm having is that when DS encounters a text file in folder "1" which has too many columns, or trailing commas or some other imperfection, the entire Job "A" aborts -without continuing to process the rest of the files in folder "1" or in folder "2".
Or even when DS encounters duplicates (ie. records in that text file which match records already existing in the oracle table being outputted to), Job "A" does not abort but instead of continuing to process all the text files in folder "1", and then all the files in folder "2", it just stops.
QUESTION:
How can I get DS to continue processing other files after it encounters a bad one and rejects it. Similarly how can I get DS to continue processing other files after it encounters duplicates?
Any help is greatly appreciated!
I have a DS problem I am hoping someone might be able to shed some light on.
BACKGROUND:
I have a DS job (A) which calls 2 other DS jobs (B,C). B and C
are jobs which read the contents of 2 designated (and seperate) folders (folders "1" and "2"
on the network. Both jobs include a sequential file stage, one transform and 1 output link to an oracle table. The contents of folder "1" are comma-delimited text files with 5 columns, whereas the contents of folder "2" are comma-delimited text files and csv's with 8 columns.The PROBLEM:
The problem I'm having is that when DS encounters a text file in folder "1" which has too many columns, or trailing commas or some other imperfection, the entire Job "A" aborts -without continuing to process the rest of the files in folder "1" or in folder "2".
Or even when DS encounters duplicates (ie. records in that text file which match records already existing in the oracle table being outputted to), Job "A" does not abort but instead of continuing to process all the text files in folder "1", and then all the files in folder "2", it just stops.
QUESTION:
How can I get DS to continue processing other files after it encounters a bad one and rejects it. Similarly how can I get DS to continue processing other files after it encounters duplicates?
Any help is greatly appreciated!