Combining multiple data sets - some missing

[nielsyding]

Hi.
I have about 45000+ small data sets that I need to combine into one data set. I have tried using the macro code below. The problem is that there's about 50 data sets that are missing and when for instance data set "parm400" is missing the code fails and the "Samletparm" data set contains no observations.
Is there a way to tell SAS to continue joining the data sets even if there's some missing in between?


%macro names(prefix,maxnum);
%do i=1 %to &maxnum;
&prefix&i
%end;
;
%mend names;

data eq.SamletParm;
set %names(eq.parm,46080);
run;

Thanks in advance.

 

[klaz2002]

My first response is to use the PROC APPEND instead of the data-set statements. The proc will write an ERROR to log but will continue.
My second response is, even using proc append you can check if the dataset exists before you call your proc append code.
ex

%macro combine_names(target_name,prefix,maxnum);
   %do i=1 %to &maxnum;
     %if %sysfunc(exist(&prefix&i)) %then %do;
        proc append 
          base = target_name
          data = &prefix&i force;
        run;
   %end;
%mend combine_names;
%combine_name(eq.SamletParm,eq.parm,46080);

I changed the name of the macro but I think you can use it the way it is.

Klaz

 

[kdt82]

If you have SAS 9.2, and your datasets follow a named range (as your example suggests) then you can use the colon modifier in the set statement.

*generate dummy datasets ;
data class1 class3 class4 ;
   set sashelp.class ;
   run ;

data all ;
   set class: ;*colon modifier to set datasets with same prefix;
   run;

 

[klaz2002]

kdt82,
Thats true, but if one of the named range datasets are not found that datastep will fail.
Klaz

 

[kdt82]

klaz2002,

I agree with you for normal named ranges, i.e name1-nameN. But the prefix modifier example I gave demonstrates that as long as the prefix is the same, it will concatenate all datasets that meet that criteria (the class2 dataset does not exist).

 

[klaz2002]

Agreed. The CLASS method would work for known named ranges. But for all other types and if you don't have SAS 9.2 (my current new place does not have it yet) you would need to implement a solution as I have laid out.

Thanks for the CLASS idea though, I will use it in the future.

Klaz