Delta Script

[stu78]

Hi,

I need to create a script that will figure out the deltas between two files e.g. yesterdays csv and todays csv; and write the delta to a new file... the unique key for the script would be $80 in the file...

So I have something like this as input;
yesterday
name, address, dob....snn
name1, address1,1/1/1...1111
name2, address",2/2/2...2222
name3, address3,3/3/3...3333
name4, address4,4/4/4...4444

Today (a new record was added (ssn 5); ssn 3 changed, and ssn1 was deleted.

today
name, address, dob....snn
name2, address",2/2/2...2222
name3, address3,3/3/3...33333
name4, address4,4/4/4...4444
name5, address5,5/5/5...5555

I need to find the delta between the two files & write to a new one... any examples please?

 

[stu78]

Hi - incorrect post above... reposting!


----------------------------------------------------------------
Hi,

I need to create a script that will figure out the deltas between two files e.g. yesterdays csv and todays csv; and write the delta to a new file... the unique key for the script would be $80 in the file...

So I have something like this as input;
yesterday
name, address, dob....snn
name1, address1,1/1/1...1111
name2, address2,2/2/2...2222
name3, address3,3/3/3...3333
name4, address4,4/4/4...4444

Today (a new record was added (ssn 5); ssn 3 changed, and ssn1 was deleted.

today
name2, address2,2/2/2...2222
name3, address3,3/3/3...33333
name4, address4,4/4/4...4444
name5, address5,5/5/5...5555

I need to find the delta between the two files & write to a new one... any examples please?

 

[PHV]

man diff

Hope This Helps, PH.
FAQ219-2884
FAQ181-2886

 

[stu78]

Diff doesn't really give me what I need... e.g. if the file gets sorted differently in the delta; diff will give me invalid results...

 

[PHV]

How is this issue different than here ?
thread271-1588055

Hope This Helps, PH.
FAQ219-2884
FAQ181-2886

 

[stu78]

This issue with result posted is that it is giving me unstructured output. i.e. a mixture of the old and new data. I only want the adds, the modifys, and the deletes - not the old data in the output... Is this possible?

 

[PHV]

What about something like this ?

awk -F, '
  NR==FNR{o[$1]=$0;next}
  $1 in o{if($0!=o[$1])print "UPD: "$0" vs "o[$1];delete o[$1];next}
  {print "INS: "$0}
  END{for(i in o)print "DEL: "o[i]}
' /path/to/yesterday /path/to/today

Hope This Helps, PH.
FAQ219-2884
FAQ181-2886

 

[stu78]

Almost 100% perfect. How can I make it print only the results from the second file? i.e. the delta file? It is showing me inserts / deletes from both files... I need only to display the results from the second...

 

[Annihilannic]

The output I get from PHV's script and your sample data is the following:

UPD: name3, address3,3/3/3...33333 vs name3, address3,3/3/3...3333
INS: name5, address5,5/5/5...5555
DEL: name1, address1,1/1/1...1111

Those are all changes specific to the second file, today. i.e. a line was updated, a line was inserted, and another was deleted. What do you mean it's showing inserts/deletes from both files? That doesn't make any sense. The simplest thing might be for you to show us what output you expect given that input data.

Annihilannic.