Searching for duplicate files 1

[rossco100]

Hi, im trying to write a script that searches for identical file names and file sizes , i know that this can be done using the -c option but i am not quite sure how to start it, any help would be great thanks.

 

[KenCunningham]

The -c option to what?

 

[rossco100]

Im trying to write a script that searches a filesystem to find all duplicate files and then provides an output.
Any help greatly appreciated.

Thanks

 

[PHV]

And what have you tried so far ?

Hope This Helps, PH.
Want to get great answers to your Tek-Tips questions? Have a look at FAQ219-2884 or FAQ181-2886

 

[futurelet]

ruby common-files.rb dir1 dir2

common-files.rb:

##  Find files common to two directories.
##  Descend into subdirectories.
lists = [ Hash.new { [] }, Hash.new { [] } ]
for i in 0 .. 1
  Dir[ ARGV[i] + "/**/*" ].each{|path|
    lists[i][
      [ File.basename(path), File.size(path) ]
    ] += [ path ]  if File.file?( path )
  }
end

lists.first.each_key { |key|
  if lists.last.include?( key )
    puts "---  #{ key[0,2].join('  ') } bytes"
    puts lists.first[ key ], lists.last[ key ]
  end 
}

 

[p5wizard]

Try this:

find . -type f -exec sum {} \;|\
 sort|\
 awk '{
 if (lsum == $1) {
  if (lfile != "" ) {
   print lfile
   lfile=""
   blank="yes"
  }
  print $3
 }
 else {
  if (blank == "yes") {
   print ""
  }
  lsum=$1
  lfile=$3
 }
}'

HTH,

p5wizard

 

[p5wizard]

You can check for the identical number of blocks in addition to the checksum value but I don't think it's absolutely necessary:

find . -type f -exec sum {} \;|\
 sort|\
 awk '{
 if ((lsum == $1) && (lblks == $2)){
  if (lfile != "" ) {
   print lfile
   lfile=""
   blank="yes"
  }
  print $3
 }
 else {
  if (blank == "yes") {
   print ""
  }
  lsum=$1
  lblks=$2
  lfile=$3
 }
}'

HTH,

p5wizard