Capturing fleeting files 2

[ddrillich]

Good Day,

An application places files under a certain directory - /appl/spool/esas/IDOLServer10/IDOL/status, processes them and then deletes them.
I think the files end up being in this directory for a very short time especially when the application rejects them.
We would like to capture these files. Is there a way to "listen" to this directory and copy out any file that is being placed there?

Regards,
Dan

 

[SamBones]

It depends on how fleeting they are. And copying is too slow. What you need to do is create a hard link to the file. This makes two directory entries for it. When the other file name is deleted, the link you created will remain with the file's contents intact.

Something like this maybe...

#!/bin/ksh

# Note: SAVE_DIR MUST  be on the same physical file system or the hard link will fail.
# Create a SAVE_DIR that you can write to on the same file system.

WATCH_DIR=/appl/spool/esas/IDOLServer10/IDOL/status
SAVE_DIR=/appl/spool/esas/IDOLServer10/IDOL/save

# 300 seconds equals five minutes
COUNT=300

print "Collecting status files"
print "Watching: ${WATCH_DIR}"
print "Saving:   ${SAVE_DIR}"

while (( COUNT ))
do
    ls -1 ${WATCH_DIR} | while read FILE
    do
        ln ${FILE} ${SAVE_DIR}/$(basename ${FILE}).saved 2>/dev/null && print "Saved: ${FILE}"
    done
    sleep 1
    (( COUNT -= 1 ))
done

print "Captured files"
ls -l ${SAVE_DIR}

This will check every second for files in that directory. It assumes there are no other files just sitting there. You can add a file name pattern to limit what it catches.

Keep in mind that if the file name used is used again, you will only get the first one. If each file name is different, you will get them as long as it exists long enough for the script to catch it. You can capture same named files with a small modification.

If the files are shorter lived than one second, you can remove the sleep, but just be warned that the COUNT will need to be increased greatly, and the script will eat CPU big time since it will be looping like crazy.

Hope this helps.

 

[ddrillich]

Beautiful thing SamBones!!!

Thank You,
Dan