Insert directory map into database

[georgeocrawford]

Hi,

My problem is this. On my computer (which can be accessed from my webserver - both machines are running OS X Server), I have a directory, 'Files', with a large number of subdirectories. I would like to recursively scan through the directory, recording the name and path of each file encountered. I would then like to enter the details for each file into a MySQL database in such a way that I can use a php script to graphically display a directory browser in a web page.

'Files' should be scanned periodically (i.e. with a cron job launching the script) and the database updated - preferably only with respect to the changed files (i.e. those moved, added or deleted since last scan).

I can't get my head round how to do this. My thinking so far is to get a php script to invoke a shell script (perl?) to scan the directory, and produce a text file with the resulting directory tree. The php script will the parse the text file, and somehow enter the details in a logical way into the database.

Problems -

1 - I don't know what the fastest method would be for this, or even whether I need a shell script or a text file at all - would php's file functions be as fast as shell? (see discussion at thread434-682722 - How can the search be stripped of all information except that relating to files moved, added or deleted since the last scan?

3 - I can't figure out the best way to record the path of each file in a database. How about two tables - FOLDERS with fields called 'Folder Name', 'Folder id' and 'Parent id', and FILES with fields called 'File name' and 'Parent id'????

My most pressing question at this stage is speed. The files are on an old G3 Mac, so I want the search to be as fast and processor-friendly as possible.

Thanks for all your help!

______________________

George

 

[PHV]

Is this command fast enough ?

find Files -type f -print

I guess the php script will easyly split the directory part and the filename part.

Hope This Help
PH.

 

[georgeocrawford]

not able to check it now, but that's the sort of thing I'm looking for.

questions -

1 - would this be the fastest method?

2 - how can I 'track changes' from the last scan? If I save scans as text files (or in some other format), is there a UNIX/PERL/whatever function which could compare them and find the differences?

______________________

George

 

[PHV]

1) The fastest utility is often a home made C program well suited for the specific need
2) To compare 2 sorted text files you can use the unix

 diff

or

 comm

commands.

Hope This Help
PH.

 

[vgersh99]

2. man comm

vlad
+----------------------------+
| #include<disclaimer.h> |
+----------------------------+

 

[georgeocrawford]

thanks for the help guys!

So, it looks like this:

1 -

find Files -type f -print

and save output to text file of name 'scan2.txt'.


2 -

comm -3 scan1.txt scan2.txt

where 'scan 1.txt' is yesterday's scan.


3 - import results into php as two strings - $string1 is the paths of all the deleted files (i.e. those that were only in scan1.txt) and $string2 is the added files (only in scan2.txt)

Should I do this as a C script (if so - help - I have ABSOLUTELY no idea how to do this), or execute each line as a shell command from within a php script (might be safer, then I'm only dealing with PHP and UNIX!) If the latter, can someone please tell me how to save the output of

find Files -type f -print

in a text file using UNIX? Also if I did it this way, maybe it's better to do this(PHP code now!):

$deleted = `comm -23 scan1.txt scan2.txt`;
$added = `comm -13 scan1.txt scan2.txt`;

better?

______________________

George

 

[PHV]

To save standard output to a file:

find Files -type f -print | sort >/path/to/scan.txt

As you'll use comm, you have to sort the file.


Hope This Help
PH.

 

[georgeocrawford]

Thanks - I'm really learning here! Just been experimenting with pipelines and the

<

and

>

symbols.

So here goes:

$shell = `find /path/to/files_directory -type f -print | sort > /path/to/new.txt`;

$deleted = `comm -23 old.txt new.txt`;
$added = `comm -13 old.txt new.txt`;

$shell = `mv -f /path/to/new.txt /path/to/old.txt`;

how's that?

______________________

George