directory status after removal of files 1

[Calator]

we run Digital UNIX V4.0D

We plan to purge some archive directories, deleting thousand of older files.

Depending on how directories are implemented by Unix,
I imagine that after removal of the files, the directory could still be quite large and may consist of many "empty" slots. Although files had been removed, most commands could still be slow while processing the directory in order to access remaining files, so performance would be poor.

If that was the case, is there a "compacting" command we need to run over the directories after removing files?

The alternative is that a directory list is rebuilt as files get removed, so that the directory consists only of occupied slots at any one time - is that the case?

 

[Mike042]

You are almost correct in your understanding of "how directories are implemented by Unix". The "directory file" contains file headers and their i-nodes. When a file is deleted the i-node number is only dis-associated from the filename so no "empty slots" appear (and hence why "directory files" never seem to get smaller when some of their files are deleted). This is demonstrated by using the command strings /path/to/dir and seeing that dir contains the filenames of all the files it has ever had in it.

As far as I know their is no easy way to "compact" the directory. However here is an idea for a simple process to do what you want for directory /path/to/dir :
1) delete all files that are not required from /path/to/dir
2) tar up the contents of /path/to/dir to a tar-file in /tmp
3) note the ownership and permissions of /path/to/dir
4) delete the directory /path/to/dir (use rm -rf)
5) re-create the directory with the correct ownership and permissions
6) un-tar the tar-file from /tmp into /path/to/dir

I hope that helps.

Mike

 

[Calator]

Hi,
Thanks a lot for the info. Before applying the method you described, I need to clarify one aspect:
- is leaving directory as is after removal of thousand of files going to affect performance of various commands over that directory? is compacting going to improve performance?
this will have to do with an undestanding of how Unix processes the directory

 

[Mike042]

I haven't carried out any performance tests to check this, other people may have. However I would guess that there might be a slight performance issue with many thousands of files. Having said that, UNIX in general does have good sorting and list processing methods, as well as 'caching' algorithms. So you might be investing a huge amount of time and effort for minimal savings. In my experience of UNIX (so far) I have not needed try to 'compact' directories.

I hope that helps.

Mike

 

[Rexxy]

if your not having any issues now, then there will be not change after the removals are done.....if there are major slows etc now, it will still need to run through the hierachial directory structure and will most likely still be just as slow....