Can someone explain defragging/Consolidation??

[jsteph]

Hi all,
Ok, I know in general what defragging/consolidate-free-space is, but my question is specific as to *what level* is the defragging done?

That is, I'd read somewhere that most (or all) defragging software only does "logical" defragging, ie, it is not physically moving data to different physical sectors that are physically contiguous on the harddrive--but instead just moveing FAT or NTFS entries to logically contiguous sectors, but on the actual platter a file's data may be several sectors apart or even on a different platter--and that we must trust the drive's controller to do what it can to physically move data--and that it really may not end up physically contiguous.

I'd read that the HD controller software does the best it can to try and keep things contiguous physically, but can't or doesn't really defrag/consolidate-free-space on demand.

For example, let's say for simplicity's sake we have a single-side, single-platter drive, and sectors are numbered 1 through 100, starting from the outer track inward. Say we have large file was written and based on the available physical sectors , it had half of it, at, say sector 1-20, and the other have at sector 51-70. Now we delete the files that were in sectors 21-50 (right between the big file). Now we run defrag. The article I read seems to state that the file in question will *not* be rewritten to move the sectors from 51-70 to physical sectors 21-40, because as far as NTFS or FAT was concerned, the file was not fragmented logically, and that only some highly specialized software direct from the HD mfr--made specifically for that drive model--would ever be able to do a true physicall defrag/consolidation.
Can anyone shed light on this?
Thanks,
--Jim

 

[technome]

If it were just moving the file entries all defragging would take seconds; defragging moves the data unless the files are in use or marked as immoveable. Defraggers with boot time defrag options (not the built in) do the best job.

........................................
Chernobyl disaster..a must see pictorial

http://www.kiddofspeed.com/default.htm

 

[jsteph]

technome,
Thanks...I know it's moving *something*, but to where is really the question. I'll try to find the article and explain it better. But basically the gist was that it may move a block of data to what *windows* ntfs or fat sees as contiguous--ie, more than just adjusting fat entries. The article seemed to state that logical sectors and tracks in windows may not necessarily be contiguous on the disk (again, using a single-side single-platter as example because the notion of cylinders, ie stacked tracks of multiple platters).

So in other words, windows may indeed be physically moving data to what it sees as sector 1000 to sector 500, because that data belongs immediately following sector 499 instead of at sector 1000. But due to the internal electronics of the hd, that sector may not be truly contiguous on the hd.

The article suggested that due to the (usually proprietary) nature of hd controllers, that there is no way Windows, Norton's, etc can truly know where to move data. So I'm just trying to verify if what I'd read is a generally accepted truth, or that there may be better, lower-level programs out there that have a better idea of what's physically on the disks themselves.
--Jim

 

[stduc]

I think you are referring to that fact that on a modern HDD the cylinders/tracks/sectors the HDD reports to the BIOS and the O/S are essential a fiction. In addition as time goes by the drives electronics spot and logically move sector mapping to remove bad sectors. This is known as the bad sector table.

Therefore a defrag program may well move the data from - say - track 100, sector 40 to another sector on another track. But if that data was in a "swapped out bad sector" it could well be moving it from physical track 1000 sector 11.

Also I drive may report 11 cylinders but only have 5 - or visa versa.

So you are right in your assumptions. However, in general a defrag always speeds things up.

One experiment I've been meaning to try for some time now is to see if there is any benefit in temporarily moving the "my documents" and "program files" folders to another disk before a defrag and then putting them back afterwards. That way all the windows stuff would be as close together as it could get.

[navy]When I married "Miss Right" I didn't realise her first name was 'always'. LOL[/navy]

 

[cdogg]

windows may indeed be physically moving data to what it sees as sector 1000 to sector 500...
But due to the internal electronics of the hd, that sector may not be truly contiguous on the hd.

I'm having a hard time understanding your point about "internal electronics" of a hard drive and the difference between how Windows and the hard drive view the sector formation. I'd have to disagree that there is a difference.

For example, when a bad sector is identified by the OS, it can be labeled on the partition table as such so that the HD Controller avoids this sector. Well going by the point you're trying to make, this would be pointless, since you (or the article) is claiming that Windows doesn't really know the true location of a sector.


The biggest reason I doubt a lot of this theory is that I used to run a defragmenter back in Windows 95/98 called Nutz & Boltz. This utility suite had a lot of tools such as a registry cleaner and powerful defragmenter that got rave reviews beating out Norton in many magazines at the time. Anyway, with the defragmenter you could click on any sector during the defrag process to see what information was currently stored. Every sector and cluster had a physical ID number that identified its position on the disk. There were options in the tool to place certain files at the beginning of the disk which had the quickest disk access times.

This tool was released back in '96. I can only imagine defragmenting utilities since then have become even more sophisticated. It would seem pointless to include such features if it wasn't really possible to know where the beginning of the disk truly was.

~cdogg
"Insanity: doing the same thing over and over again and expecting different results." - Einstein
[tab][navy]For posting policies, click [/navy]here.

 

[vacunita]

I thin what he meant is, Hard drive controllers can check the drive for damaged sectors, and set the pointer for that sector to one of the back up sectors. when this happens Windows is not aware of the location change on the surface itslf.

Its my understanding, that by the time windows starts finding damaged sectors, the drive itself has ran out of backup replacement sectors in which case Windows marks it as damaged, and moves on. However while the drive has enough backup sectors to cover any damaged ones, Windows is unaware of the change in physical location of the sector. for all it knows the sector its addressing is in the middle of the platter. but because the hard drive controller knows its damaged, it redirects the writing requests to one of the backup sectors in the outer rim of the drive for example.

----------------------------------
Ignorance is not necessarily Bliss, case in point:
Unknown has caused an Unknown Error on Unknown and must be shutdown to prevent damage to Unknown.

 

[cdogg]

vacunita,
Even then, we're only talking about a few sectors at most. In addition, there are situations when there aren't any bad sectors.

I would like to learn more about what that article was trying to say before I write it off completely, but for now, it doesn't seem to be too accurate.

~cdogg
"Insanity: doing the same thing over and over again and expecting different results." - Einstein
[tab][navy]For posting policies, click [/navy]here.