virtual memory blues 1

[lionelhill]

New Scientist has reported that someone's noticed that the contents of RAM can become fossilised on your hard disk because of the actions of virtual memory. I'm sure we've all been aware it could happen for years; you type a confidential document, the operating system feels it needs a bit of memory for something and shunts your document into its swap-file; another day something else gets written where the document was, but the thing that gets written is rather small and doesn't fill a whole sector. Now the remainder of the sector, containing bits of your document, is fixed for as long as the little something-else remains on the disk. It might hang around almost indefinitely...

(1) Why has it taken so long for anyone to worry about this?

(2) The suggested remedy is to overwrite things with zeroes as quickly as possible after you've used them. But isn't this a bit like saying "my door can't be locked, so I'll pop out to the shops as quickly as possible and hope no one breaks in"?

(3) Given that virtual memory is also the reason why your computer goes into an endless think, accompanied by wash-and-rinse cycle from the hard drive (i.e. application program thinks it's talking to RAM, and is actually talking to hard disk), has our love-affair with virtual memory been a little over enthusiastic? Especially now real memory is ever cheaper.

You'll guess my bias, but I'm just interested what others think...

 

[sleipnir214]

Some organizations have been concerned about the latent contents of hard drives for some time.

The Pentagon, for example, has only one method for reconditioning hard drives for reuse should the computer in which it resides be sold: run the drive through a metal shredder.



Want the best answers? Ask the best questions!

TANSTAAFL!!

 

[Dimandja]

This problem has been addressed by many for years(see sleipnir214's example).

Remember that the more memory is available the larger applications become, so, the more virtual memory is needed. There is no end in sight for virtual memory. In fact, there is nothing wrong with it.

It's possible to design applications that clean up thoroughly after themselves. But, is it for the effort and the additional endless think?

My computer is so busy, fossils have a short life.

 

[chiph]

This has been known for many years. Just maybe not in the popular press.

PGP has had code in it to prevent it's working memory from being swapped out since one of it's earliest versions.

Chip H.


____________________________________________________________________
If you want to get the best response to a question, please read FAQ222-2244 first

 

[lionelhill]

Chiph,
seems to me there's a bit of a conflict here: the whole idea of "virtualness" is that the user (even the programmer) should be unaware it's happening. The ideal operating system would swap your stuff out without you even knowing, when it needs your memory for a second, independant application.

Dimandja,
yup, though I still can't understand why all applications keep growing. I mean, that document I write is just the same size as it always was, and I still just want it printed the same way...
I agree there's nothing wrong with virtual memory as such, but feel the cleaning up needs the cooperation of the operating system. I can't see how I can clean up after myself when I can't choose when I'm going to land up in a swap file! The writers in New Scientist merely suggest overwriting with zeroes as soon as you can, which is hardly an absolute in security. That also suggests they feel it's not straightforward to clear up in the operating systems they tested.

All,
yes, thanks for comments. The original "news" didn't surprise me much on the novelty front, either. Having read about schemes to read hard disks by tracking the little bits of latent magnetic code just off the centre of the main tracks, or the slight deviations in the "1" and "0" pattern (0.99 vs 1.01!) left by previous data, I think the Pentagon's shredder approach is the only sensible way to go. Except possibly a furnace.

Fortunately most of us haven't got anything on our hard disks that's worth the attention!

 

[chiph]

seems to me there's a bit of a conflict here: the whole idea of "virtualness" is that the user (even the programmer) should be unaware it's happening. The ideal operating system would swap your stuff out without you even knowing, when it needs your memory for a second, independant application.

Normally, you're right. For your typical productivity application that needs more memory than is physically installed, virtual memory is the only answer. And it is transparent to the program -- it just makes a memory request (calloc(), new(), etc) and gets memory back for it to use. It doesn't care that some other part of itself or another program probably got swapped to disk in order to fulfill the request.

An exception to this is in the kernel of the operating system -- you can't have the memory management routines themselves swapped out (results in infinite page fault). So the OS designers have a flag on each memory page to indicate if it's swappable or not.

The designers of PGP knew that if memory got swapped to disk it's possible for the private key or passphrase to be written out as well (contents of variables get written out as well as the code itself). This would be a huge security hole, as all the hostile agents & forensic examiners know to examine the swap file for "left-overs". Once an attacker has either of those pieces of information, it's possible to decrypt the computer owner's messages (which in some countries means the sender gets imprisoned or executed). So the authors of PGP set that flag on their memory requests to ensure their program's data stays in physical RAM.

Chip H.


____________________________________________________________________
If you want to get the best response to a question, please read FAQ222-2244 first

 

[jwenting]

when I worked at ING Barings between 1998 and 2000 the policy was also to shred computers.
Not just the harddisks were shredded but the motherboards, CPUs and network cards as well.
All these together were used to uniquely identify hardware on the LAN and WAN as belonging to the company and for security reasons they were not allowed to ever get outside the company.

Also for security all machines had floppy drives and CD drives disabled at hardware level (except a few machines used for installation where hardware encryption was used to prevent anything from being written to floppy that could be deciphered on a computer outside the company).

There was a bit of panic when one branch was broken into and all computers stolen...

 

[Dimandja]

I once worked for a company that had a unique way of getting rid of its old computers.

A party was held on a good Friday. Everyone who had used the computer is invited and given, in turn, a huge mallet which they used to take a good whack at the computer. By the time beer was flowing, the machine lay in utter ruin of bits and scraps.

Try it sometime: liberating.

Dimandja

 

[lionelhill]

Thanks, Chiph, for the excellent explanation.

Reflecting, I sort-of feel there are enough applications where virtual memory is a problem that there may be some mileage in PGP's approach being more mainstream. Sometimes I'd rather the operating system tell me "Sorry, that memory useage is unreasonably huge/insecure" rather than trying to achieve the impossible by simulation.

Jwenting, if it's any consolation, the thieves probably had a bit of a panic too, when they realised what they nicked was more-or-less useless!