max allocatable size for malloc()? 1

[zanza]

does anyone know what the maximum amount of memory malloc() can pull is?

i know it fails when i go up to and past a gig. i have 384 megs of system memory and my page file is maxed out at 512 megs.

also, just because im already writing and youre already reading... ^_- when i ask malloc to do its thing for 1 gig, it failed, returning NULL (as it should, i think). but when i asked it to allocate 4 gigs, it actually _passed_, allocating it to god knows where, not returning NULL. and the app crashed after accessing 16k of that memory. so whats up with that?

žÅNžÅ

 

[Salem]

> does anyone know what the maximum amount of memory malloc() can pull is?
Probably around the sum of your real memory and your swap space. But this is the total memory for your OS and all your other programs, so the actual amount is somewhat less.

> but when i asked it to allocate 4 gigs, it actually _passed_,
Are you sure that it didn't just truncate the amount to some modulo N value which was actually pretty small?

Paste an actual test program which attempts to malloc this amount of memory.


--

 

[Cagliostro]

4Gb is for whole program, including the code and data. This is virtual memory written in paged files on the HDD usually. This paged memory usualy is limited to 500Mb, but you can increate it to 4gb. Anyway, there is no reason to allocate 4gb of memory. You can use memory mapped files and you will be able to process huge ammounts of data for example 100000Gb and much more, you need just a HDD where you can place a such file . Files are accessed with 64bit addresses.

Ion Filipski

 

[apatterno]

Note well:

void *malloc(size_t size);

size_t is typedef'd to a 32-bit int. Thus a value of 4 gb exceeds this value and it should overflow to zero.

[sub]I REALLY hope that helps.[/sub]
Will

 

[zanza]

i did figure out that size_t was a 32bit int, and system memory+swap space was one of my guesses.

but as to why exactly i was trying to do that... im learning about xor encryption, and i just wanted an app that would generate me a random key of whatever length i asked.

so, me being me, i want to use a key thats overly long and really quite pointless. ^_^ of course, im not actually _planning_ on using a 4 gig key, but it would be awefully funny to hand someone a cd and say "oh, by the way, its encrypted... and heres the key" and then give them a fully burned dvd.

so yeah, just out of my non-logical curiousity, i wanted to know exactly how much space malloc can pull. it could be one of those things that might just be useful to know.

heres the source for the app (im still working on perfecting the random function):

http://fzy.mine.nu/~seiki/code/key.c

žÅNžÅ

 

[Cagliostro]

malloc has nothing to do with encription

Ion Filipski

 

[zanza]

i know that. but it has lots to do with dynamically allocated strings.

žÅNžÅ

 

[sandup]

Funny guy zanza 4 gb for a key

anyway, how did you pass 2^32 in a 32-bit parameter? Or did you cleverly subtract 1?

 

[Cagliostro]

You can just use files for big ammount of data. So, use small memory buffer and big file to read/update.

Ion Filipski

 

[zanza]

sandup: if you can call that clever, yeah. thats what i did.

and ion: yeah, i know how i could actually use a 4 gig key, but i dont really want to do that right now, it was just a funny idea. at this point (as suggested by the subject of this post) im just curious about the functional limits of malloc.

žÅNžÅ

 

[sandup]

hey zanza maybe malloc() took it as a negative number

just messing around...

 

[Salem]

> im just curious about the functional limits of malloc.
Variable to say the least.

Like for instance changing your machine, adding more (or less) RAM to your current machine, or changing the number of running programs on your machine.
These will all affect the apparent maximum amount of memory you can malloc.
I'd say that anything above 100MB for a single block, you should be prepared for frequent failures to malloc memory.

There is simply no need for allocating such large blocks of data. You simply use buffers to hold the next block of cleartext and key, and keep the bulk information in files.

Oh and read this

http://www.interhack.net/people/cmcurtin/snake-oil-faq.html

That is, before you start relying on any standard rand() function for generating keys.

--

 

[zanza]

so ka... thank you salem (and everyone else who posted).

on a side note, i wont be doing any real encryption with this. im just learning for fun. but thank you for the text: tis always welcome. ^_^

>There is simply no need for allocating such large blocks of data.
i know, i know! wie ich hab gesagt (as ive said) im just using the learning through fun and asking questions when needed method.

thanks again.

žÅNžÅ