Tek-Tips is the largest IT community on the Internet today!
Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!
-
Congratulations biv343 on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!
realloc 2
- Thread starter fl0ra
- Start date
-
1
- #2
No.
Code:
./foo1x
Original pointer at: 0x804a008
Return at 0x804a008, foo at 0x804a008, Size = 24
Return at 0x804a008, foo at 0x804a008, Size = 48
Return at 0x804a008, foo at 0x804a008, Size = 96
Return at 0x804a008, foo at 0x804a008, Size = 192
Return at 0x804a008, foo at 0x804a008, Size = 384
Return at 0x804a008, foo at 0x804a008, Size = 768
Return at 0x804a008, foo at 0x804a008, Size = 1536
Return at 0x804a008, foo at 0x804a008, Size = 3072
Return at 0x804a008, foo at 0x804a008, Size = 6144
Return at 0x804a008, foo at 0x804a008, Size = 12288
Return at 0x804a008, foo at 0x804a008, Size = 24576
Return at 0x804a008, foo at 0x804a008, Size = 49152- Thread starter
- #3
Clairvoyant1332
Programmer
From the man page for realloc(3C):
So, it could change.
So, it could change.
- Thread starter
- #6
Hmm. Really don't know: posix_memalign and a heap in general probably don't work that way.
There is a lot of metadata recorded concerning allocation,
including location specific data.
Clairvoyant:
Good point. You can rely too much on empirical evidence.
Thanks.
There is a lot of metadata recorded concerning allocation,
including location specific data.
Clairvoyant:
Good point. You can rely too much on empirical evidence.
Thanks.
-
1
- #8
chipperMDW
Programmer
The whole point of realloc is to grab a chunk of memory that's big enough and still contains the data that was in there before.
If there's no room afterwards, then it's going to have to look for a place that does have that much memory available. It allocates that, and then it copies the existing data into the new spot before releasing the old data.
If it can just extend the current memory, then sure, doing that is much faster, so of course it'll probably do that rather than moving the whole thing.
> I don't know if the memory is contiguous after realloc.
> It certainly doesn't have to be.
It had better be contiguous.
If I realloced a string to be twice as big, I'd be really pissed if I got back two chunks of memory that weren't connected.
I wouldn't be able to loop through them; I'd read jibberish when I walked off the first chunk without hitting a null; I'd overwrite my other data; and I'd be open to buffer overflow attacks.
Of course it's contiguous. As mentioned, it'd be useless otherwise.
It may not be contiguous in physical memory, but as programmers, we usually don't have to care about that. In the process's virtual memory, it is most certainly contiguous (in the sense that incrementing a pointer in a loop will get you from one end to the other without passing over anything else).
If there's no room afterwards, then it's going to have to look for a place that does have that much memory available. It allocates that, and then it copies the existing data into the new spot before releasing the old data.
If it can just extend the current memory, then sure, doing that is much faster, so of course it'll probably do that rather than moving the whole thing.
> I don't know if the memory is contiguous after realloc.
> It certainly doesn't have to be.
It had better be contiguous.
If I realloced a string to be twice as big, I'd be really pissed if I got back two chunks of memory that weren't connected.
I wouldn't be able to loop through them; I'd read jibberish when I walked off the first chunk without hitting a null; I'd overwrite my other data; and I'd be open to buffer overflow attacks.
Of course it's contiguous. As mentioned, it'd be useless otherwise.
It may not be contiguous in physical memory, but as programmers, we usually don't have to care about that. In the process's virtual memory, it is most certainly contiguous (in the sense that incrementing a pointer in a loop will get you from one end to the other without passing over anything else).
Chipper:
I was referring to actual physical memory.
If you look at the context: we were discussing blocks of
memory, which obviously may not be physically contiguous.
It is the heap after all and implementation details also matter.
As far as overflows, etc..The heap has enough problems as
is. Read Chap5 of "The Shellcoder's Handbook", for an intro.
I was referring to actual physical memory.
If you look at the context: we were discussing blocks of
memory, which obviously may not be physically contiguous.
It is the heap after all and implementation details also matter.
As far as overflows, etc..The heap has enough problems as
is. Read Chap5 of "The Shellcoder's Handbook", for an intro.
chipperMDW
Programmer
Ok.
But if we're talking about physical memory, I can say that there's a possibility that, in this code
[tt]str[/tt] is in a different physical memory location at each call to [tt]printf[/tt].
In other words, a C (or other) program (that isn't an operating system or otherwise running in real mode on the processor) usually doesn't know or care where its code and data are in physical memory, only virtual memory.
If we're taking about physical memory, I should point out that your above test program probably wouldn't be indicative of the pointer being in the same physical memory location, just the same virtual one.
The heap deals with virtual memory, just like the rest of the C program.
I believe the heap is usually implemented by the C library using the C language facilities (e.g. [tt]char heap[100000];[/tt]) to set aside a huge chunk of memory, then keeping track of which blocks are free and which have been allocated.
The heap itself is not necessarily contiguous in virtual memory.
But when I say [tt]malloc(99999)[/tt], those bytes are going to be contiguous in virtual memory. If there's no place in the heap for that to be allocated from, the library may request more memory from the operating system. If there's no memory left for the OS to give, then [tt]malloc[/tt] fails.
I suppose it would be possible (but maybe non-Standard) to have a heap implementation deal out non-contiguous (in virtual memory) blocks of memory, and have the C compiler know how to compile the langauge to interface with the heap code so that pointers can be incremented properly and whatnot. I don't think anyone really does that, though; sounds pretty complex when you can do it much more easily by making it really contiguous.
But if we're talking about physical memory, I can say that there's a possibility that, in this code
Code:
char* str;
...
printf(str);
printf(str);In other words, a C (or other) program (that isn't an operating system or otherwise running in real mode on the processor) usually doesn't know or care where its code and data are in physical memory, only virtual memory.
If we're taking about physical memory, I should point out that your above test program probably wouldn't be indicative of the pointer being in the same physical memory location, just the same virtual one.
The heap deals with virtual memory, just like the rest of the C program.
I believe the heap is usually implemented by the C library using the C language facilities (e.g. [tt]char heap[100000];[/tt]) to set aside a huge chunk of memory, then keeping track of which blocks are free and which have been allocated.
The heap itself is not necessarily contiguous in virtual memory.
But when I say [tt]malloc(99999)[/tt], those bytes are going to be contiguous in virtual memory. If there's no place in the heap for that to be allocated from, the library may request more memory from the operating system. If there's no memory left for the OS to give, then [tt]malloc[/tt] fails.
I suppose it would be possible (but maybe non-Standard) to have a heap implementation deal out non-contiguous (in virtual memory) blocks of memory, and have the C compiler know how to compile the langauge to interface with the heap code so that pointers can be incremented properly and whatnot. I don't think anyone really does that, though; sounds pretty complex when you can do it much more easily by making it really contiguous.
chipperMDW
Programmer
BTW, I'm assuming a program running on a relatively modern OS that utilizes virtual memory. That is, anything but DOS.
> but in that case realloc rely a lot on consecutive chunks
> of memory...
> which might be a problem on my system...
Usually, a process can ask the OS for more memory. I've never had to do it before, so I'm not sure whether the library typically asks for more, or the programmer asks for more after noticing a failed call to [tt]malloc[/tt]. (I said above that the library may request more memory, but it may not, too).
If this is really going to be a problem, you might want to look into writing a custom allocator, or using a library that implements memory "arenas."
This way, you can essentially control which "heap" something gets allocated from. If you're going to need to be mallocing and reallocing some really big buffers, you could give them each their own arena so that they have room to grow. You can also control allocation and freeing strategies and stuff like that.
What type of platform are you on and what are you planning to alloc? I can try to find some "arena" libraries that might work.
> but in that case realloc rely a lot on consecutive chunks
> of memory...
> which might be a problem on my system...
Usually, a process can ask the OS for more memory. I've never had to do it before, so I'm not sure whether the library typically asks for more, or the programmer asks for more after noticing a failed call to [tt]malloc[/tt]. (I said above that the library may request more memory, but it may not, too).
If this is really going to be a problem, you might want to look into writing a custom allocator, or using a library that implements memory "arenas."
This way, you can essentially control which "heap" something gets allocated from. If you're going to need to be mallocing and reallocing some really big buffers, you could give them each their own arena so that they have room to grow. You can also control allocation and freeing strategies and stuff like that.
What type of platform are you on and what are you planning to alloc? I can try to find some "arena" libraries that might work.
- Thread starter
- #12
- Status