Newbie: Shared Memory Size: shmget()

[hendler]

Hi All,
This is my first post - and I am just beginning C so your help is greatly appreciated.

I'm trying to understand the size_t param for shmget() as well as the limits of shared memory in general. I'm on Linux for now, but want this to be cross platform.

When creating a shared memory segment I create the size based on filesize (character length) + a buffer + 1 (for an extra \0 since for now I am storing strings).

When I later go to read a shared memory segment I am confused as to what is happening. I have a ton of questions so I hope I am asking them correctly.

shmget(key,filesize, IPC_CREAT | 0555)

If I am retrieving a shared memory segment how would I know filesize ahead of time? Would I need to?
When I get the stat from shmctrl it is a different size than PAGE_SIZE (which I thought it rounded up to) and from the size I allocated. So what is size of shared memory segment really about?

I've ready quite a bit on the web and some of the Linux man page. But I don't understand the size allocations and limitations of shared memory segments.


PS

Should I remove IPC_CREAT if I want it to fail if it doesn't exist?

 

[SamBones]

You are a beginner and you're trying to use [tt]shmget()[/tt]? Are you sure you don't want [tt]malloc()[/tt]? What are you trying to do?

The function [tt]shmget()[/tt] is to get memory that can be shared between processes, usually used for interprocess communication. This is not really a beginner type of problem.

The function [tt]malloc()[/tt] is used to allocate memory for your program's use. It isn't shared, but is owned and used solely by your program.

 

[hendler]

Hi SamBones,

I guess my definition of beginner is different!
Yes, I understand the differences between malloc and shmget.

What I am trying to do is persist memory (some strings, and preferably a struct). The application doesn't run as a dameon - the applications life span is short, so I don't believe I have persistant memory. Reading from disk and DB is slower than I want.

I suppose there are a lot of OS specific issues with shmget since it is os dependent.

If there is a way to persist memory in malloc() I guess that would suffice. If the application had it's own protected memory where I could use actual memory addresses would that work instead of shmget()?

 

[Diancecht]

Curiose approach, and also curious the fact that saving to a file is too slow for you.

Ok, let's try to answer your question.

shmget returns a memory space that can already be allocated. The size param specifies that the memory are needs to have at least those bytes. I think it's used more when you want to create than when you want to retrieve an already created one.

Description of the function:

http://www.opengroup.org/onlinepubs/009695399/functions/shmget.html

A couple of examples:

http://jan.netcomp.monash.edu.au/OS/l9_1.html

http://www.cs.ucsd.edu/classes/sp00/cse120_A/threads.html

Just in case you want to go OS independent:

http://www.byte.com/art/9410/sec14/art3.htm

Btw, be careful, IPC resources are not unlimited. If you want to give persistence and you don't never ever free the resources, the OS can run out of them.

Cheers,
Dian

 

[Salem]

Seems to me if you're running small programs to update some database, that another significant overhead is the process create/destroy time.

How about a client/server model where a single long-lived database server resides in memory and uses normal malloc, and you communicate your requests to that server?



--

 

[hendler]

Hi Salem,
The client server model works in everything else I do in web development. Databases like MySQL are in-memory and quite fast. I used to use HeapTables in MySQL for similar goals.

The key here is that the "data" and content is inherently in flat files. After doing more research I am using shared memory to store some meta data about flat files, but the bulk of content I will use MemoryMapped files. So far, in Unix mmap works great, and I know there is a Windows equivilant.

 

[SamBones]

If you create a shared memory segment to hold and persist data, you're going to have to save the identifier to be able to find the memory segment again. You'll probably have to save the identifier in a file, so you'll have to hit the disk anyway.

If this information needs to survive a reboot, shared memory is not the way to go.

Hope this helps.

 

[hendler]

Hi Sambones.
Good insight.
No, it doesn't need to survive a reboot. That's a gotcha for some people.

The identifier is itself stored in memory, in the application. So if I understand what you mean by file identifier (generated by ftok, shmget, and shmat) there should be no disk access.

Also, any file that is mmap'd is in memory, so I wouldn't necessarily need it then.