3    Shared Memory

Shared memory and memory-mapped files allow processes to communicate by incorporating data directly into process address space. Processes communicate by sharing portions of their address space. When one process writes to a location in the shared area, the data is immediately available to other processes sharing the area. Communication is fast because there is none of the overhead associated with system calls. Data movement is reduced because data is not copied into buffers.

A process manipulates its address space by mapping or removing portions of memory objects into the process address space. When multiple processes map the same memory object, they share access to the underlying data. Shared-memory functions allow you to open and unlink the shared-memory files.

This chapter includes the following sections:

• Memory Objects, Section 3.1

• Locking Shared Memory, Section 3.2

• Using Shared Memory with Semaphores, Section 3.3

3.1    Memory Objects

The memory-mapping and shared-memory functions allow you controlled access to shared memory so that the application can coordinate the use of shared address space.

When you use a shared, mapped file, the changes initiated by a single process or multiple processes are reflected back to the file. Other processes using the same path and opening the connection to the memory object have a shared mapping of the file. Use memory-mapping or file control functions to control usage and access. If the mappings allow it, data written into the file through the address space of one process appears in the address space of all processes mapping the same portion of the file.

Memory-mapped objects are persistent; their names and contents remain until all processes that have accessed the object unlink the file.

Shared-memory regions and memory-mapped files follow the same general usage, as follows:

1. Obtain a file descriptor with a call to the open or shm_open function.

2. Map the object using the file descriptor with a call to the mmap function.

3. Unmap the object with a call to the munmap function.

4. Close the object with a call to the close function.

5. Remove the shared-memory object with a call to the shm_unlink function or, optionally, remove a memory-mapped file with a call to the unlink function.

Often shared-memory objects are created and used only while an application is executing. Files, however, may need to be saved and reused each time the application is run. The unlink and shm_unlink functions remove (delete) the file and its contents. Therefore, if you need to save a shared file, close the file but do not unlink it.

You can use memory-mapped files without using shared memory, but this chapter assumes that you will want to use them together. The following functions are used to open and unlink shared memory:

Table 3-1 lists the functions for creating and controlling memory-mapped objects.

Table 3-1:  Memory-Mapping Functions

A memory object can be created and opened by a call to the shm_open function. Then the object can be mapped into process address space. File control functions allow you to control access permissions, such as read and write permission or the timing of a file update.

Data written to an object through the address space of one process is available to all processes that map the same region. Child processes inherit the address space and all mapped regions of the parent process. When the object is opened, the child process can map it with the mmap function to establish a map reference. If the object is already mapped, the child process also inherits the mapped region.

Unrelated processes can also use the object, but they must first call the open or shm_open function (as appropriate) and then use the mmap function to establish a connection to the shared memory.

3.1.1    Opening a Shared-Memory Object

A process can create and open shared-memory regions early in the life of the application and then dynamically control access to the shared-memory object. Use the shm_open function to open (establish a connection to) a shared-memory object. After one process calls shm_open to create and name a shared memory object, each subsequent process that needs to access the shared memory object must call shm_open and specify the same name. The shared-memory object name can be either a string or a pathname.

The shm_open function provides a set of flags that prescribe the action of the function and define access modes to the shared-memory object. Shared-memory access is determined by the OR of the file status flags and access modes listed in Table 3-2.

Table 3-2:  Status Flags and Access Modes for the shm_open Function

The first process to call the shm_open function should use the O_CREAT flag to create the shared-memory object, to set the object's user ID to that of the calling process, and to set the object's group ID to the effective group ID of the calling process. This establishes an environment whereby the calling process, all cooperating processes, and all child processes share the same effective group ID with the shared-memory object.

A process can create an exclusive connection to a shared-memory object by using the O_CREAT and O_EXCL flags. In this case, other processes attempting to create the shared-memory object at the same time will fail.

The oflag argument of the shm_open function requests specific actions from the shm_open code. For example, the following code creates an exclusive shared-memory object and opens it for read and write access:

fd = shm_open("all_mine", (O_CREAT|O_EXCL|O_RDWR), 0);
 

When a shared-memory object is created, its state and name (including all associated data) are persistent. Its state and name remain until the shared memory is unlinked with a call to the shm_unlink function and until all other references to the shared memory are gone.

Example 3-1 shows the code sequence to include shared-memory objects in an application.

Example 3-1:  Including a Shared-Memory Object

#include <unistd.h>
#include <sys/types.h>
#include <sys/mman.h>
#include <fcntl.h>
 
main ()
{
      int md;
      int status;
      long pg_size;
      caddr_t virt_addr;
 
      /* Create shared-memory object */
 
      md = shm_open ("my_memory", O_CREAT|O_RDWR, 0);
      pg_size = sysconf(_SC_PAGE_SIZE);
 
      if((ftruncate(md, pg_size)) == -1){    /* Set the size */
          perror("ftruncate failure");
          exit();
      }
                                            /* Map one page */
 
      virt_addr = mmap(0, pg_size, PROT_WRITE, MAP_SHARED, md, 0);
           
.
.
.
status = munmap(virt_addr, pg_size);  /* Unmap the page */
      status = close(md);                   /*   Close file   */
      status = shm_unlink("my_memory");     /* Unlink shared-memory object */
}

3.1.2    Opening Memory-Mapped Files

The open function points to the data you intend to use; the mmap function establishes how much of the data will be mapped and how it will be accessed. Use the same access permissions that you would normally use on any call to the open function. If you intend to read the file only, specify read permission only on the open function. If you intend to read and write to the file, open the file with both read and write permission. After opening a file, call the mmap function to map the file into application address space.

When you have finished using a memory-mapped file, unmap the object by calling the munmap function, then close the object with the close function. Any memory locks resulting from a call to the mlock function associated with the address range are removed when the munmap function is called. The application could then remove the data file by calling the unlink function.

3.1.3    Mapping Memory-Mapped Files

The mmap function maps data from a file into memory. The parameters to the mmap function specify the starting address and length in bytes for the new region, access permissions, attributes of the mapped region, file descriptor, and an offset for the address. The MAP_SHARED flag indicates that the object can be accessed by other processes. A call to the munmap function unmaps the same region.

The address, length, and offset of the new mapped region should be a multiple of the page size returned by a call to the sysconf(_SC_PAGE_SIZE) function. If the length is not specified as a multiple of the page size returned by sysconf, then any reference to an address between the end of the region and the end of the page containing the end of the region is undefined. Note, too, that the offset must be aligned and sized properly. Other size parameters may also need to be aligned, depending on whether you specified MAP_FIXED.

The prot argument determines the type of access permitted to the data being mapped. As with other file permissions, the argument is constructed from the bitwise inclusive-OR of one or more of the following flags:

Whatever protection options you specify as the prot argument, the file descriptor must have been opened with at least read access. If you specify PROT_WRITE, the file descriptor must have been opened with write permission, unless MAP_PRIVATE is specified in the flags parameter.

The flags parameter provides additional information about how to handle mapped data. The flags parameter uses one of the following flags:

MAP_SHARED, MAP_PRIVATE, and MAP_FIXED are the only flags specified by POSIX 1003.1b. The MAP_ANONYMOUS, MAP_FILE, and MAP_VARIABLE flags are not part of the POSIX 1003.1b interface, but are supported by Tru64 UNIX. For more information on these flags, see the reference page for the mmap function.

The MAP_FIXED flag controls the location of the new region. No matter what flag is specified, a mapped region is never placed at address zero or at an address where it would overlap with an existing region. When multiple processes use the mapped object, the call to the mmap function can specify the address, and subsequent calls to the mmap function can use MAP_FIXED to request the same address in other processes. Cooperating processes must also use care to communicate this address among themselves. If you specify MAP_FIXED and for some reason the system is unable to place the new region at the specified address, the call fails.

The MAP_SHARED and MAP_PRIVATE flags control the visibility of modifications to the mapped file or shared-memory region. The MAP_SHARED flag specifies that modifications made to the mapped file region are immediately visible to other processes that are mapped to the same region and also use the MAP_SHARED flag. Changes to the region are written to the file.

The MAP_PRIVATE flag specifies that modifications to the region are not visible to other processes, whether or not the other process used MAP_SHARED or MAP_PRIVATE. Modifications to the region are not written to the file.

Access to the mapped region or shared-memory region is controlled by the flags specified in the prot parameter. These flags function much the way they do for any other file descriptor: access is specified as the OR of read, write, and execute, with an additional flag to indicate that data cannot be accessed. The mprotect function changes the protection on a specified address range. That range should be within the range specified on the call to the mmap function. Protection flags can interact with the MAP_SHARED, MAP_PRIVATE, and MAP_FIXED flags. See the online reference pages for mmap and mprotect for specifics.

When you unmap a mapped region or shared memory, be sure to specify an address and length in the range of the parameters used in the call to the mmap function.

3.1.4    Using File Functions

Shared-memory objects and memory-mapped files use the file system name space to map global names for memory objects. As such, POSIX.1 file control functions can be used on shared-memory objects and memory-mapped files, just as these functions are used for any other file control. Table 3-3 lists some of the file functions available.

Table 3-3:  File Functions Used with Memory-Mapped Files

You can use the fchmod function to change access permissions on a file. If you are the owner of the file or have superuser privileges, you can use the fchmod function to set the access mode and grant or deny permissions to the group, user, or others. Use the fcntl function to retrieve and set the value of the close-on-exec flag, status flags, and access modes, or to set and clear locks. Using the fcntl function, you can override locks set with the flock function. The fstat function returns information about the file, such as access permissions, link references, and type and size of file. You can use this function to obtain information for use in subsequent calls to other file control functions.

You can apply a lock to a shared-memory object or mapped file by using a variety of file control functions, including fcntl and flock. Both these functions apply a lock on an open file, but they differ in how the lock is performed and in the range of other tasks they can perform.

Note that the locks applied with these functions are for files, not file descriptors. That means that under most circumstances, file locks are not inherited across a fork. If a parent process holds a lock on a file and the parent process forks, the child process will inherit the file descriptor but not the lock on the file. A file descriptor that is duplicated with one of the dup functions does not inherit the lock.

The fcntl function is used for general file control. In addition to locking and unlocking an open file, the fcntl function is used to return or set status, return a new file descriptor, or return process IDs.

The flock function is limited to applying locks on a file and is not used for general file control.

See the online reference pages for more information on using file control functions.

3.1.5    Controlling Memory-Mapped Files

Several functions let you manipulate and control access to memory-mapped files and shared memory. These functions include msync and mprotect. Using these functions, you can modify access protections and synchronize writing to a mapped file.

The msync function synchronizes the caching operations of a memory-mapped file or shared-memory region. Using this function, you can ensure that modified pages in the mapped region are transferred to the file's underlying storage device, or you can control the visibility of modifications with respect to file system operations.

Flags used on the msync function specify whether the cache flush is to be synchronous (MS_SYNC), asynchronous (MS_ASYNC), or invalidated (MS_INVALIDATE). You can specify either the MS_SYNC or MS_ASYNC flag, but not both.

When you use the MS_SYNC flag, the msync function does not return until all write operations are complete and the integrity of the data is assured. All previous modifications to the mapped region are visible to processes using the read parameter.

When you use the MS_ASYNC flag, the msync function returns immediately after all of the write operations are scheduled.

When you invalidate previously cached copies of the pages, other users are required to get new copies of the pages from the file system the next time they are referenced. In this way, previous modifications to the file made with the write function are visible to the mapped region.

When using the msync function, you should use pages within the same address and length specified in the call to the mmap function to ensure that the entire mapped region is synchronized.

The mprotect function changes the access protection of a mapped file or shared-memory region. When using the mprotect function, use pages within the same address and length specified in the call to the mmap function. Protection flags used on the mprotect function are the same as those used on the mmap function.

Note that use of the mprotect function modifies access only to the specified region. If the access protection of some pages within the range were changed by some other means, the call to the mprotect function may fail.

3.1.6    Removing Shared Memory

When a process has finished using a shared-memory segment, you can remove the name from the file system namespace with a call to the shm_unlink function, as shown in the following example:

status = shm_unlink("my_file");

The shm_unlink function unlinks the shared-memory object. Memory objects are persistent, which means the contents remain until all references have been unmapped and the shared-memory object has been unlinked with a call to the shm_unlink function.

Every process using the shared memory should perform the cleanup tasks of unmapping and closing.

3.2    Locking Shared Memory

You can lock and unlock a shared-memory segment into physical memory to eliminate paging. The MCL_FUTURE argument to the mlockall function causes new shared-memory regions to be locked automatically. See Chapter 4 for more information on using the mlock and mlockall functions.

Example 3-2 shows how to map a file into the address space of the process and lock it into memory. When the file is unmapped, the lock on the address is removed.

Example 3-2:  Locking a Memory Object

/* This program locks the virtual memory address that */
/* was returned from the mmap() function into memory. */
 
#include <unistd.h>
#include <sys/types.h>
#include <stdio.h>
#include <sys/file.h>
#include <sys/mman.h>
#include <sys/stat.h>
#include <errno.h>
 
main()
{
int fd;
caddr_t pg_addr;
 
int size = 5000;
int mode =  S_IRWXO|S_IRWXG|S_IRWXU;
 
     /* Create a file */
 
fd = shm_open("example", O_RDWR|O_CREAT, mode);
if(fd < 0){
   perror("open error ");
   exit();
}
 
     /* Set the size */
 
if((ftruncate(fd, size)) == -1){
     perror("ftruncate failure");
     exit();
}
 
     /* Map the file into the address space of the process */
pg_addr = (caddr_t) mmap(0, size, PROT_READ|PROT_WRITE|PROT_EXEC, MAP_SHARED,
                         fd, 0);
 
if(pg_addr == (caddr_t) -1){
  perror("mmap failure");
  exit();
}
 
     /* Lock the mapped region into memory */
 
if(mlock(pg_addr,size) != 0){
  perror("mlock failure");
  exit();
}
 
     /* Unmap of the address region removes the memory lock */
     /* established on the address region by this process   */
 
if(munmap(pg_addr, size) < 0)
   perror("unmap error");
close(fd);
shm_unlink("example");
exit();
}

You can also lock the file so that other processes cannot use it, making it an exclusive resource for a process and its descendants. See Section 3.1.4 for more information on locking files.

3.3    Using Shared Memory with Semaphores

When using shared memory, processes map the same area of memory into their address space. This allows for fast interprocess communication because the data is immediately available to any other process using the same shared memory. If your application has multiple processes contending for the same shared-memory resource, you must coordinate access.

Semaphores provide an easy means of regulating access to a memory object and determining if the memory resource is available. Typically, an application will begin execution at a nonrealtime priority level, then perform the following tasks when using mapped or shared-memory objects and semaphores:

1. Create the shared-memory object.

2. Determine the address and map the region into memory.

3. Create a semaphore.

4. Adjust the process priority and scheduling policy as needed.

5. Before a read or write operation, lock (reserve) the semaphore.

6. After a read or write operation, unlock (release) the semaphore.

A process can lock the semaphore associated with a mapped or shared-memory object to indicate that the process requires exclusive access. Cooperating processes normally wait until the semaphore is unlocked before accessing a region.

See Chapter 9 for information on semaphores and for an example using semaphores and shared memory.