smobjlib.c

VXWORKS源代码

/* smObjLib.c - shared memory objects library (VxMP Option) */

/* Copyright 1984-2002 Wind River Systems, Inc. */

/*
modification history
--------------------
01s,03may02,mas  cache flush and volatile fix (SPR 68334); bridge flush fix
		 (SPR 68844)
01r,24oct01,mas  fixed diab warnings (SPR 71120); doc update (SPR 71149)
01q,20sep01,jws  doc change for SPR68418
01p,16mar99,dat  fixed declaration of smUtilTasClearRtn. (SPR 25804)
01o,14mar99,jdi  doc: removed refs to config.h and/or configAll.h (SPR 25663).
01n,22feb99,wsl  DOC: removed all reference to spurious error code.
01l,05may93,caf  tweaked for ansi.
01l,17mar94,pme  Added WindView level 2 event logging.
		 Added comments about acknowledge of BUS interrupts (SPR# 2560).
01k,13mar93,jdi  code style fix for mangen.
01j,14feb93,jdi  documentation cleanup for 5.1.
01i,29jan93,pme  added little endian support.
		 added smObjLibInit() manual. Made smObjLibInit() return
		 ERROR if already initialized.
		 suppressed ntohl() macros in smObjCpuInfoGet().
		 aligned partition addresses.
01h,08dec92,jdi  documentation cleanup.
01g,02oct92,pme  added SPARC support. documentation cleanup.
01f,29sep92,pme  fixed bug in smObjInit (objCpuTbl was not converted to
		 local value in pSmObjHdr).
		 Added version number. comments cleanup.
		 set PART_OVH to 32 bytes.
01e,23aug92,jcf  cleanup.
01d,30jul92,pme  added SM_OBJ_SIZE(), cleaned up smObjSetup.
                 changed parameter to smObjBeat.
		 added smObjTasClearRoutine for use with smLockGive().
		 added shared memory layout chart.
01c,24jul92,elh  changed parameters to smIsAlive.
01b,24jul92,gae  fixed number of parameters to logMsg().
01a,07feb92,pme  written.
*/

/*
DESCRIPTION
This library contains miscellaneous functions used by the shared memory
objects facility (VxMP).  Shared memory objects provide high-speed
synchronization and communication among tasks running on separate CPUs that
have access to a common shared memory.  Shared memory objects are system
objects (e.g., semaphores and message queues) that can be used across
processors.

The main uses of shared memory objects are interprocessor synchronization,
mutual exclusion on multiprocessor shared data structures, and high-speed
data exchange.

Routines for displaying shared memory objects statistics are provided  
by the smObjShow module.

SHARED MEMORY MASTER CPU
One CPU node acts as the shared memory objects master.  This CPU
initializes the shared memory area and sets up the shared memory anchor.
These steps are performed by the master calling smObjSetup().  This
routine should be called only once by the master CPU.  Usually smObjSetup()
is called from usrSmObjInit().  (See "Configuration" below.)

Once smObjSetup() has completed successfully, there is little functional
difference between the master CPU and other CPUs using shared memory objects,
except that the master is responsible for maintaining the heartbeat in
the shared memory objects header.

ATTACHING TO SHARED MEMORY
Each CPU, master or non-master, that will use shared memory objects
must attach itself to the shared memory objects facility, which
must already be initialized.

Before it can attach to a shared memory region, each CPU must allocate
and initialize a shared memory descriptor (SM_DESC), which describes the
individual CPU's attachment to the shared memory objects facility.  Since
the shared memory descriptor is used only by the local CPU, it is not
necessary for the descriptor itself to be located in shared memory.  In
fact, it is preferable for the descriptor to be allocated from the CPU's
local memory, since local memory is usually more efficiently accessed.

The shared memory descriptor is initialized by calling smObjInit().  This
routine takes a number of parameters which specify the characteristics of
the calling CPU and its access to shared memory.

Once the shared memory descriptor has been initialized, the CPU can
attach itself to the shared memory region.  This is done by calling
smObjAttach().

When smObjAttach() is called, it verifies that the shared memory anchor
contains the value SM_READY and that the heartbeat located in the shared
memory objects header is incrementing.  If either of these conditions is
not met, the routine will check periodically until either SM_READY or an
incrementing heartbeat is recognized or a time limit is reached.  The
limit is expressed in seconds, and 600 seconds (10 minutes) is the
default.  If the time limit is reached before SM_READY or a heartbeat is
found, ERROR is returned and `errno' is set to S_smLib_DOWN .

ADDRESS CONVERSION
This library also provides routines for converting between local and
global shared memory addresses, smObjLocalToGlobal() and
smObjGlobalToLocal().  A local shared memory address is the address
required by the local CPU to reach a location in shared memory.  A global
shared memory address is a value common to all CPUs in the system used to
reference a shared memory location.  A global shared memory address is
always an offset from the shared memory anchor.

SPIN-LOCK MECHANISM
The shared memory objects facilities use a spin-lock mechanism based on an
indivisible read-modify-write (RMW) operation on a shared memory location
which acts as a low-level mutual exclusion device.  The spin-lock mechanism
is called with a system-wide configuration parameter,
SM_OBJ_MAX_TRIES, which specifies the maximum number of RMW tries on a
spin-lock location.

Care must be taken that the number of RMW tries on a spin-lock on a
particular CPU never reaches SM_OBJ_MAX_TRIES, otherwise system
behavior becomes unpredictable.  The default value should be sufficient
for reliable operation.

The routine smObjTimeoutLogEnable() can be used to enable or disable the
printing of a message should a shared memory object call fail while trying
to take a spin-lock.

RELATION TO BACKPLANE DRIVER
Shared memory objects and the shared memory network (backplane) driver use 
common underlying shared memory utilities.  They also use the same anchor, 
the same shared memory header, and the same interrupt when they are used at
the same time.

LIMITATIONS
A maximum of twenty CPUs can be used concurrently with shared memory
objects.  Each CPU in the system must have a hardware test-and-set (TAS)
mechanism, which is called via the system-dependent routine sysBusTas().

The use of shared memory objects raises interrupt latency, because internal
mechanisms lock interrupts while manipulating critical shared data
structures.  Interrupt latency does not depend on the number of objects or
CPUs used.

GETTING STATUS INFORMATION
The routine smObjShow() displays useful information regarding the current
status of shared memory objects, including the number of tasks using
shared objects, shared semaphores, and shared message queues, the number
of names in the database, and also the maximum number of tries to get
spin-lock access for the calling CPU.

INTERNAL

Routine structure

    smObjSetup smObjLocalToGlobal smObjGlobalToLocal   smObjAttach   smObjInit
     /    \                                            /    |            |
 smSetup   ------------------------                smAttach |          smInit
          /                        \                        |
   smMemPartInit               smNameInit                   |    
                                                            |
                                                            |
                                                            |
  smObjEventSend-*             smObjNotifyHandler-*         | 
   /   |      |                 /      |                    |
  /    \ smObjCpuInfoGet       /  smObjEventProcess         |     smObjTcbInit  
  |     -------------  --------                             |
smUtilIntGen         \/                              smUtilIntConnect
                      |
                  smDllConcat

        *                      *                     (o)
        |                      |                      |
   SM_OBJ_LOCK_TAKE    SM_OBJ_LOCK_GIVE         smMemPartAlloc


Memory layout

 Shared Memory         Shared Memory           Shared Memory
    Anchor              Descriptor             CPU Descriptors
 (SM_ANCHOR)             (SM_HDR)              (SM_CPU_DESC)
--------------   ----->--------------   ------>--------------
|readyValue  |   |     | tasType    |   |      | status     |
|    .       |   |     | maxCpus    |   |      | intType    |
|smHeader    |----     | cpuTable   |----      | intArg1    | CPU 0 
|    .       |         |     .      |          | intArg2    | descriptor
|    .       |         |     .      |          | intArg3    |
|smPktHeader |----     |     .      |          |------------|
|    .       |   |     |     .      |          |    .       | CPU 1
|smObjHeader |-- |     --------------          |------------|
|    .       | | |                             |            |
|    .       | | |                             ~            ~
|    .       | | |                             |            | CPU n
-------------- | |                             --------------
               | |   Shared Memory Packet
               | |     Memory Header 
               | |    (SM_PKT_MEM_HDR)
               | ----->--------------                             Free Packets
               |       | heartBeat  |                        ---->-------------
               |       | freeList   |------------------------|    |           |
               |       | pktCpuTbl  |----                         |           |
               |       | maxPktBytes|   |                         |           |
               |       |     .      |   |                         |           |
               |       --------------   |                         |           |
               |                        |                         |           |
               |                        |          per CPU        |           |
               |                        |     Packet Descriptor   |           |
               |                        |      (SM_PKT_CPU_DESC)  -------------
               |                        ------>----------------
               |                               |  status      | CPU 0
               |                               |  inputList   | Packet Desc
               |                               |  freeList    |
               |                               |--------------|
               |                               |      .       | CPU 1
               |                               |      .       |
               |                               |--------------|
               |                               |              |
               |                               ~              ~
               |                               |              | CPU n
               |                               |              |
               |                               ----------------
               |       Shared memory               per CPU
               |       objects header           sm Objects Desc
               |      (SM_OBJ_MEM_HDR)         (SM_OBJ_CPU_DESC)
               ------->--------------   ------>----------------
                       | heartBeat  |   |      |  status      | CPU 0 
                       | initDone   |   |      |  smObjEventQ | Object Desc 
                       | Partitions |   |      |--------------| 
                       |    .       |   |      |      .       | CPU 1 
                       | Name DTB   |   |      |      .       | 
                       | objCpuTbl  |----      |--------------| 
                       |    .       |          |              | 
                       | max values |          |              | 
                       |    .       |          ~              ~ 
                       | statistics |          |              | CPU n
                       --------------          ---------------- 

CONFIGURATION
When the component INCLUDE_SM_OBJ is included, the init and setup
routines in this library are called automatically during VxWorks
initialization.

AVAILABILITY
This module is distributed as a component of the unbundled shared memory
objects support option, VxMP.

INCLUDE FILES: smObjLib.h

SEE ALSO: smObjShow, semSmLib, msgQSmLib, smMemLib, smNameLib, usrSmObjInit(),
\tb VxWorks Programmer's Guide: Shared Memory Objects
*/

/* includes */

#include "vxWorks.h"
#include "errno.h"
#include "intLib.h"
#include "logLib.h"
#include "cacheLib.h"
#include "qFifoGLib.h"
#include "smDllLib.h"
#include "smLib.h"
#include "smUtilLib.h"
#include "semLib.h"
#include "string.h"
#include "stdlib.h"
#include "sysLib.h"
#include "wdLib.h"
#include "private/smNameLibP.h"
#include "private/smMemLibP.h"
#include "private/smObjLibP.h"
#include "private/smFixBlkLibP.h"
#include "private/msgQSmLibP.h"
#include "private/windLibP.h"
#include "private/workQLibP.h"
#include "private/funcBindP.h"
#include "private/eventP.h"

/* defines */

#define PART_OVH      32	/* bytes overhead per partition */

#define SM_OBJ_VERSION 1	/* version number */

/* macro to calculate the size of a shared memory allocated object */

#define SM_OBJ_SIZE(type) (ROUND_UP (sizeof (type), SM_ALIGN_BOUNDARY) \
			   + sizeof (SM_BLOCK_HDR))

/* typedefs */

/* cached CPU Information Structure */

typedef struct sm_obj_cpu_info  /* SM_OBJ_CPU_INFO */
    {
    BOOL             cached;    /* TRUE if info for this CPU is cached */
    SM_CPU_DESC      smCpuDesc;	/* cpu notification info */
    SM_OBJ_EVENT_Q * pCpuEventQ;/* pointer to cpu event queue */
    } SM_OBJ_CPU_INFO;

/* globals */

SM_HDR volatile *         pSmHdr;    /* pointer to shared memory header */
SM_OBJ_MEM_HDR volatile * pSmObjHdr; /* pointer to shared memory obj header */
SM_OBJ_DESC               smObjDesc; /* shared memory object descriptor */

FUNCPTR	smObjTasRoutine;        /* test and set routine */
FUNCPTR smObjTasClearRoutine; 	/* clear routine */
int     smObjSpinTries;         /* maximum # of tries to obtain lock access */
int     smObjProcNum;           /* processor number */
BOOL    smObjTimeoutLog;	/* TRUE : print message if lock take fails */

int smObjInitialized = FALSE;	/* smObj facility not initialized */

SM_FIX_BLK_PART_ID smTcbPartId;	/* shared TCB partition id */

/* locals */

LOCAL SM_DL_LIST       eventList;	/* event list to be processed */
LOCAL SM_OBJ_EVENT_Q * pLocalEventQ;	/* cached local event Q pointer */
LOCAL SM_OBJ_CPU_INFO  smObjCpuInfoTbl [SM_OBJ_MAX_CPU];
					/* smObj cpu info table */ 
LOCAL int              smObjHeartBeatRate; 
					/* heartbeat incrementing rate */
LOCAL WDOG_ID          smObjWdog;	/* watch dog for heartbeat */

/* forward */

LOCAL STATUS  smObjNotifyHandler (void);
LOCAL STATUS  smObjCpuInfoGet (SM_OBJ_DESC * pSmObjDesc, int cpuNum, 
                               SM_OBJ_CPU_INFO * pSmObjCpuInfo);
LOCAL void    smObjBeat (UINT * pHeartBeat); 

extern VOIDFUNCPTR smUtilTasClearRtn;


/******************************************************************************
*
* smObjLibInit - install the shared memory objects facility (VxMP Option)
*
* This routine installs the shared memory objects facility.  It is called
* automatically when the component INCLUDE_SM_OBJ is included.
*
* AVAILABILITY
* This routine is distributed as a component of the unbundled shared memory
* objects support option, VxMP.
* 
* RETURNS: OK, or ERROR if the shared memory objects facility has already been
* installed. 
*/

STATUS smObjLibInit (void)

   {
   if (!smObjInitialized)
	{
   	semSmLibInit ();	/* initialize shared semaphore facility */
   	msgQSmLibInit ();	/* initialize shared message queue facility */
   	smMemPartLibInit ();	/* initialize shared memory manager facility */

	smObjInitialized = TRUE;
	return (OK);
	}

   return (ERROR);
   }

/******************************************************************************
*
* smObjSetup - initialize the shared memory objects facility (VxMP Option)
*
* This routine initializes the shared memory objects facility by filling the
* shared memory header.  It must be called only once by the shared memory
* master CPU.  It is called automatically only by the master CPU, when the
* component INCLUDE_SM_OBJ is included.
*
* Any CPU on the system backplane can use the shared memory objects facility;
* however, the facility must first be initialized on the master CPU.  Then
* before other CPUs are attached to the shared memory area by smObjAttach(), 
* each must initialize its own shared memory objects descriptor using
* smObjInit().  This mechanism is similar to the one used by the shared memory
* network driver.
*
* The <smObjParams> parameter is a pointer to a structure containing the
* values used to describe the shared memory objects setup.  This 
* structure is defined as follows in smObjLib.h:
* \cs
* typedef struct sm_obj_params    /@ setup parameters @/
*     {
*     BOOL        allocatedPool;  /@ TRUE if shared memory pool is malloced @/
*     SM_ANCHOR * pAnchor;        /@ shared memory anchor                   @/
*     char *      smObjFreeAdrs;  /@ start address of shared memory pool    @/
*     int         smObjMemSize;   /@ memory size reserved for shared memory @/
*     int         maxCpus;        /@ max number of CPUs in the system       @/
*     int         maxTasks;       /@ max number of tasks using smObj        @/
*     int         maxSems;        /@ max number of shared semaphores        @/
*     int         maxMsgQueues;   /@ max number of shared message queues    @/