vnd.par.lib

su 的源代码库

VND - large out-of-core multidimensional block matrix transpose
The VND routines are intended for avoiding the need for virtual
memory when prototyping large out-of-core problems where 
multidimensional matrix transposes are required.  The data are 
stored in a multidimensional block matrix transpose format where 
the dimension of the matrix is user defined.  The data are accessed 
by the user as vectors for any given dimension with all the other 
dimensions fixed or defined as a key.  The vector elements all have 
the same user defined number of bytes per element (or node).  The user
specifies how much memory he would like to allocate for buffering
VND I/O operations.  The VND routines break up the multidimensional
matrix into chunks small enough to fit within this memory limitation
if at all possible.  If not, the routines use the smallest amount
of memory that they can.  If the data fits in memory, then it
is put into memory and no I/O operations are required except to 
save on disk.

A dimension over which the data need not be transposed can be defined as the
panel dimension.  If panels are used, then the file structure becomes
that of a sequence of block matrix transpose structures, one for 
each panel.  Normally, only one panel is stored in memory at a time.
The user has the option to specify 2 or more simultaneously open
panels at the expense of using more memory.  The variable "iopanel"
ranges from 0 to the number of open panels minus 1.  The variable
ipanel" ranges from 0 to the number of panels in the VND files
minus 1.  Two calls to VNDrw with different values of "iopanel"
should not have the same value of "ipanel" unless both are only reading.

There is a preferred dimension for Input/Output efficiency.  That is
the first dimension (idim=0, all of the indicies are zero based).  In
the first dimension, a slice through the block matrix structure can
be read or written in one request.  In all all other dimensions, reading
or writing a slice requires multiple I/O requests.  Therefore, it
is best to choose the first dimension for the dimension with the
most I/O activity.

Real-to-Complex FFT's increase the number of bytes per node and
decrease the number of nodes in a give dimension by a factor
of 2.  To allow this option in dimension 0, the number of nodes
per block in dimension 0  is forced to be even.  This allows
one  to modify the number of bytes per node and number of nodes
in the first dimension to change the data from real to complex
should that be necessary.  This option is not possible for
other than the first dimension.  Clearly, which dimension is 
chosen to be the first is arbitrary other than for considerations
of Input/Output efficiency and this option.

For some applications, it may be desirable to have two dimensions
of size NA and NB in memory at a time to allow rapid changes in
access direction.  Define a single VND dimension with
size NA*NB and access values using the parameters defining starting
element and element increment in VNDrw.  This option keeps user
buffer sizes small and minimizes I/O overhead for some problems.
For implementing the Fowler DMO algorithm, one would use
temporal frequency and velocity together as a function of CMP
position in a single dimension.  This would allow the VND
routines to avoid swapping block matrix slices during the initial 
write of the VND files as the input CMP gathers are stacked for
a range of velocities, Fourier transformed from time to frequency,
and written out to the VND file.  
 
The Unix operating system typically has a 2 gigabyte limit for each 
file system.  When this limit is too small, the VND routines can spread 
the data out over multiple file systems as defined by the user.  Only 
one physical file is actually open at one time to avoid the Unix limitations
on the number of open files allowed.  The code is written in ANSI C and 
should not be limited to the Unix operating system.

The call to VNDop with mode equal to 2 can take a long time as the
first call initializes the file structure to have all zero bytes.
This may involve a significant amount of I/O for large VND files.

The VND user C interface consists of 4 routines:
VND *VNDop(int mode, long lwmax, int ndim, long *N, long npanels, 
		long nbytes, char *file, int ndir, char**dir, int noppanels)
	
	Open block matrix file structure.
	VND *VNDop	returns as pointer to VND I/O control structure
			or NULL if failed
 
	int mode	0 for read only
			1 for read and write (existing file)
			2 for write and read (create new file)
			3 for don't open, just compute buffer sizes 
			and fill out structure
	long lwmax	Maximum memory buffer size in bytes to use
	int ndim	Number of block matrix transpose dimensions
	long N[ndim]	List of number of nodes for each dimension
	long npanels	Number of panels of block matrix structure
	long nbytes	Number of bytes per node
	char *file	File name from which other file names will be 
			constructed.  The actual files used will have
			.VND0, .VND1, ... extentions.  This name
			should not have such an extention.
	int ndir	Number of file directories available for holding data.
			Set ndir=0 to use one file in current directory.
			Set ndir=5 to spread data out over 5 files of
			roughly equal size in the directories dir[j].
			Set ndir=-1 to look at .VND file for directories
			to use.  File format is "number of directories"
			on first line and then one line per directory name.
	char **dir	dir[j] = name of file directory for storing data.
			If dir[0]="/users/junk" and file="file", then the 
			first physical file will be "/users/junk/file.VND0"
	int noppanels	Max number of panels to be open at one time


	VNDop returns a value of NULL if open failed.

void VNDrw(char iop, int iopanel, VND *vnd, int idim, long *key, long ipanel, 
		char *values, long start, long incr, long nvalues,
		int msgnum, char *msg)

	Read or write a vector for dimension "idim" 
	char iop	'r' for read; 'w' for write
	int iopanel	Index of current open panel buffer 
			(Ranges from 0 to noppanels-1.  Used to
			allow access to multiple panels simultaneously.
			For most applications, noppanels = 1 and 
			iopanel will always equal 0.)	
	VND *vnd	Structure holding block matrix file information.
	int idim	Current dimension for read or write
                        (idim=0 is first dimension)
	long key[]	List of node positions for each dimension 
			(Value for dimension "idim" needs to be there
			but it's value is irrelevant.)
	long ipanel	Panel number for read or write.
			(ipanel=0 is first panel)
	char values[]	Returns as values read or input as values to be 
			written.
	long start	starting node (0 based, usually equals 0)
	long incr	node increment (usually 1)
	long nvalues	number of values to read or write 
			(usually same as number of values in 
			the specified dimension)
	int msgnum	A message number to be written out if error occurs.
	char *msg	A message to be written out if error occurs.
void VNDcl(VND *vnd, int mode)

	Close VND file structure.
	VND *vnd	Structure holding VND block matrix file information.
	int mode	0 means keep 
			1 means delete
void VNDflush(VND *vnd)

	Flush out VND buffers to file.  
	This may be important in applications where 
	checkpoint/restart capabilities are needed.  
	After VNDflush, the data written by the VNDrw
	routine are safely stored on disk rather than 
	just in a memory buffer where it could be lost in
	the event of a machine failure.

	VND *vnd	VND control structure

When doing real to complex FFT's, the following routines may
be helpful.
void VNDr2c(VND *vnd)

	Convert dimension 0 from real to complex by scaling
	the number of bytes/node by 2 and halving the
	number of nodes.  (Note: only works for dimension 0)
void VNDc2r(VND *vnd)

	Convert dimension 0 from complex to real by dividing
	the number of bytes/node by 2 and doubling the
	number of nodes.  (Note: only works for dimension 0)
When working in only 2 dimensions, the following routines may
be convenient as they have fewer parameters and don't require
the user to allocate arrays for N or key.  These routines
call the VND routines and are completely compatible.  A FORTRAN
equivalent routine exists with the same calling sequence for
all but the V2Dop routine.  The open routine assumes that
a single VND file can be opened in the current directory.
VND * V2Dop(int mode, long lwmax, long nbytes, char *file,long n0, long n1)

	Open 2-D VND file structure.
	int mode	0 for read only
			1 for read and write (existing file)
			2 for write and read (create new file)
			3 for don't open, just compute buffer sizes 
			and fill out structure
	long lwmax	Maximum memory buffer size in bytes to use
	long nbytes	Number of bytes per node
	char *file	File name from which VND file name will be 
			constructed.  The actual file used will have
			.VND0 extention.  This name
			should not have such an extention.
	long n0		number of nodes in dimension 0
	long n1		number of nodes in dimension 1
void V2Dr0(VND *vnd,long key,char *buf,int msgnum)	read dimension 0
void V2Dr1(VND *vnd,long key,char *buf,int msgnum)	read dimension 1
void V2Dw0(VND *vnd,long key,char *buf,int msgnum)	write dimension 0
void V2Dw1(VND *vnd,long key,char *buf,int msgnum)	write dimension 1
Memory management within the VND routines is based upon
the following routines.  These routines, when used together,
provide automatic error checking at the end of a job
during the VNDfree() step for any buffer overruns telling
the user if an overrun occured at the beginning of the
buffer or at the end.  To ensure that memory overruns
have not occured, a few extra bytes of information must
be stored for each buffer that is not available to the
user program.  Therefore, memory allocated
by VNDemalloc() must be freed by VNDfree() rather than
the standard free() routine.  It is sometimes convenient
to find out the total amount of currently allocated
VND memory.  A routine exists for that purpose.  At the
end of a job, you can ensure that all allocated VND 
buffers have been freed and checked for overruns by
seeing if the total allocated memory at that point
is zero.
void *VNDemalloc(size_t n,char *msg)

	Allocate memory and return a pointer.  
	Give error message and abort if fail.
	size_t n	number of bytes of memory to allocate
	char *msg	error message to print if abort
void VNDmemchk( void *p , char * msg)			

	check for memory overrun and abort if error found
	(can distinguish between overrun at beginning of
	allocated buffer versus overrun at end of allocated
	buffer)
	void *p		pointer returned from VNDemalloc
	char *msg	error message to print out if error found
long VNDtotalmem()

	return total current memory allocated by VNDemalloc
	in characters
void VNDfree( void *p, char *msg)

	free memory allocated by VNDemalloc
	(also calls VNDmemchk to check for memory 
	overflows during job)
	void *p		pointer returned from VNDemalloc
	char *msg	error message to print out if error found

	
The following routine returns a unique temporary file
root name for VND files. VNDtempname() 
returns a name made up from the input root string, the
process id, and a unique integer.  This is the only VND
routine calling a UNIX specific function or using the
header files <sys/types.h> or <sys/errno.h>.  The other
routines should be portable to non-Unix systems.
char *VNDtempname(char *root)

Memory associated with a VNDtempname() character string
must be freed using VNDfree() rather than free().
A short test program might check the VND_HOME file for
a .VND file for file systems over which to spread the data.
This has been accomplished by coding "ndir=-1" in VNDop.
A 3 dimensional 50x50x5 block matrix system is set up over
1 panel. The user wishes to use a memory buffer of 1000 
integers. The data are initialized to equal the first dimension
index and written out in vectors along the first dimension (idim=0).
Then, the data are then read from the file along vectors in 
the second dimension (idim=1) and dumped to check and see if 
they are correct.  Note that it would have required memory the
size of 12500 integers to hold the entire 3-D matrix in memory.

include "VND.h"
int main(){
	int i;
	int j;      
	int k;      
	static long N[3]={50,50,5};
	long key[3];
	long buf[50];  
	VND *vnd;
	char **dir;

	fprintf(stdout,"Open the VND file and initialize values to 0\n");
	vnd=VNDop(2,9999,3,N,1,sizeof(long),"george",-1,dir,1);
	VNDdump(vnd,stdout);

	fprintf(stdout,"Write out data in first dimension slices\n");
	fprintf(stdout,
   		"Values = i+j*100+k*10000 where i,j,k are the keys\n");
	fprintf(stdout,"in the first, second, and third dimensions\n");
	for(k=0;k<N[2];k++) {
 		key[2]=k;
   		for(j=0;j<N[1];j++) {
			key[1]=j;