coda.txt

ARM 嵌入式 系统 设计与实例开发 实验教材 二源码

NOTE: 
This is one of the technical documents describing a component of
Coda -- this document describes the client kernel-Venus interface.

For more information:
  http://www.coda.cs.cmu.edu
For user level software needed to run Coda:
  ftp://ftp.coda.cs.cmu.edu

To run Coda you need to get a user level cache manager for the client,
named Venus, as well as tools to manipulate ACLs, to log in, etc.  The
client needs to have the Coda filesystem selected in the kernel
configuration.

The server needs a user level server and at present does not depend on
kernel support.







  The Venus kernel interface
  Peter J. Braam
  v1.0, Nov 9, 1997

  This document describes the communication between Venus and kernel
  level filesystem code needed for the operation of the Coda file sys-
  tem.  This document version is meant to describe the current interface
  (version 1.0) as well as improvements we envisage.
  ______________________________________________________________________

  Table of Contents























































  1. Introduction

  2. Servicing Coda filesystem calls

  3. The message layer

     3.1 Implementation details

  4. The interface at the call level

     4.1 Data structures shared by the kernel and Venus
     4.2 The pioctl interface
     4.3 root
     4.4 lookup
     4.5 getattr
     4.6 setattr
     4.7 access
     4.8 create
     4.9 mkdir
     4.10 link
     4.11 symlink
     4.12 remove
     4.13 rmdir
     4.14 readlink
     4.15 open
     4.16 close
     4.17 ioctl
     4.18 rename
     4.19 readdir
     4.20 vget
     4.21 fsync
     4.22 inactive
     4.23 rdwr
     4.24 odymount
     4.25 ody_lookup
     4.26 ody_expand
     4.27 prefetch
     4.28 signal

  5. The minicache and downcalls

     5.1 INVALIDATE
     5.2 FLUSH
     5.3 PURGEUSER
     5.4 ZAPFILE
     5.5 ZAPDIR
     5.6 ZAPVNODE
     5.7 PURGEFID
     5.8 REPLACE

  6. Initialization and cleanup

     6.1 Requirements


  ______________________________________________________________________
  0wpage

  11..  IInnttrroodduuccttiioonn



  A key component in the Coda Distributed File System is the cache
  manager, _V_e_n_u_s.


  When processes on a Coda enabled system access files in the Coda
  filesystem, requests are directed at the filesystem layer in the
  operating system. The operating system will communicate with Venus to
  service the request for the process.  Venus manages a persistent
  client cache and makes remote procedure calls to Coda file servers and
  related servers (such as authentication servers) to service these
  requests it receives from the operating system.  When Venus has
  serviced a request it replies to the operating system with appropriate
  return codes, and other data related to the request.  Optionally the
  kernel support for Coda may maintain a minicache of recently processed
  requests to limit the number of interactions with Venus.  Venus
  possesses the facility to inform the kernel when elements from its
  minicache are no longer valid.

  This document describes precisely this communication between the
  kernel and Venus.  The definitions of so called upcalls and downcalls
  will be given with the format of the data they handle. We shall also
  describe the semantic invariants resulting from the calls.

  Historically Coda was implemented in a BSD file system in Mach 2.6.
  The interface between the kernel and Venus is very similar to the BSD
  VFS interface.  Similar functionality is provided, and the format of
  the parameters and returned data is very similar to the BSD VFS.  This
  leads to an almost natural environment for implementing a kernel-level
  filesystem driver for Coda in a BSD system.  However, other operating
  systems such as Linux and Windows 95 and NT have virtual filesystem
  with different interfaces.

  To implement Coda on these systems some reverse engineering of the
  Venus/Kernel protocol is necessary.  Also it came to light that other
  systems could profit significantly from certain small optimizations
  and modifications to the protocol. To facilitate this work as well as
  to make future ports easier, communication between Venus and the
  kernel should be documented in great detail.  This is the aim of this
  document.

  0wpage

  22..  SSeerrvviicciinngg CCooddaa ffiilleessyysstteemm ccaallllss

  The service of a request for a Coda file system service originates in
  a process PP which accessing a Coda file. It makes a system call which
  traps to the OS kernel. Examples of such calls trapping to the kernel
  are _r_e_a_d_, _w_r_i_t_e_, _o_p_e_n_, _c_l_o_s_e_, _c_r_e_a_t_e_, _m_k_d_i_r_, _r_m_d_i_r_, _c_h_m_o_d in a Unix
  context.  Similar calls exist in the Win32 environment, and are named
  _C_r_e_a_t_e_F_i_l_e_, .

  Generally the operating system handles the request in a virtual
  filesystem (VFS) layer, which is named I/O Manager in NT and IFS
  manager in Windows 95.  The VFS is responsible for partial processing
  of the request and for locating the specific filesystem(s) which will
  service parts of the request.  Usually the information in the path
  assists in locating the correct FS drivers.  Sometimes after extensive
  pre-processing, the VFS starts invoking exported routines in the FS
  driver.  This is the point where the FS specific processing of the
  request starts, and here the Coda specific kernel code comes into
  play.

  The FS layer for Coda must expose and implement several interfaces.
  First and foremost the VFS must be able to make all necessary calls to
  the Coda FS layer, so the Coda FS driver must expose the VFS interface
  as applicable in the operating system. These differ very significantly
  among operating systems, but share features such as facilities to
  read/write and create and remove objects.  The Coda FS layer services
  such VFS requests by invoking one or more well defined services
  offered by the cache manager Venus.  When the replies from Venus have
  come back to the FS driver, servicing of the VFS call continues and
  finishes with a reply to the kernel's VFS. Finally the VFS layer
  returns to the process.

  As a result of this design a basic interface exposed by the FS driver
  must allow Venus to manage message traffic.  In particular Venus must
  be able to retrieve and place messages and to be notified of the
  arrival of a new message. The notification must be through a mechanism
  which does not block Venus since Venus must attend to other tasks even
  when no messages are waiting or being processed.






                     Interfaces of the Coda FS Driver

  Furthermore the FS layer provides for a special path of communication
  between a user process and Venus, called the pioctl interface. The
  pioctl interface is used for Coda specific services, such as
  requesting detailed information about the persistent cache managed by
  Venus. Here the involvement of the kernel is minimal.  It identifies
  the calling process and passes the information on to Venus.  When
  Venus replies the response is passed back to the caller in unmodified
  form.

  Finally Venus allows the kernel FS driver to cache the results from
  certain services.  This is done to avoid excessive context switches
  and results in an efficient system.  However, Venus may acquire
  information, for example from the network which implies that cached
  information must be flushed or replaced. Venus then makes a downcall
  to the Coda FS layer to request flushes or updates in the cache.  The
  kernel FS driver handles such requests synchronously.

  Among these interfaces the VFS interface and the facility to place,
  receive and be notified of messages are platform specific.  We will
  not go into the calls exported to the VFS layer but we will state the
  requirements of the message exchange mechanism.

  0wpage

  33..  TThhee mmeessssaaggee llaayyeerr



  At the lowest level the communication between Venus and the FS driver
  proceeds through messages.  The synchronization between processes
  requesting Coda file service and Venus relies on blocking and waking
  up processes.  The Coda FS driver processes VFS- and pioctl-requests
  on behalf of a process P, creates messages for Venus, awaits replies
  and finally returns to the caller.  The implementation of the exchange
  of messages is platform specific, but the semantics have (so far)
  appeared to be generally applicable.  Data buffers are created by the
  FS Driver in kernel memory on behalf of P and copied to user memory in
  Venus.

  The FS Driver while servicing P makes upcalls to Venus.  Such an
  upcall is dispatched to Venus by creating a message structure.  The
  structure contains the identification of P, the message sequence
  number, the size of the request and a pointer to the data in kernel
  memory for the request.  Since the data buffer is re-used to hold the
  reply from Venus, there is a field for the size of the reply.  A flags
  field is used in the message to precisely record the status of the
  message.  Additional platform dependent structures involve pointers to
  determine the position of the message on queues and pointers to
  synchronization objects.  In the upcall routine the message structure
  is filled in, flags are set to 0, and it is placed on the _p_e_n_d_i_n_g
  queue.  The routine calling upcall is responsible for allocating the
  data buffer; its structure will be described in the next section.

  A facility must exist to notify Venus that the message has been
  created, and implemented using available synchronization objects in
  the OS. This notification is done in the upcall context of the process
  P. When the message is on the pending queue, process P cannot proceed
  in upcall.  The (kernel mode) processing of P in the filesystem
  request routine must be suspended until Venus has replied.  Therefore
  the calling thread in P is blocked in upcall.  A pointer in the
  message structure will locate the synchronization object on which P is
  sleeping.

  Venus detects the notification that a message has arrived, and the FS
  driver allow Venus to retrieve the message with a getmsg_from_kernel
  call. This action finishes in the kernel by putting the message on the
  queue of processing messages and setting flags to READ.  Venus is
  passed the contents of the data buffer. The getmsg_from_kernel call
  now returns and Venus processes the request.

  At some later point the FS driver receives a message from Venus,
  namely when Venus calls sendmsg_to_kernel.  At this moment the Coda FS
  driver looks at the contents of the message and decides if:


  +o  the message is a reply for a suspended thread P.  If so it removes
     the message from the processing queue and marks the message as
     WRITTEN.  Finally, the FS driver unblocks P (still in the kernel
     mode context of Venus) and the sendmsg_to_kernel call returns to
     Venus.  The process P will be scheduled at some point and continues
     processing its upcall with the data buffer replaced with the reply
     from Venus.

  +o  The message is a _d_o_w_n_c_a_l_l.  A downcall is a request from Venus to
     the FS Driver. The FS driver processes the request immediately
     (usually a cache eviction or replacement) and when it finishes
     sendmsg_to_kernel returns.

  Now P awakes and continues processing upcall.  There are some
  subtleties to take account of. First P will determine if it was woken
  up in upcall by a signal from some other source (for example an
  attempt to terminate P) or as is normally the case by Venus in its
  sendmsg_to_kernel call.  In the normal case, the upcall routine will
  deallocate the message structure and return.  The FS routine can proceed
  with its processing.







                      Sleeping and IPC arrangements

  In case P is woken up by a signal and not by Venus, it will first look
  at the flags field.  If the message is not yet READ, the process P can
  handle its signal without notifying Venus.  If Venus has READ, and
  the request should not be processed, P can send Venus a signal message
  to indicate that it should disregard the previous message.  Such
  signals are put in the queue at the head, and read first by Venus.  If
  the message is already marked as WRITTEN it is too late to stop the
  processing.  The VFS routine will now continue.  (-- If a VFS request
  involves more than one upcall, this can lead to complicated state, an
  extra field "handle_signals" could be added in the message structure
  to indicate points of no return have been passed.--)



  33..11..  IImmpplleemmeennttaattiioonn ddeettaaiillss

  The Unix implementation of this mechanism has been through the
  implementation of a character device associated with Coda.  Venus
  retrieves messages by doing a read on the device, replies are sent
  with a write and notification is through the select system call on the
  file descriptor for the device.  The process P is kept waiting on an
  interruptible wait queue object.

  In Windows NT and the DPMI Windows 95 implementation a DeviceIoControl
  call is used.  The DeviceIoControl call is designed to copy buffers
  from user memory to kernel memory with OPCODES. The sendmsg_to_kernel
  is issued as a synchronous call, while the getmsg_from_kernel call is
  asynchronous.  Windows EventObjects are used for notification of
  message arrival.  The process P is kept waiting on a KernelEvent
  object in NT and a semaphore in Windows 95.

  0wpage

  44..  TThhee iinntteerrffaaccee aatt tthhee ccaallll lleevveell


  This section describes the upcalls a Coda FS driver can make to Venus.
  Each of these upcalls make use of two structures: inputArgs and
  outputArgs.   In pseudo BNF form the structures take the following
  form:


  struct inputArgs {
      u_long opcode;
      u_long unique;     /* Keep multiple outstanding msgs distinct */
      u_short pid;                 /* Common to all */
      u_short pgid;                /* Common to all */
      struct CodaCred cred;        /* Common to all */

      <union "in" of call dependent parts of inputArgs>
  };

  struct outputArgs {
      u_long opcode;
      u_long unique;       /* Keep multiple outstanding msgs distinct */
      u_long result;

      <union "out" of call dependent parts of inputArgs>
  };



  Before going on let us elucidate the role of the various fields. The
  inputArgs start with the opcode which defines the type of service
  requested from Venus. There are approximately 30 upcalls at present
  which we will discuss.   The unique field labels the inputArg with a
  unique number which will identify the message uniquely.  A process and
  process group id are passed.  Finally the credentials of the caller
  are included.

  Before delving into the specific calls we need to discuss a variety of
  data structures shared by the kernel and Venus.