admin_guide.txt

openPBS的开放源代码

        OpenPBS (Portable Batch System) v2.3 Software License

Copyright (c) 1999-2000 Veridian Information Solutions, Inc.
All rights reserved.

---------------------------------------------------------------------------
For a license to use or redistribute the OpenPBS software under conditions
other than those described below, or to purchase support for this software,
please contact Veridian Systems, PBS Products Department ("Licensor") at:

   www.OpenPBS.org  [phone]                  [e-mail]
                       877 902-4PBS (US toll-free)
---------------------------------------------------------------------------

This license covers use of the OpenPBS v2.3 software (the "Software") at
your site or location, and, for certain users, redistribution of the
Software to other sites and locations.  Use and redistribution of
OpenPBS v2.3 in source and binary forms, with or without modification,
are permitted provided that all of the following conditions are met.
After December 31, 2001, only conditions 3-6 must be met:

1. Commercial and/or non-commercial use of the Software is permitted
   provided a current software registration is on file at www.OpenPBS.org.
   If use of this software contributes to a publication, product, or
   service, proper attribution must be given; see www.OpenPBS.org/credit.html

2. Redistribution in any form is only permitted for non-commercial,
   non-profit purposes.  There can be no charge for the Software or any
   software incorporating the Software.  Further, there can be no
   expectation of revenue generated as a consequence of redistributing
   the Software.

3. Any Redistribution of source code must retain the above copyright notice
   and the acknowledgment contained in paragraph 6, this list of conditions
   and the disclaimer contained in paragraph 7.

4. Any Redistribution in binary form must reproduce the above copyright
   notice and the acknowledgment contained in paragraph 6, this list of
   conditions and the disclaimer contained in paragraph 7 in the
   documentation and/or other materials provided with the distribution.

5. Redistributions in any form must be accompanied by information on how to
   obtain complete source code for the OpenPBS software and any
   modifications and/or additions to the OpenPBS software.  The source code
   must either be included in the distribution or be available for no more
   than the cost of distribution plus a nominal fee, and all modifications
   and additions to the Software must be freely redistributable by any party
   (including Licensor) without restriction.

6. All advertising materials mentioning features or use of the Software must
   display the following acknowledgment:

    "This product includes software developed by NASA Ames Research Center,
    Lawrence Livermore National Laboratory, and Veridian Information
    Solutions, Inc.
    Visit www.OpenPBS.org for OpenPBS software support,
    products, and information."

7. DISCLAIMER OF WARRANTY

THIS SOFTWARE IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND. ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT
ARE EXPRESSLY DISCLAIMED.

IN NO EVENT SHALL VERIDIAN CORPORATION, ITS AFFILIATED COMPANIES, OR THE
U.S. GOVERNMENT OR ANY OF ITS AGENCIES BE LIABLE FOR ANY DIRECT OR INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA,
OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE,
EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

This license will be governed by the laws of the Commonwealth of Virginia,
without reference to its choice of law rules.

-----------------------------------------------------------------------------

                       Administrator's Guide
                   DEC Custom PBS Scheduler V.2.3
			      July 2000



This document covers the following information:

	o Introduction
	o Summary of new features in Ver. 2.0
	o Overview of custom scheduler 
	o Installing the custom scheduler
	o Rebuilding PBS to use custom scheduler
	o Required modifications to existing PBS configuration
	o Configuring the custom scheduler
	o Using the new features provided
	o General Comments


Introduction
------------

This package contains the sources for a PBS scheduler (pbs_sched), which
was designed to be run on a cluster of DEC Alpha workstations with
different CPU and memory configurations. The function of the scheduler
is to choose a job or jobs that fit the resources. When a suitable job
is found, the scheduler will ask PBS to run that job on one of the
execution hosts. This scheduler assumes a 1:1 correlation between the
executions queues and execution hosts. The name of the queue is taken
as the name of the host that jobs in that queue should be run in.


Summary of new features in Ver.2.0
--------------------------------------

Version of 2.0 of the custom Dec/Compaq PBS scheduler includes the
following new features. These are discussed in more detail below, and
in the scheduler's configuration file.

	o Fair-Access Controls - Administrator can set per-queue, per-user
	  limits on the maximum number of running jobs and a maximum amount
	  of "remaining" runtime (in minutes) for all jobs owned by a given
	  user.

	o Additional Queue/Job Attributes - The following new attributes
	  have been added to PBS, and are supported by this scheduler:

		featureA (string)
		featureB (string)
		featureC (string)
		featureD (integer)
		featureE (integer)
		featureF (integer)
		featureG (boolean)
		featureH (boolean)
		featureI (boolean)

	o Priority Based Scheduling - Jobs are assigned a priority value
	  based on the priority of the jobs originating queue (the queue
	  to which the job is submitted). Jobs are then sorted by their
	  priority values, ties are broken by the requested cputime.
		

Overview of custom scheduler internals
--------------------------------------

This section provides a high level overview of the workings of the
custom PBS scheduler. The configuration file itself (discussed below)
contains additional information.

* Overview Of Operation

Please be sure to read the section titled 'Configuring The Scheduler'
below before attempting to start the scheduler.

The basic mode of operation for the scheduler is this: 

 - Jobs are submitted to the PBS server by users.  The server enqueues
	them in the default queue (c.f. qmgr(1)). 

 - The scheduler wakes up and performs the following actions:

   + Get a list of jobs from the server.  Typically, the scheduler and
	server are run on the front-end, and only a resmom is needed on
	the execution hosts.  See the section on Scheduler Deployment.

   + Get available resource information from each execution host.  The
	resmom running on each host is queried for a set of resources
	for the host.  Scheduling decisions are made based upon these
	resources, queue limits, time of day (optional), etc, etc.

   + Get information about the queues from the server.  The queues over
	which the scheduler has control are listed in the scheduler's
	configuration files.  The queues may be listed as batch or submit,
	queues.  A job list is then created from the jobs on the submit
	queue(s).

   + If a job fits on a queue and does not violate any policy requirements
	(like primetime walltime limits), ask PBS to move the job to that
	queue, and start it running. If this succeeds, account for the
	expected resource drain and continue. 

   + If the job is not runnable at this time, the job comment will be
	modified to reflect the reason the job was not runnable (see the
	section on Lazy Comments).  Note that this reason may change from
	iteration to iteration, and that there may be several reasons that
	the job is not runnable now.

   + Clean up all allocated resources, and go back to sleep until the next
	round of scheduling is requested.

The scheduler attempts to pack the jobs into the queues as closely as
possible into the queues.  Queues are packed in a "first come, first
served" order.  

The PBS server will wake up the scheduler when jobs arrive or terminate,
so jobs should be scheduled immediately if the resources are (or become) 
available for them.  There is also a periodic run every few minutes.


* The Configuration File

The scheduler's configuration file is a flat ASCII file.  Comments are
allowed anywhere, and begin with a '#' character.  Any non-comment lines
are considered to be statements, and must conform to the syntax :

	<option> <argument>

The descriptions of the options below describe the type of argument that
is expected for each of the options.  Arguments must be one of :

	<boolean>	A boolean value.  The strings "true", "yes", "on" and 
			"1" are all true, anything else evaluates to false.
	<hostname>	A hostname registered in the DNS system.
	<integer>	An integral (typically non-negative) decimal value.
	<pathname>	A valid pathname (i.e. "/usr/local/pbs/pbs_acctdir").
	<queue_spec>	The name of a PBS queue.  Either 'queue@exechost' or
			just 'queue'.  If the hostname is not specified, it
			defaults to the name of the local host machine.
	<real>		A real valued number (i.e. the number 0.80).
	<string>	An uninterpreted string passed to other programs.
	<time_spec>	A string of the form HH:MM:SS (i.e. 00:30:00 for
			thirty minutes, 4:00:00 for four hours).
	<variance>	Negative and positive deviation from a value.  The
			syntax is '-mm%,+nn%' (i.e. '-10%,+15%' for minus
			10 percent and plus 15% from some value).

Syntactical errors in the configuration file are caught by the parser, and
the offending line number and/or configuration option/argument is noted in
the scheduler logs.  The scheduler will not start while there are syntax
errors in its configuration files.

Before starting up, the scheduler attempts to find common errors in the
configuration files.  If it discovers a problem, it will note it in the
logs (possibly suggesting a fix) and exit.

The following is a complete list of the recognized options :

    BATCH_QUEUES			<queue_spec>[,<queue_spec>...]
    ENFORCE_PRIME_TIME			<boolean>
    HIGH_SYSTIME			<integer>
    PRIME_TIME_END			<time_spec>
    PRIME_TIME_START			<time_spec>
    PRIME_TIME_WALLT_LIMIT		<time_spec>
    SCHED_HOST				<hostname>
    SCHED_RESTART_ACTION		<string>
    SUBMIT_QUEUE			<queue_spec>
    TARGET_LOAD_PCT			<integer>
    TARGET_LOAD_VARIANCE		<variance>
    SORTED_JOB_DUMPFILE			<string>
    FAIR_ACCESS 			<access_spec>


Key options are described in greater detail below, the rest are discussed
in the configuration file.

* Queue and Associated Execution Host Definations

The queues on the following lists are ordered from highest scheduling 
priority to lowest.  These are comma separated lists, if more space is 
required, the list can be split into multiple lines.  Each line must be
prefaced by the appropriate configuration option directive.

All queues are associated with a particular execution host.  They may be
specified either as 'queuename' or 'queuename@exechost'.  If only the name
is given, the canonical name of the local host will be automatically
appended to the queue name.

The "normal" scheduling algorithm picks jobs off the SUBMIT_QUEUE and
attempts to run them on the BATCH_QUEUES.  Jobs are enqueued onto the
SUBMIT_QUEUE via the 'qsub' command (set the default queue name in PBS
with the 'set server default_queue' qmgr command), and remain there
until they are rejected, run, or deleted.  The host attached to the
SUBMIT_QUEUE is ignored - it is assumed to be on the server.

Note that this implies that the SUBMIT_QUEUE's resources_max values must
be the union of all the BATCH_QUEUES' resources.

SUBMIT_QUEUE			funnel

BATCH_QUEUES is a list of execution queues onto which the scheduler should
move and run the jobs it chooses from the SUBMIT_QUEUES.  The algorithm used
in the scheduler relies on these queues being arranged from "smallest" to
"largest", as jobs are tested against the list of queues in the order listed,
and run on the queue which first provides enough resources for the job.
 
BATCH_QUEUES			piglet@piglet,evelyn@evelyn

The following options are used to optimize system load average and scheduler 
effectiveness. It is a good idea to monitor system load as the user community 
grows, shrinks, or changes its focus from porting and debugging to 
production. These defaults were selected for a 64 processor system with 16gb 
of memory. 

Target Load Average refers to a target percentage of the maximum system 
load average (1 point for each processor on the machine).  It may vary
as much as the +/- percentages listed in TARGET_LOAD_VARIANCE.  Jobs may
or may not be scheduled if the load is too high or too low, even if the