rfc369.txt

来自「RFC 的详细文档！」· 文本 代码 · 共 620 行 · 第 1/2 页

Network Working Group                                         J. Pickens
Request for Comments: 369               UCSB COMPUTER SYSTEMS LABORATORY
NIC: 6801                                                   25 July 1972


                     EVALUATION OF ARPANET SERVICES
                      January through March, 1972

ABSTRACT

   RFC #302, Exercising the ARPANET, described a group organized at UCSB
   to investigate the network resources.  The stated goals were to
   develop problem solving capability and, in the process, produce
   helpful criticism for the nodes investigated.  This report summarizes
   the group's experiences and finding and suggests network refinements
   to improve user satisfaction.

   The group's encounter with ARPANET included many unexpected problems
   and difficulties.  Most worthy of mention are software heterogeneity
   and inadequate documentation.

   From this first hand experience the group has formulated criteria for
   ease in use of network resources.  The report presents these criteria
   as well as suggestions for improved documentation, better utilization
   of current resources, and a plea for regular usage of inter-personal
   communications facilities.  Individual sites have been graded on
   reliability, response, and friendliness.  Comments regarding specific
   sites have been included to help in adapting to the needs of
   uninitiated users.

   Despite problems encountered in the initial nine week exposure,
   enough was learned of ARPANET resources to enable the group to write
   useful software.  Programs to effect automatic login, file transfer,
   and interprocess communication have been written and put to use.

TABLE OF CONTENTS

      BACKGROUND
         Approach.......................................  2
         Goals..........................................  2
      THE SURVEY
         Extent and Duration............................  3
         Statistical Results............................  3
      CRITIQUE OF ARPANET SERVICES
         A Site Measurement Parameter, "Friendliness"...  4
         Software Critique..............................  5
         Community Spirit...............................  5
         Economics......................................  6



Pickens                                                         [Page 1]

RFC 369              EVALUATION OF ARPANET SERVICES            July 1972


      SUGGESTIONS FOR IMPROVEMENT
         Software.......................................  6
         Community Spirit...............................  7
      CONCLUSION........................................  8
      APPENDIX A
         Sample of Survey Questionnaire................   9
      APPENDIX B
         Grades and Comments for Specifics Sites.......  10


BACKGROUND

Approach

   The test group was organized from a group of Electrical Engineering
   graduate students in Computer Science.  Within the group was
   represented a substantial degree of experience with high level
   languages and time sharing systems (such as the Dartmouth BASIC and
   UCSB mathematical graphics systems).  However, no one had experience
   in exercising ARPANET, and few knew what resources the ARPANET
   represented.  After two weeks of presentation from Jim White and
   Roland Bryan, the group was turned loose for open experimentation.

   Enthusiasm was high as each group managed to locate and decode the
   login procedures for various nodes and began to learn how to use the
   available resources.  In fact, half of the weekly seminar time was
   devoted to sharing learned experiences and procedures.  Interest,
   however, lagged some as the quarter progressed due to poor network
   site reliability, few active nodes, and hard to locate documentation
   (only five out of fourteen students remained active after the first
   quarter).

Goals

   The primary goal of the group was to learn how to use and to evaluate
   network resources.  It was decided to be fair but direct in
   evaluating each site, including UCSB.  Since the level of networking
   experience was initially low, the evaluation criteria was dictated
   mostly by gut feelings.

   At the conclusion of the first quarter's effort, a questionnaire was
   given to the students (a sample of which is included in Appendix A).









Pickens                                                         [Page 2]

RFC 369              EVALUATION OF ARPANET SERVICES            July 1972


   The group response is summarized for overall performance below.  Data
   for individual sites is presented in Appendix B. Some of the
   questions asked were the following:

      Estimate percentage of time spent in various trouble states
      Estimate the mean time to failure
      Describe personal experience with the network
      Suggest improvements
      Grade the investigated nodes on the factors of reliability,
      response, and friendliness

THE SURVEY

Extent and Duration

   During the period in which the major effort was expended (January-
   March, 1972) relatively few nodes were active.  Experimentation,
   therefore, concentrated most heavily on UCSB, BBN-TENEX, MIT-MULTICS,
   and SRI-ARC.  Minor investigation was performed of HARV-10, UCLA-NMC,
   and UCLA-CCN.  The remaining sites were either inactive or
   inaccessible for lack of documentation.

   Activity included the following:

      Game playing (e.g., chess, life, and doctor at BBN-TENEX)
      Text and file manipulation (e.g., COL, NLS, TECO)
      Inter-personal communication (LINK and SNDMSG)
      On line compilation (e.g., TENEX FORTRAN, MULTICS PL/1).

Statistical Results

   Figure 1 below summarizes the overall response to the questionnaire
   given to the group after nine weeks experience with the ARPANET.
   Individual exposure varied from ten to sixty hours, and twelve
   students responded.  Each survey item is presented as a group average
   (sum/12) and is supplemented with a low and a high value to show the
   range of response.  The questions were slightly ambiguous in that
   they failed to distinguish between node inactivity and local NCP
   inactivity.  Also, some figures may reflect individual students'
   inadequacy in understanding local and foreign procedures.
   Nevertheless, the data is interesting as a look into uninitiated user
   experience.









Pickens                                                         [Page 3]

RFC 369              EVALUATION OF ARPANET SERVICES            July 1972


Figure 1

   Survey Item                                Average    Low    High

   % of time unable to log in any site         12,4%     2%     25%
   % of time unable to log into desired site   35.7      20     75
   % of time foreign site suddenly crashes     13        5      50
   % of time local site suddenly crashes       12.5      5      25
   % of time trouble free operation            35        0      80
   Approximate mean-time-between-failure       1h       5 min   2 hrs

   TOTAL TIME INVESTED                         32.3hrs  10 hrs  60 hrs

   First to be noted is that considering the entire ARPANET complex, no
   one approximated the mean-time-between-failure at more than two
   hours!  Secondly, the average time for "trouble free" operation was
   35%, a figure untenable for regular user usage.  In all fairness,
   however, some sites were much more "trouble free" than others, and
   individuals tend to define the term by the level of their own
   competence and experience, thus explaining the high of 80% and the
   low of 0%.

CRITIQUE OF ARPANET SERVICES

A Site Measurement Parameter, Friendliness

   Much discussed by the group was the concept of "friendliness",
   especially as it applies to on-line systems.  The following
   definition of friendliness is offered, based on direct network
   experience.

   Friendliness is:

      Concise, complete, and available documentation.
      Easy system usage (e.g., minimum numbers of keys for login
      system and job status readily available).
      Easy to reach help both on-line people and on-line files.
      No messages overkill (as sometimes unexpectedly occurs
      during login).
      Reasonable reliability and response time
      Concise, but informative error diagnostics

   The reader can probably think of more criteria, but these were the
   outstanding points of friendliness generated specifically by the
   group's experience.






Pickens                                                         [Page 4]

RFC 369              EVALUATION OF ARPANET SERVICES            July 1972


Software Critique

   1) Initial experimentation concentrated on login procedures, canned
   scenarios (e.g., Abhay K. Bhushan's ARPANET scenario, RFC #254), game
   playing, and inter-personal communication.  As the effort continued,
   attempts were made to solve problems at various nodes.  One student,
   for example, programmed a Newton-Raphson root finder in PL/1 at MIT-
   MULTICS a blackbody problem in FORTRAN at BBN-TENEX and MIT-MULTICS,
   and in PL/1 at MIT-MULTICS; and a Discrete Fourier Transform in BASIC
   at BBN-TENEX.  It is the group's conclusion that small problems can
   be written in a half hour, entered and edited in fifteen minutes and
   debugged in another fifteen minutes.  For small problems the current
   ARPANET software resources are quite adequate.

   2) By far the most annoying difficulty was obtaining adequate
   documentation.  The resource notebook was found to be interesting but
   of limited utility.

   3) Information about each node's NCP, which was requested in
   February, 1972, is still unavailable.

   4) Significant variations in procedures were found in executing
   similar tasks on different nodes.  Consider, for example, the wide
   variety of text editors with unique file naming, editing, and
   manipulation commands (TENEX, TECO, COL, NLS...).  Consider, too, the
   wide variety of compilation, load and execute procedures (RJE for
   UCSB edit, save, compile, save, load, execute for TENEX systems).
   Even more disparate are the "executive level" commands with all their
   varieties (TENEX's "Control-C", UCLA-NMC's "X", UCSB's "RESET" ...
   all of which return to the "top-lvel").  Software heterogeneity is a
   stumbling block to the user.

   5) Residents of large nodes are hard pressed to find problems which
   should be solved outside of the local environment.  With UCSB's
   mathematical graphics on-line system and direct access to batch, the
   group experienced apprehensive twinges spending hours on the network
   solving problems which could be solved in minutes locally.

Community Spirit

   1) Individuals sometimes got the impression (erroneously it is hoped)
   that some researchers in the ARPA community had little desire to
   consult and/or help.  On the other hand, others bent over backwards
   in giving assistance.  The group had hoped for a more consistent
   response.






Pickens                                                         [Page 5]

RFC 369              EVALUATION OF ARPANET SERVICES            July 1972


   2) There was difficulty in locating the source of responsibility for
   resource development.  It seemed to the seminar group that the
   complete distribution of responsibility negated incentive to locate,
   document, and create useful network resources.

Economics

   Network economics at levels above as well as the communications
   level, are a big user problem, e.g., if distributed computing is
   allowed, then distributed billing is a necessity.  It is frustrating
   to watch accounts randomly die at different nodes and have to spend
   weeks in monetary renovation.  This problem was experienced with a
   site which (a) randomly changed passwords and then (b) eliminated its
   free account.  Also there is a problem with double connect charges,
   e.g., $4.00 per hour at UCSB to sign on to BBN-TENEX at $8.00 per
   hour, which totals to $12.00 per hour!

SUGGESTIONS FOR IMPROVEMENT

   In spite of the many difficulties and frustrations, the class was
   impressed with the potential of ARPANET and produced several
   suggestions for improvement.

Software