filesys.c

应用平台 UNIX含源程序和SCO OpenServer 5下运行的目标文件。

/*
Howdy folks.  We've been hacking text databases here on the New OED project
for years, and have been disappointed at the performance of Unix filesystems
in general, and also at the tools available for measuring that performance.

So I wrote a program to address this need.  It fails to meet the SPEC
criterion of being real working code that does something useful, but it does
produce very interesting numbers, most of which I believe.

This posting includes a summary of the results of running the benchmark on all
the machines I could get my hands on, some subjective commentary, and the
source code for the benchmark, the comments in which should be read by
everyone before beginning to think about believing these numbers.

Cheers, Tim Bray, Open Text Systems, Waterloo, Ont.

           -------Sequential Output-------- ---Sequential Input-- --Random-
           -Per Char- --Block--- -Rewrite-- -Per Char- --Block--- --Seeks--
           K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU K/sec %CPU /sec %CPU
 
DEC 5810     330 73.6   690 62.8   195 29.0   274 61.6   649 45.9 27.7  4.9
             434 92.7   732 63.3   206 29.2   382 80.7   701 47.6 34.3  5.1
             418 88.1   718 65.0   196 28.7   386 81.7   744 52.3 27.1  5.8

MipsM/2K     316 35.2   381  8.8   302 11.4   611 67.9  1369 27.4 28.5  1.2
             345 38.1   435 10.0   226  8.8   399 44.1   876 17.3 21.3  1.3
             362 40.0   440 10.5   371 15.0   785 89.1  1762 37.3 26.4  1.8

NeXT         243 92.0   352 48.8   210 38.9   217 95.0   782 72.4 28.4 12.7
             243 92.9   360 50.0   216 40.7   251 95.8   773 71.4 28.7 14.3
             245 93.1   346 48.2   215 40.3   218 95.7   777 71.3 26.7 13.7

Seq.Symm.    178 99.1   951 90.7   307 48.1   151 95.7   694 57.2 35.6  6.6
             176 98.2   926 85.8   285 43.4   150 95.4   655 51.5 33.4  6.7

VAX 8650     197 81.7   262 22.3   116 14.1   255 81.5   434 16.6 22.1  2.5
             186 77.7   214 18.6    99 12.7   144 46.9   361 15.1 16.8  2.2

Sun 4/260    298 83.2   791 58.2   296 26.3   312 94.5   840 44.8 ****
             328 92.6   796 59.7   288 26.4   298 91.2   726 39.7 ****

386          229 91.9   241 27.5   207 46.4   209 91.9   437 56.0 34.4  7.0

IBM R 6000   654 89.0  1178 22.6   434 13.1   750 94.2  1539 20.9 39.6  3.0
             712 96.5  1258 24.1   444 13.4   727 91.3  1546 21.7 33.7  2.3

Commentary: feel free to disbelieve

DEC 5810: Ultrix 3.1C-0 (Rev 42), 1.2G filesystem on disk, 91% full.
 The R2000 in here is slower than the R3000 in the Mips M/2K, but the
 I/O rates are competitive; however this box has to burn more CPU to
 achieve those rates.  Note that block input and output are about the
 same speed, making it slower than the MIPS on input but faster on
 output.  

Mips M/2000: UMIPS 4.0, 564M filesystem on MIPS disk, 83% full.
 This is a very fast CPU, and it shows in the %CPU columns.  The
 block-mode input number is very good - over 1.25 effective
 megabytes/sec off the disk.  Note the block-in vs. block-out anomaly 
 with respect to the DEC machine.  Also, the seeks/sec. are unexciting.

NeXT: SW Version 1.0, 446M filesystem on NeXT HD, 82% full.
 One of the slower CPUs in the crowd, and it shows.  Given its price
 relative to some of these other boxes, this is excellent performance;
 the NeXT is supposed to have fancy DMAs.  Didn't have an optical I
 could write on, but that would be very interesting.

Sequent: 10 386 processors, Dynix 3.0.17.9, 564M filesystem, 82% full.
 The individual processors in here are fairly-slow 386es, and that shows
 in the %CPU column.  But burning 100% of a 386 to do the filesystem and
 leaving the other 9 to do useful work seems smart to me.  The block I/O
 throughput and in particular the seeks/second are very good.  Looks
 like a good database machine. 

VAX 8650: 4.3bsd, 584M filesystem on CDC Wren VI on UDA, 33% full.
 Well, the myth of VAX as good I/O machine seems to be biting the dust
 here.  An 8650 is a pretty fast CPU, and has an elaborate I/O
 architecture.  There must be a bottleneck here - candidates are the
 UDA, the Unibus, and 4.3bsd.  Note the %CPU figures are very
 low, which may just be a result of the I/O being so slow, or some CISC
 artifact.  It's even slow at seeking.  Seven years ago, we used to have
 some 780s with RP07's on MASSBUSes - I wouldn't be surprised if they
 measured faster than this.

Sun 4/260: SunOS 4.0.3, 526M filesystem on Fuji M2344, 87% full.
 Not bad, consistently in the top half.  Some of the speed seems to be 
 purchased by burning CPU - the %CPU figures are quite high.  Couldn't
 get meaningful numbers on seeks/sec because we don't have the disk
 space to spare.  With 32M of memory, operating on a 50M file, the
 caching drove the seeks/sec up over 60.
 
386: 20 MHz AMI motherboard, 386/ix V2.0.2, 180M filesys on MaxStor ESDI
     disk, 51% full.
 I think this is pretty damn good performance for a machine which is
 an order of magnitude less in cost than some of the other boxes here.
 In particular, its random I/O performance seems to rival that of the
 big RISC chips, both the DEC and MIPS implementations (8M memory for a
 50M file, so it ain't caching much).  The scuttlebutt says that these
 can be made still faster using caching controllers. 

IBM R6000, Beta Test AIX, 450M filesystem on "Cheap SCSI", 34% full.
 Was never able to run the tests on an idle system, something was
 burning CPU all the time, if vmstat(1) is to be believed, but I
 couldn't find a candidate using ps(1) - except perhaps something called
 "kproc", but that sounds like a fake kernel task.  Also, there was only
 about 50 Mb free on the filesystem, so the sysadmin did something with
 "logical volumes" and the "chfs(1)" command then the filesys was 250M
 bigger - might this be muddying the waters?  A test run before the chfs
 showed *unbelievable* rates, like 5+ Mb system coming off the filesys
 using read(2).  Anyhow, the conclusion is  clear - this puppy is an 
 I/O monster.
*/

/*
 *
 * fsx is a File System eXerciser.  It is a benchmark that attempts to
 * quantify the performance of several filesystem operations that have been
 * observed to be bottlenecks in I/O-intensive applications, specifically
 * the text database work done in connection with the New Oxford English
 * Dictionary Project at the University of Waterloo.
 * 
 * It performs a series of tests on a file of known size.  By default, that
 * size is 50 Mb.  For each test, fsx reports the bytes processed per
 * elapsed second, per CPU second, and the % CPU usage (user and system).
 * 
 * In each case, some care is taken to keep optimizers, no matter how
 * clever, from noticing it's all bogus.  The idea is to make sure that
 * these are real transfers from user space to the physical disk.  The
 * tests are:
 * 
 * 1. Sequential Output
 * 
 * 1.1 Per-Character.  The file is written using the putc() stdio macro.
 * The loop that does the writing should be small enough to fit into any
 * reasonable I-cache.  The CPU exercised here is that required to do the
 * stdio code plus the OS file space allocation.
 * 
 * 1.2 Block.  The file is created using write(2).  The CPU consumption
 * should be just the OS file space allocation.
 * 
 * 1.3 Rewrite.  Each BUFSIZ of the file is read with read(2), dirtied, and
 * rewritten with write(2), requiring an lseek(2).  Since no space
 * allocation is done, and the I/O is well-localized, this should test the
 * effectiveness of the filesystem cache and the speed of data transfer.
 * 
 * 2. Sequential Input
 * 
 * 2.1 Per-Character.  The file is read using the getc() stdio macro.  Once
 * again, the inner loop is small.  This should exercise only stdio and
 * sequential input.
 * 
 * 2.2 Block.  The file is read using read(2).  This should be a very pure
 * test of sequential input performance.
 * 
 * 3. Random Seeks
 * 
 * This test seeks 1000 times to locations in the file specified by
 * random() in bsd systems, drand48() on sysV systems.  In each case, the
 * block is read with read(2).  In 10% of cases, it is dirtied and written
 * back with write(2).
 * 
 * AXIOM: For any unix filesystem, the effective number of lseek(2) calls
 * per second declines asymptotically to near 30, once the effect of
 * caching is defeated.
 * 
 * The size of the file has a strong nonlinear effect on the results of
 * this test.  Many Unix systems that have the memory available will make
 * aggressive efforts to cache the whole thing, and report random I/O rates
 * in the thousands per second, which is ridiculous.  As an extreme
 * example, an IBM RISC 6000 with 64 Mb of memory reported 3,722 per second
 * on a 50 Mb file.
 *
 * COPYRIGHT NOTICE: 
 * Copyright (c) Tim Bray, 1990.
 * Everybody is hereby granted rights to use, copy, and modify this program, 
 *  provided only that the copyright notice above and the disclaimer below
 *  are preserved without change.
 * DISCLAIMER:
 * This program is provided AS IS with no warranty of any kind, and
 * The author makes no representation with respect to the adequacy of this
 *  program for any particular purpose or with respect to its adequacy to 
 *  produce any particular result, and
 * The author shall not be liable for loss or damage arising out of
 *  the use of this program regardless of how sustained, and
 * In no event shall the author be liable for special, direct, indirect
 *  or consequential damage, loss, costs or fees or expenses of any
 *  nature or kind.
 */

#include <stdio.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/time.h>
#ifdef i386
#include <limits.h>
#include <sys/times.h>
#else
#include <sys/resource.h>
#endif

#define IntSize (4)
#define Elapsed (0)
#define CPU (1)
#define Searches (1000)
#define UpdateSeek (10)

static double cpu_so_far();
static void   doseek();
static void   get_delta_t();
static void   io_error();
static void   newfile();
static void   report();
static double time_so_far();
static void   timestamp();
static void   usage();
#ifdef i386
static long   random();
#endif

typedef enum
{
  Putc,
  ReWrite,
  FastWrite,
  Getc,
  FastRead,
  Binary,
  TestCount
} tests_t;

static int    basetime;
static double delta[(int) TestCount][2];
static char * myname;
static double last_cpustamp = 0.0;
static double last_timestamp = 0.0;
static int    lseek_count = 0;

main(argc, argv)
  int    argc;
  char * argv[];
{
  int    buf[BUFSIZ / IntSize];
  int    bufindex;
  int    chars[256];
  char * dir;
  int    fd;
  char   name[BUFSIZ];
  int    next;
  int    size;
  FILE * stream;
  int    words;

  myname = argv[0];
  fd = -1;
  basetime = (int) time((time_t *) NULL);
  size = 50;
  dir = ".";

  for (next = 1; next < argc - 1; next++)
    if (argv[next][0] == '-')
    { /* option? */
      if (strcmp(argv[next] + 1, "d") == 0)
	dir = argv[next + 1];
      else if (strcmp(argv[next] + 1, "s") == 0)
	size = atoi(argv[next + 1]);
      else
	usage();
      next++;
    } /* option? */
    else
      usage();

  if (size < 1)
    usage();
  size *= (1024 * 1024);
  sprintf(name, "%s/fsx.%d", dir, getpid());
  printf("File '%s', size: %d\n", name, size);
  fflush(stdout);

  /*
   * Fill up a file, writing it a char at a time with the stdio putc()
   *  facility.  This models the activity of a negligible-CPU filter
   *  and exercises kernel file space allocation.  Should be measured with
   *  disks of varying degrees of fullness.
   */
  printf("Writing with putc()...");
  newfile(name, &fd, &stream, 1);
  fflush(stdout);
  timestamp();
  for (words = 0; words < size; words++)
    if (putc(words & 0x7f, stream) == EOF)
      io_error("putc");
  
  /*
   * note that we always close the file before measuring time, in an
   *  effort to force as much of the I/O out as we can
   */
  if (fclose(stream) == -1)
    io_error("fclose after putc");
  get_delta_t((int) Putc);
  printf("done\n");
  fflush(stdout);

  /*
   * Now read & rewrite it, a block at a time.  Change one word in each block
   *  in case somebody's being clever about rewritten dirty blocks.
   * This exercises sequential read-write performance and hence the cache,
   *  but avoids any space-allocation work.
   */
  newfile(name, &fd, &stream, 0);
  if (lseek(fd, (off_t) 0, 0) == (off_t) -1)
    io_error("lseek(2) before rewrite");
  printf("Rewriting...");
  fflush(stdout);
  timestamp();
  bufindex = 0;
  do
  { /* while we can read a block */
    if ((words = read(fd, (char *) buf, BUFSIZ)) == -1)
      io_error("read(2)");
    if (bufindex == words / IntSize)
      bufindex = 0;
    buf[bufindex++]++;
    if (lseek(fd, (off_t) -words, 1) == -1)