perldebguts.pod

MSYS在windows下模拟了一个类unix的终端

The most significant information in the output is about the particular I<node>
of the compiled regex that is currently being tested against the target string.
The format of these lines is

C<    >I<STRING-OFFSET> <I<PRE-STRING>> <I<POST-STRING>>   |I<ID>:  I<TYPE>

The I<TYPE> info is indented with respect to the backtracking level.
Other incidental information appears interspersed within.

=head1 Debugging Perl memory usage

Perl is a profligate wastrel when it comes to memory use.  There
is a saying that to estimate memory usage of Perl, assume a reasonable
algorithm for memory allocation, multiply that estimate by 10, and
while you still may miss the mark, at least you won't be quite so
astonished.  This is not absolutely true, but may provide a good
grasp of what happens.

Assume that an integer cannot take less than 20 bytes of memory, a
float cannot take less than 24 bytes, a string cannot take less
than 32 bytes (all these examples assume 32-bit architectures, the
result are quite a bit worse on 64-bit architectures).  If a variable
is accessed in two of three different ways (which require an integer,
a float, or a string), the memory footprint may increase yet another
20 bytes.  A sloppy malloc(3) implementation can inflate these
numbers dramatically.

On the opposite end of the scale, a declaration like

  sub foo;

may take up to 500 bytes of memory, depending on which release of Perl
you're running.

Anecdotal estimates of source-to-compiled code bloat suggest an
eightfold increase.  This means that the compiled form of reasonable
(normally commented, properly indented etc.) code will take
about eight times more space in memory than the code took
on disk.

There are two Perl-specific ways to analyze memory usage:
$ENV{PERL_DEBUG_MSTATS} and B<-DL> command-line switch.  The first
is available only if Perl is compiled with Perl's malloc(); the
second only if Perl was built with C<-DDEBUGGING>.  See the
instructions for how to do this in the F<INSTALL> podpage at 
the top level of the Perl source tree.

=head2 Using C<$ENV{PERL_DEBUG_MSTATS}>

If your perl is using Perl's malloc() and was compiled with the
necessary switches (this is the default), then it will print memory
usage statistics after compiling your code when C<< $ENV{PERL_DEBUG_MSTATS}
> 1 >>, and before termination of the program when C<<
$ENV{PERL_DEBUG_MSTATS} >= 1 >>.  The report format is similar to
the following example:

  $ PERL_DEBUG_MSTATS=2 perl -e "require Carp"
  Memory allocation statistics after compilation: (buckets 4(4)..8188(8192)
     14216 free:   130   117    28     7     9   0   2     2   1 0 0
		437    61    36     0     5
     60924 used:   125   137   161    55     7   8   6    16   2 0 1
		 74   109   304    84    20
  Total sbrk(): 77824/21:119. Odd ends: pad+heads+chain+tail: 0+636+0+2048.
  Memory allocation statistics after execution:   (buckets 4(4)..8188(8192)
     30888 free:   245    78    85    13     6   2   1     3   2 0 1
		315   162    39    42    11
    175816 used:   265   176  1112   111    26  22  11    27   2 1 1
		196   178  1066   798    39
  Total sbrk(): 215040/47:145. Odd ends: pad+heads+chain+tail: 0+2192+0+6144.

It is possible to ask for such a statistic at arbitrary points in
your execution using the mstat() function out of the standard
Devel::Peek module.

Here is some explanation of that format:

=over 4

=item C<buckets SMALLEST(APPROX)..GREATEST(APPROX)>

Perl's malloc() uses bucketed allocations.  Every request is rounded
up to the closest bucket size available, and a bucket is taken from
the pool of buckets of that size.

The line above describes the limits of buckets currently in use.
Each bucket has two sizes: memory footprint and the maximal size
of user data that can fit into this bucket.  Suppose in the above
example that the smallest bucket were size 4.  The biggest bucket
would have usable size 8188, and the memory footprint would be 8192.

In a Perl built for debugging, some buckets may have negative usable
size.  This means that these buckets cannot (and will not) be used.
For larger buckets, the memory footprint may be one page greater
than a power of 2.  If so, case the corresponding power of two is
printed in the C<APPROX> field above.

=item Free/Used

The 1 or 2 rows of numbers following that correspond to the number
of buckets of each size between C<SMALLEST> and C<GREATEST>.  In
the first row, the sizes (memory footprints) of buckets are powers
of two--or possibly one page greater.  In the second row, if present,
the memory footprints of the buckets are between the memory footprints
of two buckets "above".

For example, suppose under the previous example, the memory footprints
were

     free:    8     16    32    64    128  256 512 1024 2048 4096 8192
	   4     12    24    48    80

With non-C<DEBUGGING> perl, the buckets starting from C<128> have
a 4-byte overhead, and thus a 8192-long bucket may take up to
8188-byte allocations.

=item C<Total sbrk(): SBRKed/SBRKs:CONTINUOUS>

The first two fields give the total amount of memory perl sbrk(2)ed
(ess-broken? :-) and number of sbrk(2)s used.  The third number is
what perl thinks about continuity of returned chunks.  So long as
this number is positive, malloc() will assume that it is probable
that sbrk(2) will provide continuous memory.

Memory allocated by external libraries is not counted.

=item C<pad: 0>

The amount of sbrk(2)ed memory needed to keep buckets aligned.

=item C<heads: 2192>

Although memory overhead of bigger buckets is kept inside the bucket, for
smaller buckets, it is kept in separate areas.  This field gives the
total size of these areas.

=item C<chain: 0>

malloc() may want to subdivide a bigger bucket into smaller buckets.
If only a part of the deceased bucket is left unsubdivided, the rest
is kept as an element of a linked list.  This field gives the total
size of these chunks.

=item C<tail: 6144>

To minimize the number of sbrk(2)s, malloc() asks for more memory.  This
field gives the size of the yet unused part, which is sbrk(2)ed, but
never touched.

=back

=head2 Example of using B<-DL> switch

Below we show how to analyse memory usage by 

  do 'lib/auto/POSIX/autosplit.ix';

The file in question contains a header and 146 lines similar to

  sub getcwd;

B<WARNING>: The discussion below supposes 32-bit architecture.  In 
newer releases of Perl, memory usage of the constructs discussed
here is greatly improved, but the story discussed below is a real-life
story.  This story is mercilessly terse, and assumes rather more than cursory
knowledge of Perl internals.  Type space to continue, `q' to quit. 
(Actually, you just want to skip to the next section.)

Here is the itemized list of Perl allocations performed during parsing
of this file:

 !!! "after" at test.pl line 3.
    Id  subtot   4   8  12  16  20  24  28  32  36  40  48  56  64  72  80 80+
  0 02   13752   .   .   .   . 294   .   .   .   .   .   .   .   .   .   .   4
  0 54    5545   .   .   8 124  16   .   .   .   1   1   .   .   .   .   .   3
  5 05      32   .   .   .   .   .   .   .   1   .   .   .   .   .   .   .   .
  6 02    7152   .   .   .   .   .   .   .   .   .   . 149   .   .   .   .   .
  7 02    3600   .   .   .   .   . 150   .   .   .   .   .   .   .   .   .   .
  7 03      64   .  -1   .   1   .   .   2   .   .   .   .   .   .   .   .   .
  7 04    7056   .   .   .   .   .   .   .   .   .   .   .   .   .   .   .   7
  7 17   38404   .   .   .   .   .   .   .   1   .   . 442 149   .   . 147   .
  9 03    2078  17 249  32   .   .   .   .   2   .   .   .   .   .   .   .   .


To see this list, insert two C<warn('!...')> statements around the call:

  warn('!');
  do 'lib/auto/POSIX/autosplit.ix';
  warn('!!! "after"');

and run it with Perl's B<-DL> option.  The first warn() will print
memory allocation info before parsing the file and will memorize
the statistics at this point (we ignore what it prints).  The second
warn() prints increments with respect to these memorized data.  This
is the printout shown above.

Different I<Id>s on the left correspond to different subsystems of
the perl interpreter.  They are just the first argument given to
the perl memory allocation API named New().  To find what C<9 03>
means, just B<grep> the perl source for C<903>.  You'll find it in
F<util.c>, function savepvn().  (I know, you wonder why we told you
to B<grep> and then gave away the answer.  That's because grepping
the source is good for the soul.)  This function is used to store
a copy of an existing chunk of memory.  Using a C debugger, one can
see that the function was called either directly from gv_init() or
via sv_magic(), and that gv_init() is called from gv_fetchpv()--which
was itself called from newSUB().  Please stop to catch your breath now.

B<NOTE>: To reach this point in the debugger and skip the calls to
savepvn() during the compilation of the main program, you should
set a C breakpoint
in Perl_warn(), continue until this point is reached, and I<then> set
a C breakpoint in Perl_savepvn().  Note that you may need to skip a
handful of Perl_savepvn() calls that do not correspond to mass production
of CVs (there are more C<903> allocations than 146 similar lines of
F<lib/auto/POSIX/autosplit.ix>).  Note also that C<Perl_> prefixes are
added by macroization code in perl header files to avoid conflicts
with external libraries.

Anyway, we see that C<903> ids correspond to creation of globs, twice
per glob - for glob name, and glob stringification magic.

Here are explanations for other I<Id>s above: 

=over 4

=item C<717> 

Creates bigger C<XPV*> structures.  In the case above, it
creates 3 C<AV>s per subroutine, one for a list of lexical variable
names, one for a scratchpad (which contains lexical variables and
C<targets>), and one for the array of scratchpads needed for
recursion.  

It also creates a C<GV> and a C<CV> per subroutine, all called from
start_subparse().

=item C<002>

Creates a C array corresponding to the C<AV> of scratchpads and the
scratchpad itself.  The first fake entry of this scratchpad is
created though the subroutine itself is not defined yet.

It also creates C arrays to keep data for the stash.  This is one HV,
but it grows; thus, there are 4 big allocations: the big chunks are not
freed, but are kept as additional arenas for C<SV> allocations.

=item C<054>

Creates a C<HEK> for the name of the glob for the subroutine.  This
name is a key in a I<stash>.

Big allocations with this I<Id> correspond to allocations of new
arenas to keep C<HE>.

=item C<602>

Creates a C<GP> for the glob for the subroutine.

=item C<702>

Creates the C<MAGIC> for the glob for the subroutine.

=item C<704>

Creates I<arenas> which keep SVs.

=back

=head2 B<-DL> details

If Perl is run with B<-DL> option, then warn()s that start with `!'
behave specially.  They print a list of I<categories> of memory
allocations, and statistics of allocations of different sizes for
these categories.

If warn() string starts with

=over 4

=item C<!!!> 

print changed categories only, print the differences in counts of allocations.

=item C<!!> 

print grown categories only; print the absolute values of counts, and totals.

=item C<!>

print nonempty categories, print the absolute values of counts and totals.

=back

=head2 Limitations of B<-DL> statistics

If an extension or external library does not use the Perl API to
allocate memory, such allocations are not counted.

=head1 SEE ALSO

L<perldebug>,
L<perlguts>,
L<perlrun>
L<re>,
and
L<Devel::Dprof>.