📄 statisticstracker.html
字号:
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><!--NewPage--><HTML><HEAD><!-- Generated by javadoc (build 1.5.0_06) on Wed Sep 27 16:03:04 PDT 2006 --><TITLE>StatisticsTracker (Heritrix 1.10.1)</TITLE><META NAME="keywords" CONTENT="org.archive.crawler.admin.StatisticsTracker class"><LINK REL ="stylesheet" TYPE="text/css" HREF="../../../../stylesheet.css" TITLE="Style"><SCRIPT type="text/javascript">function windowTitle(){ parent.document.title="StatisticsTracker (Heritrix 1.10.1)";}</SCRIPT><NOSCRIPT></NOSCRIPT></HEAD><BODY BGCOLOR="white" onload="windowTitle();"><!-- ========= START OF TOP NAVBAR ======= --><A NAME="navbar_top"><!-- --></A><A HREF="#skip-navbar_top" title="Skip navigation links"></A><TABLE BORDER="0" WIDTH="100%" CELLPADDING="1" CELLSPACING="0" SUMMARY=""><TR><TD COLSPAN=2 BGCOLOR="#EEEEFF" CLASS="NavBarCell1"><A NAME="navbar_top_firstrow"><!-- --></A><TABLE BORDER="0" CELLPADDING="0" CELLSPACING="3" SUMMARY=""> <TR ALIGN="center" VALIGN="top"> <TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1"> <A HREF="../../../../overview-summary.html"><FONT CLASS="NavBarFont1"><B>Overview</B></FONT></A> </TD> <TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1"> <A HREF="package-summary.html"><FONT CLASS="NavBarFont1"><B>Package</B></FONT></A> </TD> <TD BGCOLOR="#FFFFFF" CLASS="NavBarCell1Rev"> <FONT CLASS="NavBarFont1Rev"><B>Class</B></FONT> </TD> <TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1"> <A HREF="class-use/StatisticsTracker.html"><FONT CLASS="NavBarFont1"><B>Use</B></FONT></A> </TD> <TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1"> <A HREF="package-tree.html"><FONT CLASS="NavBarFont1"><B>Tree</B></FONT></A> </TD> <TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1"> <A HREF="../../../../deprecated-list.html"><FONT CLASS="NavBarFont1"><B>Deprecated</B></FONT></A> </TD> <TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1"> <A HREF="../../../../index-all.html"><FONT CLASS="NavBarFont1"><B>Index</B></FONT></A> </TD> <TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1"> <A HREF="../../../../help-doc.html"><FONT CLASS="NavBarFont1"><B>Help</B></FONT></A> </TD> </TR></TABLE></TD><TD ALIGN="right" VALIGN="top" ROWSPAN=3><EM></EM></TD></TR><TR><TD BGCOLOR="white" CLASS="NavBarCell2"><FONT SIZE="-2"> <A HREF="../../../../org/archive/crawler/admin/StatisticsSummary.html" title="class in org.archive.crawler.admin"><B>PREV CLASS</B></A> NEXT CLASS</FONT></TD><TD BGCOLOR="white" CLASS="NavBarCell2"><FONT SIZE="-2"> <A HREF="../../../../index.html?org/archive/crawler/admin/StatisticsTracker.html" target="_top"><B>FRAMES</B></A> <A HREF="StatisticsTracker.html" target="_top"><B>NO FRAMES</B></A> <SCRIPT type="text/javascript"> <!-- if(window==top) { document.writeln('<A HREF="../../../../allclasses-noframe.html"><B>All Classes</B></A>'); } //--></SCRIPT><NOSCRIPT> <A HREF="../../../../allclasses-noframe.html"><B>All Classes</B></A></NOSCRIPT></FONT></TD></TR><TR><TD VALIGN="top" CLASS="NavBarCell3"><FONT SIZE="-2"> SUMMARY: <A HREF="#nested_classes_inherited_from_class_org.archive.crawler.settings.ComplexType">NESTED</A> | <A HREF="#field_summary">FIELD</A> | <A HREF="#constructor_summary">CONSTR</A> | <A HREF="#method_summary">METHOD</A></FONT></TD><TD VALIGN="top" CLASS="NavBarCell3"><FONT SIZE="-2">DETAIL: <A HREF="#field_detail">FIELD</A> | <A HREF="#constructor_detail">CONSTR</A> | <A HREF="#method_detail">METHOD</A></FONT></TD></TR></TABLE><A NAME="skip-navbar_top"></A><!-- ========= END OF TOP NAVBAR ========= --><HR><!-- ======== START OF CLASS DATA ======== --><H2><FONT SIZE="-1">org.archive.crawler.admin</FONT><BR>Class StatisticsTracker</H2><PRE>java.lang.Object <IMG SRC="../../../../resources/inherit.gif" ALT="extended by ">javax.management.Attribute <IMG SRC="../../../../resources/inherit.gif" ALT="extended by "><A HREF="../../../../org/archive/crawler/settings/Type.html" title="class in org.archive.crawler.settings">org.archive.crawler.settings.Type</A> <IMG SRC="../../../../resources/inherit.gif" ALT="extended by "><A HREF="../../../../org/archive/crawler/settings/ComplexType.html" title="class in org.archive.crawler.settings">org.archive.crawler.settings.ComplexType</A> <IMG SRC="../../../../resources/inherit.gif" ALT="extended by "><A HREF="../../../../org/archive/crawler/settings/ModuleType.html" title="class in org.archive.crawler.settings">org.archive.crawler.settings.ModuleType</A> <IMG SRC="../../../../resources/inherit.gif" ALT="extended by "><A HREF="../../../../org/archive/crawler/framework/AbstractTracker.html" title="class in org.archive.crawler.framework">org.archive.crawler.framework.AbstractTracker</A> <IMG SRC="../../../../resources/inherit.gif" ALT="extended by "><B>org.archive.crawler.admin.StatisticsTracker</B></PRE><DL><DT><B>All Implemented Interfaces:</B> <DD>java.io.Serializable, java.lang.Runnable, javax.management.DynamicMBean, <A HREF="../../../../org/archive/crawler/event/CrawlStatusListener.html" title="interface in org.archive.crawler.event">CrawlStatusListener</A>, <A HREF="../../../../org/archive/crawler/event/CrawlURIDispositionListener.html" title="interface in org.archive.crawler.event">CrawlURIDispositionListener</A>, <A HREF="../../../../org/archive/crawler/framework/StatisticsTracking.html" title="interface in org.archive.crawler.framework">StatisticsTracking</A></DD></DL><HR><DL><DT><PRE>public class <B>StatisticsTracker</B><DT>extends <A HREF="../../../../org/archive/crawler/framework/AbstractTracker.html" title="class in org.archive.crawler.framework">AbstractTracker</A><DT>implements <A HREF="../../../../org/archive/crawler/event/CrawlURIDispositionListener.html" title="interface in org.archive.crawler.event">CrawlURIDispositionListener</A>, java.io.Serializable</DL></PRE><P>This is an implementation of the AbstractTracker. It is designed to function with the WUI as well as performing various logging activity. <p> At the end of each snapshot a line is written to the 'progress-statistics.log' file. <p> The header of that file is as follows: <pre> [timestamp] [discovered] [queued] [downloaded] [doc/s(avg)] [KB/s(avg)] [dl-failures] [busy-thread] [mem-use-KB]</pre> First there is a <b>timestamp</b>, accurate down to 1 second. <p> <b>discovered</b>, <b>queued</b>, <b>downloaded</b> and <b>dl-failures</b> are (respectively) the discovered URI count, pending URI count, successfully fetched count and failed fetch count from the frontier at the time of the snapshot. <p> <b>KB/s(avg)</b> is the bandwidth usage. We use the total bytes downloaded to calculate average bandwidth usage (KB/sec). Since we also note the value each time a snapshot is made we can calculate the average bandwidth usage during the last snapshot period to gain a "current" rate. The first number is the current and the average is in parenthesis. <p> <b>doc/s(avg)</b> works the same way as doc/s except it show the number of documents (URIs) rather then KB downloaded. <p> <b>busy-threads</b> is the total number of ToeThreads that are not available (and thus presumably busy processing a URI). This information is extracted from the crawl controller. <p> Finally mem-use-KB is extracted from the run time environment (<code>Runtime.getRuntime().totalMemory()</code>). <p> In addition to the data collected for the above logs, various other data is gathered and stored by this tracker. <ul> <li> Successfully downloaded documents per fetch status code <li> Successfully downloaded documents per document mime type <li> Amount of data per mime type <li> Successfully downloaded documents per host <li> Amount of data per host <li> Disposition of all seeds (this is written to 'reports.log' at end of crawl) <li> Successfully downloaded documents per host per source </ul><P><P><DL><DT><B>Author:</B></DT> <DD>Parker Thompson, Kristinn Sigurdsson</DD><DT><B>See Also:</B><DD><A HREF="../../../../org/archive/crawler/framework/StatisticsTracking.html" title="interface in org.archive.crawler.framework"><CODE>StatisticsTracking</CODE></A>, <A HREF="../../../../org/archive/crawler/framework/AbstractTracker.html" title="class in org.archive.crawler.framework"><CODE>AbstractTracker</CODE></A>, <A HREF="../../../../serialized-form.html#org.archive.crawler.admin.StatisticsTracker">Serialized Form</A></DL><HR><P><!-- ======== NESTED CLASS SUMMARY ======== --><A NAME="nested_class_summary"><!-- --></A><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY=""><TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor"><TH ALIGN="left" COLSPAN="2"><FONT SIZE="+2"><B>Nested Class Summary</B></FONT></TH></TR></TABLE> <A NAME="nested_classes_inherited_from_class_org.archive.crawler.settings.ComplexType"><!-- --></A><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY=""><TR BGCOLOR="#EEEEFF" CLASS="TableSubHeadingColor"><TH ALIGN="left"><B>Nested classes/interfaces inherited from class org.archive.crawler.settings.<A HREF="../../../../org/archive/crawler/settings/ComplexType.html" title="class in org.archive.crawler.settings">ComplexType</A></B></TH></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD><CODE><A HREF="../../../../org/archive/crawler/settings/ComplexType.MBeanAttributeInfoIterator.html" title="class in org.archive.crawler.settings">ComplexType.MBeanAttributeInfoIterator</A></CODE></TD></TR></TABLE> <!-- =========== FIELD SUMMARY =========== --><A NAME="field_summary"><!-- --></A><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY=""><TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor"><TH ALIGN="left" COLSPAN="2"><FONT SIZE="+2"><B>Field Summary</B></FONT></TH></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected long</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/admin/StatisticsTracker.html#averageDepth">averageDepth</A></B></CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected int</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/admin/StatisticsTracker.html#busyThreads">busyThreads</A></B></CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected float</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/admin/StatisticsTracker.html#congestionRatio">congestionRatio</A></B></CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected double</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/admin/StatisticsTracker.html#currentDocsPerSecond">currentDocsPerSecond</A></B></CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected int</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/admin/StatisticsTracker.html#currentKBPerSec">currentKBPerSec</A></B></CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected long</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/admin/StatisticsTracker.html#deepestUri">deepestUri</A></B></CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected long</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/admin/StatisticsTracker.html#discoveredUriCount">discoveredUriCount</A></B></CODE><BR>⌨️ 快捷键说明
复制代码
Ctrl + C
搜索代码
Ctrl + F
全屏模式
F11
切换主题
Ctrl + Shift + D
显示快捷键
?
增大字号
Ctrl + =
减小字号
Ctrl + -