serialized-form.html

用JAVA编写的,在做实验的时候留下来的,本来想删的,但是传上来,大家分享吧

<H3>
currentDocsPerSecond</H3>
<PRE>
double <B>currentDocsPerSecond</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
currentKBPerSec</H3>
<PRE>
int <B>currentKBPerSec</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
totalKBPerSec</H3>
<PRE>
long <B>totalKBPerSec</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
busyThreads</H3>
<PRE>
int <B>busyThreads</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
totalProcessedBytes</H3>
<PRE>
long <B>totalProcessedBytes</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
congestionRatio</H3>
<PRE>
float <B>congestionRatio</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
deepestUri</H3>
<PRE>
long <B>deepestUri</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
averageDepth</H3>
<PRE>
long <B>averageDepth</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
mimeTypeDistribution</H3>
<PRE>
java.util.Hashtable&lt;K,V&gt; <B>mimeTypeDistribution</B></PRE>
<DL>
<DD>Keep track of the file types we see (mime type -> count)
<P>
<DL>
</DL>
</DL>
<HR>
<H3>
mimeTypeBytes</H3>
<PRE>
java.util.Hashtable&lt;K,V&gt; <B>mimeTypeBytes</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
statusCodeDistribution</H3>
<PRE>
java.util.Hashtable&lt;K,V&gt; <B>statusCodeDistribution</B></PRE>
<DL>
<DD>Keep track of fetch status codes
<P>
<DL>
</DL>
</DL>
<HR>
<H3>
seedsCrawled</H3>
<PRE>
int <B>seedsCrawled</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
seedsNotCrawled</H3>
<PRE>
int <B>seedsNotCrawled</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
sExitMessage</H3>
<PRE>
java.lang.String <B>sExitMessage</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR SIZE="4" NOSHADE>

<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#CCCCFF" CLASS="TableSubHeadingColor">
<TH ALIGN="center"><FONT SIZE="+2">
<B>Package</B> <B>org.archive.crawler.datamodel</B></FONT></TH>
</TR>
</TABLE>

<P>
<A NAME="org.archive.crawler.datamodel.CandidateURI"><!-- --></A>
<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#CCCCFF" CLASS="TableSubHeadingColor">
<TH ALIGN="left" COLSPAN="2"><FONT SIZE="+2">
<B>Class <A HREF="org/archive/crawler/datamodel/CandidateURI.html" title="class in org.archive.crawler.datamodel">org.archive.crawler.datamodel.CandidateURI</A> extends java.lang.Object implements Serializable</B></FONT></TH>
</TR>
</TABLE>

<P>
<B>serialVersionUID:&nbsp;</B>-7152937921526560388L

<P>
<A NAME="serialized_methods"><!-- --></A>
<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor">
<TH ALIGN="left" COLSPAN="1"><FONT SIZE="+2">
<B>Serialization Methods</B></FONT></TH>
</TR>
</TABLE>

<P>
<A NAME="readObject(java.io.ObjectInputStream)"><!-- --></A><H3>
readObject</H3>
<PRE>
private void <B>readObject</B>(java.io.ObjectInputStream&nbsp;stream)
                 throws java.io.IOException,
                        java.lang.ClassNotFoundException</PRE>
<DL>
<DD>Custom deserialization to reconstruct UURI instances from more
 compact Strings.
<P>
<DD><DL>

<DT><B>Throws:</B>
<DD><CODE>java.io.IOException</CODE>
<DD><CODE>java.lang.ClassNotFoundException</CODE></DD>
</DL>
</DL>
<HR>
<A NAME="writeObject(java.io.ObjectOutputStream)"><!-- --></A><H3>
writeObject</H3>
<PRE>
private void <B>writeObject</B>(java.io.ObjectOutputStream&nbsp;stream)
                  throws java.io.IOException</PRE>
<DL>
<DD>Custom serialization writing 'uuri' and 'via' as Strings, rather
 than the bloated full serialization of their object classes, and 
 an empty alist as 'null'. Shrinks serialized form by 50% or more
 in short tests.
<P>
<DD><DL>

<DT><B>Throws:</B>
<DD><CODE>java.io.IOException</CODE></DD>
</DL>
</DL>
<A NAME="serializedForm"><!-- --></A>
<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor">
<TH ALIGN="left" COLSPAN="1"><FONT SIZE="+2">
<B>Serialized Fields</B></FONT></TH>
</TR>
</TABLE>

<H3>
schedulingDirective</H3>
<PRE>
int <B>schedulingDirective</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
isSeed</H3>
<PRE>
boolean <B>isSeed</B></PRE>
<DL>
<DD>Seed status
<P>
<DL>
</DL>
</DL>
<HR>
<H3>
forceRevisit</H3>
<PRE>
boolean <B>forceRevisit</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
pathFromSeed</H3>
<PRE>
java.lang.String <B>pathFromSeed</B></PRE>
<DL>
<DD>String of letters indicating how this URI was reached from a seed.
 <pre>
 P precondition
 R redirection
 E embedded (as frame, src, link, codebase, etc.)
 X speculative embed (as from javascript, some alternate-format extractors
 L link</pre>
 For example LLLE (an embedded image on a page 3 links from seed).
<P>
<DL>
</DL>
</DL>
<HR>
<H3>
viaContext</H3>
<PRE>
java.lang.CharSequence <B>viaContext</B></PRE>
<DL>
<DD>Context of URI's discovery, as per the 'context' in Link
<P>
<DL>
</DL>
</DL>
<HR>
<H3>
cachedCandidateURIString</H3>
<PRE>
java.lang.String <B>cachedCandidateURIString</B></PRE>
<DL>
<DD>Cache of this candidate uuri as a string.

 Profiling shows us spending about 1-2% of total elapsed time in
 toString.
<P>
<DL>
</DL>
</DL>
<HR>
<H3>
classKey</H3>
<PRE>
java.lang.String <B>classKey</B></PRE>
<DL>
<DD>Frontier/Scheduler lifecycle info.
 This is an identifier set by the Frontier for its
 purposes. Usually its the name of the Frontier queue
 this URI gets queued to.  Values can be host + port
 or IP, etc.
<P>
<DL>
</DL>
</DL>

<P>
<A NAME="org.archive.crawler.datamodel.Checkpoint"><!-- --></A>
<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#CCCCFF" CLASS="TableSubHeadingColor">
<TH ALIGN="left" COLSPAN="2"><FONT SIZE="+2">
<B>Class <A HREF="org/archive/crawler/datamodel/Checkpoint.html" title="class in org.archive.crawler.datamodel">org.archive.crawler.datamodel.Checkpoint</A> extends java.lang.Object implements Serializable</B></FONT></TH>
</TR>
</TABLE>

<P>
<B>serialVersionUID:&nbsp;</B>5121498771788002844L

<P>
<A NAME="serialized_methods"><!-- --></A>
<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor">
<TH ALIGN="left" COLSPAN="1"><FONT SIZE="+2">
<B>Serialization Methods</B></FONT></TH>
</TR>
</TABLE>

<P>
<A NAME="readObject(java.io.ObjectInputStream)"><!-- --></A><H3>
readObject</H3>
<PRE>
private void <B>readObject</B>(java.io.ObjectInputStream&nbsp;s)
                 throws java.io.IOException,
                        java.lang.ClassNotFoundException</PRE>
<DL>
<DD><DL>

<DT><B>Throws:</B>
<DD><CODE>java.io.IOException</CODE>
<DD><CODE>java.lang.ClassNotFoundException</CODE></DD>
</DL>
</DL>
<A NAME="serializedForm"><!-- --></A>
<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor">
<TH ALIGN="left" COLSPAN="1"><FONT SIZE="+2">
<B>Serialized Fields</B></FONT></TH>
</TR>
</TABLE>

<H3>
directory</H3>
<PRE>
java.io.File <B>directory</B></PRE>
<DL>
<DL>
</DL>
</DL>

<P>
<A NAME="org.archive.crawler.datamodel.CrawlHost"><!-- --></A>
<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#CCCCFF" CLASS="TableSubHeadingColor">
<TH ALIGN="left" COLSPAN="2"><FONT SIZE="+2">
<B>Class <A HREF="org/archive/crawler/datamodel/CrawlHost.html" title="class in org.archive.crawler.datamodel">org.archive.crawler.datamodel.CrawlHost</A> extends java.lang.Object implements Serializable</B></FONT></TH>
</TR>
</TABLE>

<P>
<A NAME="serializedForm"><!-- --></A>
<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor">
<TH ALIGN="left" COLSPAN="1"><FONT SIZE="+2">
<B>Serialized Fields</B></FONT></TH>
</TR>
</TABLE>

<H3>
hostname</H3>
<PRE>
java.lang.String <B>hostname</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
countryCode</H3>
<PRE>
java.lang.String <B>countryCode</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
ip</H3>
<PRE>
java.net.InetAddress <B>ip</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
ipFetched</H3>
<PRE>
long <B>ipFetched</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
substats</H3>
<PRE>
<A HREF="org/archive/crawler/datamodel/CrawlSubstats.html" title="class in org.archive.crawler.datamodel">CrawlSubstats</A> <B>substats</B></PRE>
<DL>
<DL>
</DL>
</DL>
<HR>
<H3>
ipTTL</H3>
<PRE>
long <B>ipTTL</B></PRE>
<DL>
<DD>TTL gotten from dns record.

 From rfc2035:
 <pre>
 TTL       a 32 bit unsigned integer that specifies the time
           interval (in seconds) that the resource record may be
           cached before it should be discarded.  Zero values are
           interpreted to mean that the RR can only be used for the
           transaction in progress, and should not be cached.
 </pre>
<P>
<DL>
</DL>
</DL>
<HR>
<H3>
earliestNextURIEmitTime</H3>