experimentalwarcwriterprocessor.html

来自「网络爬虫开源代码」· HTML 代码 · 共 547 行 · 第 1/5 页

 Goes against the pending release of 0.12 of the WARC specification, the
 "Marcel Marceau" release. See <a href="https://archive-access.svn.sourceforge.net/svnroot/archive-access/branches/gjm_warc_0_12/warc/warc_file_format.html">latest revision</a>
 for current state.  The 0.10 WARC implemenation has been moved to
 <A HREF="../../../../org/archive/crawler/writer/ExperimentalV10WARCWriterProcessor.html" title="class in org.archive.crawler.writer"><CODE>ExperimentalV10WARCWriterProcessor</CODE></A>.
 
 <p>TODO: Remove ANVLRecord. Rename NameValue or use RFC822
 (commons-httpclient?) or find something else.
<P>

<P>
<DL>
<DT><B>Author:</B></DT>
  <DD>stack</DD>
<DT><B>See Also:</B><DD><A HREF="../../../../serialized-form.html#org.archive.crawler.writer.ExperimentalWARCWriterProcessor">Serialized Form</A></DL>
<HR>

<P>
<!-- ======== NESTED CLASS SUMMARY ======== -->

<A NAME="nested_class_summary"><!-- --></A>
<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor">
<TH ALIGN="left" COLSPAN="2"><FONT SIZE="+2">
<B>Nested Class Summary</B></FONT></TH>
</TR>
</TABLE>
&nbsp;<A NAME="nested_classes_inherited_from_class_org.archive.crawler.settings.ComplexType"><!-- --></A>
<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#EEEEFF" CLASS="TableSubHeadingColor">
<TH ALIGN="left"><B>Nested classes/interfaces inherited from class org.archive.crawler.settings.<A HREF="../../../../org/archive/crawler/settings/ComplexType.html" title="class in org.archive.crawler.settings">ComplexType</A></B></TH>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD><CODE><A HREF="../../../../org/archive/crawler/settings/ComplexType.MBeanAttributeInfoIterator.html" title="class in org.archive.crawler.settings">ComplexType.MBeanAttributeInfoIterator</A></CODE></TD>
</TR>
</TABLE>
&nbsp;
<!-- =========== FIELD SUMMARY =========== -->

<A NAME="field_summary"><!-- --></A>
<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor">
<TH ALIGN="left" COLSPAN="2"><FONT SIZE="+2">
<B>Field Summary</B></FONT></TH>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1">
<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>
<TD><CODE><B><A HREF="../../../../org/archive/crawler/writer/ExperimentalWARCWriterProcessor.html#ATTR_WRITE_METADATA">ATTR_WRITE_METADATA</A></B></CODE>

<BR>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Key for whether to write 'metadata' type records where possible</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1">
<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>
<TD><CODE><B><A HREF="../../../../org/archive/crawler/writer/ExperimentalWARCWriterProcessor.html#ATTR_WRITE_REQUESTS">ATTR_WRITE_REQUESTS</A></B></CODE>

<BR>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Key for whether to write 'request' type records where possible</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1">
<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>
<TD><CODE><B><A HREF="../../../../org/archive/crawler/writer/ExperimentalWARCWriterProcessor.html#ATTR_WRITE_REVISIT_FOR_IDENTICAL_DIGESTS">ATTR_WRITE_REVISIT_FOR_IDENTICAL_DIGESTS</A></B></CODE>

<BR>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Key for whether to write 'revisit' type records when
 consecutive identical digest</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1">
<CODE>static&nbsp;java.lang.String</CODE></FONT></TD>
<TD><CODE><B><A HREF="../../../../org/archive/crawler/writer/ExperimentalWARCWriterProcessor.html#ATTR_WRITE_REVISIT_FOR_NOT_MODIFIED">ATTR_WRITE_REVISIT_FOR_NOT_MODIFIED</A></B></CODE>

<BR>
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Key for whether to write 'revisit' type records for server
 "304 not modified" responses</TD>
</TR>
</TABLE>
&nbsp;<A NAME="fields_inherited_from_class_org.archive.crawler.framework.WriterPoolProcessor"><!-- --></A>
<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#EEEEFF" CLASS="TableSubHeadingColor">
<TH ALIGN="left"><B>Fields inherited from class org.archive.crawler.framework.<A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html" title="class in org.archive.crawler.framework">WriterPoolProcessor</A></B></TH>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD><CODE><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#ANNOTATION_UNWRITTEN">ANNOTATION_UNWRITTEN</A>, <A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#ATTR_COMPRESS">ATTR_COMPRESS</A>, <A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#ATTR_MAX_BYTES_WRITTEN">ATTR_MAX_BYTES_WRITTEN</A>, <A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#ATTR_MAX_SIZE_BYTES">ATTR_MAX_SIZE_BYTES</A>, <A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#ATTR_PATH">ATTR_PATH</A>, <A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#ATTR_POOL_MAX_ACTIVE">ATTR_POOL_MAX_ACTIVE</A>, <A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#ATTR_POOL_MAX_WAIT">ATTR_POOL_MAX_WAIT</A>, <A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#ATTR_PREFIX">ATTR_PREFIX</A>, <A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#ATTR_SKIP_IDENTICAL_DIGESTS">ATTR_SKIP_IDENTICAL_DIGESTS</A>, <A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#ATTR_SUFFIX">ATTR_SUFFIX</A>, <A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#DEFAULT_COMPRESS">DEFAULT_COMPRESS</A></CODE></TD>
</TR>
</TABLE>
&nbsp;<A NAME="fields_inherited_from_class_org.archive.crawler.framework.Processor"><!-- --></A>
<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#EEEEFF" CLASS="TableSubHeadingColor">
<TH ALIGN="left"><B>Fields inherited from class org.archive.crawler.framework.<A HREF="../../../../org/archive/crawler/framework/Processor.html" title="class in org.archive.crawler.framework">Processor</A></B></TH>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD><CODE><A HREF="../../../../org/archive/crawler/framework/Processor.html#ATTR_DECIDE_RULES">ATTR_DECIDE_RULES</A>, <A HREF="../../../../org/archive/crawler/framework/Processor.html#ATTR_ENABLED">ATTR_ENABLED</A>, <A HREF="../../../../org/archive/crawler/framework/Processor.html#attrDecideRules">attrDecideRules</A></CODE></TD>
</TR>
</TABLE>
&nbsp;<A NAME="fields_inherited_from_class_org.archive.crawler.settings.ComplexType"><!-- --></A>
<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#EEEEFF" CLASS="TableSubHeadingColor">
<TH ALIGN="left"><B>Fields inherited from class org.archive.crawler.settings.<A HREF="../../../../org/archive/crawler/settings/ComplexType.html" title="class in org.archive.crawler.settings">ComplexType</A></B></TH>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD><CODE><A HREF="../../../../org/archive/crawler/settings/ComplexType.html#definition">definition</A>, <A HREF="../../../../org/archive/crawler/settings/ComplexType.html#definitionMap">definitionMap</A></CODE></TD>
</TR>
</TABLE>
&nbsp;<A NAME="fields_inherited_from_class_org.archive.crawler.datamodel.CoreAttributeConstants"><!-- --></A>
<TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#EEEEFF" CLASS="TableSubHeadingColor">
<TH ALIGN="left"><B>Fields inherited from interface org.archive.crawler.datamodel.<A HREF="../../../../org/archive/crawler/datamodel/CoreAttributeConstants.html" title="interface in org.archive.crawler.datamodel">CoreAttributeConstants</A></B></TH>