📄 writerpoolprocessor.html
字号:
<B>Constructor Summary</B></FONT></TH></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#WriterPoolProcessor(java.lang.String)">WriterPoolProcessor</A></B>(java.lang.String name)</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#WriterPoolProcessor(java.lang.String, java.lang.String)">WriterPoolProcessor</A></B>(java.lang.String name, java.lang.String description)</CODE><BR> </TD></TR></TABLE> <!-- ========== METHOD SUMMARY =========== --><A NAME="method_summary"><!-- --></A><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY=""><TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor"><TH ALIGN="left" COLSPAN="2"><FONT SIZE="+2"><B>Method Summary</B></FONT></TH></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#checkBytesWritten()">checkBytesWritten</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#checkpointRecover()">checkpointRecover</A></B>()</CODE><BR> Called out of <A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#initialTasks()"><CODE>initialTasks()</CODE></A> when recovering a checkpoint.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#crawlCheckpoint(java.io.File)">crawlCheckpoint</A></B>(java.io.File checkpointDir)</CODE><BR> Called by <A HREF="../../../../org/archive/crawler/framework/CrawlController.html" title="class in org.archive.crawler.framework"><CODE>CrawlController</CODE></A> when checkpointing.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#crawlEnded(java.lang.String)">crawlEnded</A></B>(java.lang.String sExitMessage)</CODE><BR> Called when a CrawlController has ended a crawl and is about to exit.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#crawlEnding(java.lang.String)">crawlEnding</A></B>(java.lang.String sExitMessage)</CODE><BR> Called when a CrawlController is ending a crawl (for any reason)</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#crawlPaused(java.lang.String)">crawlPaused</A></B>(java.lang.String statusMessage)</CODE><BR> Called when a CrawlController is actually paused (all threads are idle).</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#crawlPausing(java.lang.String)">crawlPausing</A></B>(java.lang.String statusMessage)</CODE><BR> Called when a CrawlController is going to be paused.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#crawlResuming(java.lang.String)">crawlResuming</A></B>(java.lang.String statusMessage)</CODE><BR> Called when a CrawlController is resuming a crawl that had been paused.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#crawlStarted(java.lang.String)">crawlStarted</A></B>(java.lang.String message)</CODE><BR> Called on crawl start.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> java.lang.Object</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#getAttributeUnchecked(java.lang.String)">getAttributeUnchecked</A></B>(java.lang.String name)</CODE><BR> Version of getAttributes that catches and logs exceptions and returns null if failure to fetch the attribute.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected java.lang.String</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#getCheckpointStateFile()">getCheckpointStateFile</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected java.lang.String[]</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#getDefaultPath()">getDefaultPath</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected java.lang.String</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#getHostAddress(org.archive.crawler.datamodel.CrawlURI)">getHostAddress</A></B>(<A HREF="../../../../org/archive/crawler/datamodel/CrawlURI.html" title="class in org.archive.crawler.datamodel">CrawlURI</A> curi)</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> int</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#getMaxSize()">getMaxSize</A></B>()</CODE><BR> Max size we want files to be (bytes).</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> long</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#getMaxToWrite()">getMaxToWrite</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> java.util.List</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#getOutputDirs()">getOutputDirs</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected <A HREF="../../../../org/archive/io/WriterPool.html" title="class in org.archive.io">WriterPool</A></CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#getPool()">getPool</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> int</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#getPoolMaximumActive()">getPoolMaximumActive</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> int</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#getPoolMaximumWait()">getPoolMaximumWait</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> java.lang.String</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#getPrefix()">getPrefix</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected java.util.concurrent.atomic.AtomicInteger</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#getSerialNo()">getSerialNo</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> java.lang.String</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#getSuffix()">getSuffix</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected long</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#getTotalBytesWritten()">getTotalBytesWritten</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#initialTasks()">initialTasks</A></B>()</CODE><BR> Classes subclassing this one should override this method to perform processor specific actions.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected abstract void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#innerProcess(org.archive.crawler.datamodel.CrawlURI)">innerProcess</A></B>(<A HREF="../../../../org/archive/crawler/datamodel/CrawlURI.html" title="class in org.archive.crawler.datamodel">CrawlURI</A> curi)</CODE><BR> Writes a CrawlURI and its associated data to store file.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> boolean</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#isCompressed()">isCompressed</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected int</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#loadCheckpointSerialNumber()">loadCheckpointSerialNumber</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#saveCheckpointSerialNumber(java.io.File, int)">saveCheckpointSerialNumber</A></B>(java.io.File checkpointDir, int serialNo)</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/WriterPoolProcessor.html#setPool(org.archive.io.WriterPool)">setPool</A></B>(<A HREF="../../../../org/archive/io/WriterPool.html" title="class in org.archive.io">WriterPool</A> pool)</CODE><BR>⌨️ 快捷键说明
复制代码
Ctrl + C
搜索代码
Ctrl + F
全屏模式
F11
切换主题
Ctrl + Shift + D
显示快捷键
?
增大字号
Ctrl + =
减小字号
Ctrl + -