crawlcontroller.html
来自「网络爬虫开源代码」· HTML 代码 · 共 1,398 行 · 第 1/5 页
HTML
1,398 行
<CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#addCrawlStatusListener(org.archive.crawler.event.CrawlStatusListener)">addCrawlStatusListener</A></B>(<A HREF="../../../../org/archive/crawler/event/CrawlStatusListener.html" title="interface in org.archive.crawler.event">CrawlStatusListener</A> cl)</CODE><BR> Register for CrawlStatus events.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#addCrawlURIDispositionListener(org.archive.crawler.event.CrawlURIDispositionListener)">addCrawlURIDispositionListener</A></B>(<A HREF="../../../../org/archive/crawler/event/CrawlURIDispositionListener.html" title="interface in org.archive.crawler.event">CrawlURIDispositionListener</A> cl)</CODE><BR> Register for CrawlURIDisposition events.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#addOrderToManifest()">addOrderToManifest</A></B>()</CODE><BR> Add order file contents to manifest.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#addToManifest(java.lang.String, char, boolean)">addToManifest</A></B>(java.lang.String file, char type, boolean bundle)</CODE><BR> Add a file to the manifest of files used/generated by the current crawl.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> boolean</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#atFinish()">atFinish</A></B>()</CODE><BR> Evaluate if the crawl should stop because it is finished, without actually stopping the crawl.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#beginCrawlStop()">beginCrawlStop</A></B>()</CODE><BR> Start the process of stopping the crawl.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#checkFinish()">checkFinish</A></B>()</CODE><BR> Evaluate if the crawl should stop because it is finished.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>(package private) void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#checkpoint()">checkpoint</A></B>()</CODE><BR> Run checkpointing.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#checkpointBdb(java.io.File)">checkpointBdb</A></B>(java.io.File checkpointDir)</CODE><BR> Checkpoint bdb.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#checkpointBigMaps(java.io.File)">checkpointBigMaps</A></B>(java.io.File cpDir)</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#closeLogFiles()">closeLogFiles</A></B>()</CODE><BR> Close all log files and remove handlers from loggers.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>(package private) void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#completePause()">completePause</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#completeStop()">completeStop</A></B>()</CODE><BR> Called when the last toethread exits.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected <A HREF="../../../../org/archive/crawler/framework/exceptions/FatalConfigurationException.html" title="class in org.archive.crawler.framework.exceptions">FatalConfigurationException</A></CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#convertToFatalConfigurationException(java.lang.Exception)">convertToFatalConfigurationException</A></B>(java.lang.Exception e)</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#copySettings(java.io.File)">copySettings</A></B>(java.io.File checkpointDir)</CODE><BR> Copy off the settings.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#fireCrawledURIDisregardEvent(org.archive.crawler.datamodel.CrawlURI)">fireCrawledURIDisregardEvent</A></B>(<A HREF="../../../../org/archive/crawler/datamodel/CrawlURI.html" title="class in org.archive.crawler.datamodel">CrawlURI</A> curi)</CODE><BR> Allows an external class to raise a CrawlURIDispostion crawledURIDisregard event that will be broadcast to all listeners that have registered with the CrawlController.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#fireCrawledURIFailureEvent(org.archive.crawler.datamodel.CrawlURI)">fireCrawledURIFailureEvent</A></B>(<A HREF="../../../../org/archive/crawler/datamodel/CrawlURI.html" title="class in org.archive.crawler.datamodel">CrawlURI</A> curi)</CODE><BR> Allows an external class to raise a CrawlURIDispostion crawledURIFailure event that will be broadcast to all listeners that have registered with the CrawlController.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#fireCrawledURINeedRetryEvent(org.archive.crawler.datamodel.CrawlURI)">fireCrawledURINeedRetryEvent</A></B>(<A HREF="../../../../org/archive/crawler/datamodel/CrawlURI.html" title="class in org.archive.crawler.datamodel">CrawlURI</A> curi)</CODE><BR> Allows an external class to raise a CrawlURIDispostion crawledURINeedRetry event that will be broadcast to all listeners that have registered with the CrawlController.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#fireCrawledURISuccessfulEvent(org.archive.crawler.datamodel.CrawlURI)">fireCrawledURISuccessfulEvent</A></B>(<A HREF="../../../../org/archive/crawler/datamodel/CrawlURI.html" title="class in org.archive.crawler.datamodel">CrawlURI</A> curi)</CODE><BR> Allows an external class to raise a CrawlURIDispostion crawledURISuccessful event that will be broadcast to all listeners that have registered with the CrawlController.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#freeReserveMemory()">freeReserveMemory</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> int</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#getActiveToeCount()">getActiveToeCount</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> <A HREF="../../../../org/archive/util/bdbje/EnhancedEnvironment.html" title="class in org.archive.util.bdbje">EnhancedEnvironment</A></CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#getBdbEnvironment()">getBdbEnvironment</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected java.lang.String</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#getBdbLogFileName(long)">getBdbLogFileName</A></B>(long index)</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE><TABLE BORDER="0" CELLPADDING="0" CELLSPACING="0" SUMMARY=""><TR ALIGN="right" VALIGN=""><TD NOWRAP><FONT SIZE="-1"><CODE><K,V> java.util.Map<K,V></CODE></FONT></TD></TR></TABLE></CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#getBigMap(java.lang.String, java.lang.Class, java.lang.Class)">getBigMap</A></B>(java.lang.String dbName, java.lang.Class<? super K> keyClass, java.lang.Class<? super V> valueClass)</CODE><BR> Call this method to get instance of the crawler BigMap implementation.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected boolean</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#getCheckpointCopyBdbjeLogs()">getCheckpointCopyBdbjeLogs</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> <A HREF="../../../../org/archive/crawler/datamodel/Checkpoint.html" title="class in org.archive.crawler.datamodel">Checkpoint</A></CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#getCheckpointRecover()">getCheckpointRecover</A></B>()</CODE><BR> Get recover checkpoint.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>static <A HREF="../../../../org/archive/crawler/datamodel/Checkpoint.html" title="class in org.archive.crawler.datamodel">Checkpoint</A></CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#getCheckpointRecover(org.archive.crawler.datamodel.CrawlOrder)">getCheckpointRecover</A></B>(<A HREF="../../../../org/archive/crawler/datamodel/CrawlOrder.html" title="class in org.archive.crawler.datamodel">CrawlOrder</A> order)</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> java.io.File</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#getCheckpointsDisk()">getCheckpointsDisk</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> com.sleepycat.bind.serial.StoredClassCatalog</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#getClassCatalog()">getClassCatalog</A></B>()</CODE><BR> <B>Deprecated.</B> <I>use EnhancedEnvironment's getClassCatalog() instead</I></TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> java.io.File</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#getDisk()">getDisk</A></B>()</CODE><BR> Get the 'working' directory of the current crawl.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> <A HREF="../../../../org/archive/crawler/framework/ProcessorChain.html" title="class in org.archive.crawler.framework">ProcessorChain</A></CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#getFirstProcessorChain()">getFirstProcessorChain</A></B>()</CODE><BR> Get the first processor chain.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> <A HREF="../../../../org/archive/crawler/framework/Frontier.html" title="interface in org.archive.crawler.framework">Frontier</A></CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#getFrontier()">getFrontier</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> java.io.File</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/framework/CrawlController.html#getLogsDir()">getLogsDir</A></B>()</CODE><BR>⌨️ 快捷键说明
复制代码Ctrl + C
搜索代码Ctrl + F
全屏模式F11
增大字号Ctrl + =
减小字号Ctrl + -
显示快捷键?