📄 adaptiverevisitfrontier.html
字号:
<TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#batchSchedule(org.archive.crawler.datamodel.CandidateURI)">batchSchedule</A></B>(<A HREF="../../../../org/archive/crawler/datamodel/CandidateURI.html" title="class in org.archive.crawler.datamodel">CandidateURI</A> caUri)</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected long</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#calculateSnoozeTime(org.archive.crawler.datamodel.CrawlURI)">calculateSnoozeTime</A></B>(<A HREF="../../../../org/archive/crawler/datamodel/CrawlURI.html" title="class in org.archive.crawler.datamodel">CrawlURI</A> curi)</CODE><BR> Calculates how long a host queue needs to be snoozed following the crawling of a URI.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected java.lang.String</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#canonicalize(org.archive.crawler.datamodel.CandidateURI)">canonicalize</A></B>(<A HREF="../../../../org/archive/crawler/datamodel/CandidateURI.html" title="class in org.archive.crawler.datamodel">CandidateURI</A> cauri)</CODE><BR> Canonicalize passed CandidateURI.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected java.lang.String</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#canonicalize(org.archive.net.UURI)">canonicalize</A></B>(<A HREF="../../../../org/archive/net/UURI.html" title="class in org.archive.net">UURI</A> uuri)</CODE><BR> Canonicalize passed uuri.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> float</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#congestionRatio()">congestionRatio</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#considerIncluded(org.archive.net.UURI)">considerIncluded</A></B>(<A HREF="../../../../org/archive/net/UURI.html" title="class in org.archive.net">UURI</A> u)</CODE><BR> Notify Frontier that it should consider the given UURI as if already scheduled.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#crawlCheckpoint(java.io.File)">crawlCheckpoint</A></B>(java.io.File checkpointDir)</CODE><BR> Called by <A HREF="../../../../org/archive/crawler/framework/CrawlController.html" title="class in org.archive.crawler.framework"><CODE>CrawlController</CODE></A> when checkpointing.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#crawlEnded(java.lang.String)">crawlEnded</A></B>(java.lang.String sExitMessage)</CODE><BR> Called when a CrawlController has ended a crawl and is about to exit.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#crawlEnding(java.lang.String)">crawlEnding</A></B>(java.lang.String sExitMessage)</CODE><BR> Called when a CrawlController is ending a crawl (for any reason)</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#crawlPaused(java.lang.String)">crawlPaused</A></B>(java.lang.String statusMessage)</CODE><BR> Called when a CrawlController is actually paused (all threads are idle).</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#crawlPausing(java.lang.String)">crawlPausing</A></B>(java.lang.String statusMessage)</CODE><BR> Called when a CrawlController is going to be paused.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#crawlResuming(java.lang.String)">crawlResuming</A></B>(java.lang.String statusMessage)</CODE><BR> Called when a CrawlController is resuming a crawl that had been paused.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#crawlStarted(java.lang.String)">crawlStarted</A></B>(java.lang.String message)</CODE><BR> Called on crawl start.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected <A HREF="../../../../org/archive/crawler/datamodel/UriUniqFilter.html" title="interface in org.archive.crawler.datamodel">UriUniqFilter</A></CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#createAlreadyIncluded()">createAlreadyIncluded</A></B>()</CODE><BR> Create a UriUniqFilter that will serve as record of already seen URIs.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> long</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#deepestUri()">deepestUri</A></B>()</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#deleted(org.archive.crawler.datamodel.CrawlURI)">deleted</A></B>(<A HREF="../../../../org/archive/crawler/datamodel/CrawlURI.html" title="class in org.archive.crawler.datamodel">CrawlURI</A> curi)</CODE><BR> Notify Frontier that a CrawlURI has been deleted outside of the normal next()/finished() lifecycle.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> long</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#deleteURIs(java.lang.String)">deleteURIs</A></B>(java.lang.String match)</CODE><BR> Delete any URI that matches the given regular expression from the list of discovered and pending URIs.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> long</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#discoveredUriCount()">discoveredUriCount</A></B>()</CODE><BR> Number of <i>discovered</i> URIs.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#disregardDisposition(org.archive.crawler.datamodel.CrawlURI)">disregardDisposition</A></B>(<A HREF="../../../../org/archive/crawler/datamodel/CrawlURI.html" title="class in org.archive.crawler.datamodel">CrawlURI</A> curi)</CODE><BR> </TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> long</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#disregardedUriCount()">disregardedUriCount</A></B>()</CODE><BR> Number of URIs that were scheduled at one point but have been <i>disregarded</i>.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> long</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#failedFetchCount()">failedFetchCount</A></B>()</CODE><BR> Number of URIs that <i>failed</i> to process.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html#failureDisposition(org.archive.crawler.datamodel.CrawlURI)">failureDisposition</A></B>(<A HREF="../../../../org/archive/crawler/datamodel/CrawlURI.html" title="class in org.archive.crawler.datamodel">CrawlURI</A> curi)</CODE><BR> The CrawlURI has encountered a problem, and will not be retried.</TD></TR>⌨️ 快捷键说明
复制代码
Ctrl + C
搜索代码
Ctrl + F
全屏模式
F11
切换主题
Ctrl + Shift + D
显示快捷键
?
增大字号
Ctrl + =
减小字号
Ctrl + -