📄 moduletype.html
字号:
<TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/framework/AbstractTracker.html" title="class in org.archive.crawler.framework">AbstractTracker</A></B></CODE><BR> A partial implementation of the StatisticsTracking interface.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/framework/CrawlScope.html" title="class in org.archive.crawler.framework">CrawlScope</A></B></CODE><BR> A CrawlScope instance defines which URIs are "in" a particular crawl.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/framework/Filter.html" title="class in org.archive.crawler.framework">Filter</A></B></CODE><BR> Base class for filter classes.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/framework/Processor.html" title="class in org.archive.crawler.framework">Processor</A></B></CODE><BR> Base class for URI processing classes.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/framework/Scoper.html" title="class in org.archive.crawler.framework">Scoper</A></B></CODE><BR> Base class for Scopers.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/framework/WriterPoolProcessor.html" title="class in org.archive.crawler.framework">WriterPoolProcessor</A></B></CODE><BR> Abstract implementation of a file pool processor.</TD></TR></TABLE> <P><A NAME="org.archive.crawler.frontier"><!-- --></A><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY=""><TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor"><TH ALIGN="left" COLSPAN="2"><FONT SIZE="+2">Uses of <A HREF="../../../../../org/archive/crawler/settings/ModuleType.html" title="class in org.archive.crawler.settings">ModuleType</A> in <A HREF="../../../../../org/archive/crawler/frontier/package-summary.html">org.archive.crawler.frontier</A></FONT></TH></TR></TABLE> <P><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY=""><TR BGCOLOR="#CCCCFF" CLASS="TableSubHeadingColor"><TH ALIGN="left" COLSPAN="2">Subclasses of <A HREF="../../../../../org/archive/crawler/settings/ModuleType.html" title="class in org.archive.crawler.settings">ModuleType</A> in <A HREF="../../../../../org/archive/crawler/frontier/package-summary.html">org.archive.crawler.frontier</A></FONT></TH></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/frontier/AbstractFrontier.html" title="class in org.archive.crawler.frontier">AbstractFrontier</A></B></CODE><BR> Shared facilities for Frontier implementations.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/frontier/AdaptiveRevisitFrontier.html" title="class in org.archive.crawler.frontier">AdaptiveRevisitFrontier</A></B></CODE><BR> A Frontier that will repeatedly visit all encountered URIs.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/frontier/BdbFrontier.html" title="class in org.archive.crawler.frontier">BdbFrontier</A></B></CODE><BR> A Frontier using several BerkeleyDB JE Databases to hold its record of known hosts (queues), and pending URIs.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/frontier/DomainSensitiveFrontier.html" title="class in org.archive.crawler.frontier">DomainSensitiveFrontier</A></B></CODE><BR> <B>Deprecated.</B> <I>As of release 1.10.0. Replaced by <A HREF="../../../../../org/archive/crawler/frontier/BdbFrontier.html" title="class in org.archive.crawler.frontier"><CODE>BdbFrontier</CODE></A> and <A HREF="../../../../../org/archive/crawler/prefetch/QuotaEnforcer.html" title="class in org.archive.crawler.prefetch"><CODE>QuotaEnforcer</CODE></A>.</I></TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/frontier/WorkQueueFrontier.html" title="class in org.archive.crawler.frontier">WorkQueueFrontier</A></B></CODE><BR> A common Frontier base using several queues to hold pending URIs.</TD></TR></TABLE> <P><A NAME="org.archive.crawler.postprocessor"><!-- --></A><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY=""><TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor"><TH ALIGN="left" COLSPAN="2"><FONT SIZE="+2">Uses of <A HREF="../../../../../org/archive/crawler/settings/ModuleType.html" title="class in org.archive.crawler.settings">ModuleType</A> in <A HREF="../../../../../org/archive/crawler/postprocessor/package-summary.html">org.archive.crawler.postprocessor</A></FONT></TH></TR></TABLE> <P><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY=""><TR BGCOLOR="#CCCCFF" CLASS="TableSubHeadingColor"><TH ALIGN="left" COLSPAN="2">Subclasses of <A HREF="../../../../../org/archive/crawler/settings/ModuleType.html" title="class in org.archive.crawler.settings">ModuleType</A> in <A HREF="../../../../../org/archive/crawler/postprocessor/package-summary.html">org.archive.crawler.postprocessor</A></FONT></TH></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/postprocessor/ContentBasedWaitEvaluator.html" title="class in org.archive.crawler.postprocessor">ContentBasedWaitEvaluator</A></B></CODE><BR> A WaitEvaluator that compares the CrawlURIs content type to a configurable regular expression.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/postprocessor/CrawlStateUpdater.html" title="class in org.archive.crawler.postprocessor">CrawlStateUpdater</A></B></CODE><BR> A step, late in the processing of a CrawlURI, for updating the per-host information that may have been affected by the fetch.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/postprocessor/FrontierScheduler.html" title="class in org.archive.crawler.postprocessor">FrontierScheduler</A></B></CODE><BR> 'Schedule' with the Frontier CandidateURIs being carried by the passed CrawlURI.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/postprocessor/ImageWaitEvaluator.html" title="class in org.archive.crawler.postprocessor">ImageWaitEvaluator</A></B></CODE><BR> A specialized ContentBasedWaitEvaluator.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/postprocessor/LinksScoper.html" title="class in org.archive.crawler.postprocessor">LinksScoper</A></B></CODE><BR> Determine which extracted links are within scope.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/postprocessor/LowDiskPauseProcessor.html" title="class in org.archive.crawler.postprocessor">LowDiskPauseProcessor</A></B></CODE><BR> Processor module which uses 'df -k', where available and with the expected output format (on Linux), to monitor available disk space and pause the crawl if free space on monitored filesystems falls below certain thresholds.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/postprocessor/SupplementaryLinksScoper.html" title="class in org.archive.crawler.postprocessor">SupplementaryLinksScoper</A></B></CODE><BR> Run CandidateURI links carried in the passed CrawlURI through a filter and 'handle' rejections.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/postprocessor/TextWaitEvaluator.html" title="class in org.archive.crawler.postprocessor">TextWaitEvaluator</A></B></CODE><BR> A specialized ContentBasedWaitEvaluator.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/postprocessor/WaitEvaluator.html" title="class in org.archive.crawler.postprocessor">WaitEvaluator</A></B></CODE><BR> A processor that determines when a URI should be revisited next.</TD></TR></TABLE> <P><A NAME="org.archive.crawler.prefetch"><!-- --></A><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY=""><TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor"><TH ALIGN="left" COLSPAN="2"><FONT SIZE="+2">Uses of <A HREF="../../../../../org/archive/crawler/settings/ModuleType.html" title="class in org.archive.crawler.settings">ModuleType</A> in <A HREF="../../../../../org/archive/crawler/prefetch/package-summary.html">org.archive.crawler.prefetch</A></FONT></TH></TR></TABLE> <P><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY=""><TR BGCOLOR="#CCCCFF" CLASS="TableSubHeadingColor"><TH ALIGN="left" COLSPAN="2">Subclasses of <A HREF="../../../../../org/archive/crawler/settings/ModuleType.html" title="class in org.archive.crawler.settings">ModuleType</A> in <A HREF="../../../../../org/archive/crawler/prefetch/package-summary.html">org.archive.crawler.prefetch</A></FONT></TH></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD><TD><CODE><B><A HREF="../../../../../org/archive/crawler/prefetch/PreconditionEnforcer.html" title="class in org.archive.crawler.prefetch">PreconditionEnforcer</A></B></CODE><BR> Ensures the preconditions for a fetch -- such as DNS lookup or acquiring and respecting a robots.txt policy -- are satisfied before a URI is passed to subsequent stages.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE> class</CODE></FONT></TD>⌨️ 快捷键说明
复制代码
Ctrl + C
搜索代码
Ctrl + F
全屏模式
F11
切换主题
Ctrl + Shift + D
显示快捷键
?
增大字号
Ctrl + =
减小字号
Ctrl + -