package-summary.html

一个开源的网页爬虫一个开源的网页爬虫一个开源的网页爬虫一个开源的网页爬虫一个开源的网页爬虫一个开源的网页爬虫

</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/NotMatchesListRegExpDecideRule.html" title="class in org.archive.crawler.deciderules">NotMatchesListRegExpDecideRule</A></B></TD>
<TD>Rule applies configured decision to any URIs which do *not*
 match the supplied regexp.</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/NotMatchesRegExpDecideRule.html" title="class in org.archive.crawler.deciderules">NotMatchesRegExpDecideRule</A></B></TD>
<TD>Rule applies configured decision to any URIs which do *not*
 match the supplied regexp.</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/NotOnDomainsDecideRule.html" title="class in org.archive.crawler.deciderules">NotOnDomainsDecideRule</A></B></TD>
<TD>Rule applies configured decision to any URIs that are
 *not* in one of the domains in the configured set of
 domains, filled from the seed set.</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/NotOnHostsDecideRule.html" title="class in org.archive.crawler.deciderules">NotOnHostsDecideRule</A></B></TD>
<TD>Rule applies configured decision to any URIs that
 are *not* on one of the hosts in the configured set of
 hosts, filled from the seed set.</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/NotSurtPrefixedDecideRule.html" title="class in org.archive.crawler.deciderules">NotSurtPrefixedDecideRule</A></B></TD>
<TD>Rule applies configured decision to any URIs that, when 
 expressed in SURT form, do *not* begin with one of the prefixes
 in the configured set.</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/OnDomainsDecideRule.html" title="class in org.archive.crawler.deciderules">OnDomainsDecideRule</A></B></TD>
<TD>Rule applies configured decision to any URIs that
 are on one of the domains in the configured set of
 domains, filled from the seed set.</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/OnHostsDecideRule.html" title="class in org.archive.crawler.deciderules">OnHostsDecideRule</A></B></TD>
<TD>Rule applies configured decision to any URIs that
 are on one of the hosts in the configured set of
 hosts, filled from the seed set.</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/PathologicalPathDecideRule.html" title="class in org.archive.crawler.deciderules">PathologicalPathDecideRule</A></B></TD>
<TD>Rule REJECTs any URI which contains an excessive number of identical, 
 consecutive path-segments (eg http://example.com/a/a/a/boo.html == 3 '/a' 
 segments)</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/PredicatedDecideRule.html" title="class in org.archive.crawler.deciderules">PredicatedDecideRule</A></B></TD>
<TD>Rule which applies the configured decision only if a 
 test evaluates to true.</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/PrerequisiteAcceptDecideRule.html" title="class in org.archive.crawler.deciderules">PrerequisiteAcceptDecideRule</A></B></TD>
<TD>Rule which ACCEPTs all 'prerequisite' URIs (those with a 'P' in
 the last hopsPath position).</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/RejectDecideRule.html" title="class in org.archive.crawler.deciderules">RejectDecideRule</A></B></TD>
<TD>Rule which answers REJECT to everything evaluated.</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/ScopePlusOneDecideRule.html" title="class in org.archive.crawler.deciderules">ScopePlusOneDecideRule</A></B></TD>
<TD>Rule allows one level of discovery beyond configured scope
 (e.g.</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/SeedAcceptDecideRule.html" title="class in org.archive.crawler.deciderules">SeedAcceptDecideRule</A></B></TD>
<TD>Rule which ACCEPTs all 'seed' URIs (those for which 
 isSeed is true).</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/SurtPrefixedDecideRule.html" title="class in org.archive.crawler.deciderules">SurtPrefixedDecideRule</A></B></TD>
<TD>Rule applies configured decision to any URIs that, when 
 expressed in SURT form, begin with one of the prefixes
 in the configured set.</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/TooManyHopsDecideRule.html" title="class in org.archive.crawler.deciderules">TooManyHopsDecideRule</A></B></TD>
<TD>Rule REJECTs any CrawlURIs whose total number of hops (length of the 
 hopsPath string, traversed links of any type) is over a threshold.</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/TooManyPathSegmentsDecideRule.html" title="class in org.archive.crawler.deciderules">TooManyPathSegmentsDecideRule</A></B></TD>
<TD>Rule REJECTs any CrawlURIs whose total number of path-segments (as
 indicated by the count of '/' characters not including the first '//')
 is over a given threshold.</TD>
</TR>
<TR BGCOLOR="white" CLASS="TableRowColor">
<TD WIDTH="15%"><B><A HREF="../../../../org/archive/crawler/deciderules/TransclusionDecideRule.html" title="class in org.archive.crawler.deciderules">TransclusionDecideRule</A></B></TD>
<TD>Rule ACCEPTs any CrawlURIs whose path-from-seed ('hopsPath' -- see
 <A HREF="../../../../org/archive/crawler/datamodel/CandidateURI.html#getPathFromSeed()"><CODE>CandidateURI.getPathFromSeed()</CODE></A>) ends 
 with at least one, but not more than, the given number of 
 non-navlink ('L') hops.</TD>
</TR>
</TABLE>
&nbsp;

<P>
<A NAME="package_description"><!-- --></A><H2>
Package org.archive.crawler.deciderules Description
</H2>

<P>
Provides classes for a simple decision rules framework.

<p>Each 'step' in a decision rule set which can
affect an objects ultimate fate is called a <code>DecideRule</code>.
Each DecideRule renders a decision (possibly neutral) on the
passed objects fate.</p>

<p>Possible decisions are:
<ul>
  <li>ACCEPT means the object is ruled-in for further processing</li>
  <li>REJECT means the object is ruled-out for further processing</li>
  <li>PASS means this particular DecideRule has no opinion</li>
</ul>
</p>

<p>As previously outlined, each DecideRule is applied in turn;
the last one to express a non-PASS preference wins.</p>

<p>For example, if the rules are:
<ul><li>AcceptDecideRule -- ACCEPTs all (establishing a default)</li>
<li>TooManyHopsDecideRule(max-hops=3) -- REJECTS all with
      hopsPath.length()>3, PASSes otherwise</li>
<li>PrerequisiteAcceptDecideRule -- ACCEPTs any with 'P' as
      last hop, PASSes otherwise (this allows 'LLL's which
      need a 'LLLP' prerequisite a chance to complete)</li>
</ul>
Then, you have a crawl that will go 3 hops (of any type)
from the seeds, with a special affordance to get prerequisites
 of 3-hop items (which may be 4 "hops" out)</p>
   
<p>To allow this style of decision processing to be plugged into the
   existing Filter and Scope slots:
<ul><li>There's a DecidingFilter which takes an (ordered) map of
     DecideRules</li>
<li>There's a DecidingScope which takes the same</li>
</ul>
</p>

<p>See <a href="http://crawler.archive.org/cgi-bin/wiki.pl?NewScopingModel">NewScopingModel</a>
for background.</p>
<P>

<P>
<DL>
</DL>
<HR>


<!-- ======= START OF BOTTOM NAVBAR ====== -->
<A NAME="navbar_bottom"><!-- --></A>
<A HREF="#skip-navbar_bottom" title="Skip navigation links"></A>
<TABLE BORDER="0" WIDTH="100%" CELLPADDING="1" CELLSPACING="0" SUMMARY="">
<TR>
<TD COLSPAN=2 BGCOLOR="#EEEEFF" CLASS="NavBarCell1">
<A NAME="navbar_bottom_firstrow"><!-- --></A>
<TABLE BORDER="0" CELLPADDING="0" CELLSPACING="3" SUMMARY="">
  <TR ALIGN="center" VALIGN="top">
  <TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1">    <A HREF="../../../../overview-summary.html"><FONT CLASS="NavBarFont1"><B>Overview</B></FONT></A>&nbsp;</TD>
  <TD BGCOLOR="#FFFFFF" CLASS="NavBarCell1Rev"> &nbsp;<FONT CLASS="NavBarFont1Rev"><B>Package</B></FONT>&nbsp;</TD>
  <TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1">    <FONT CLASS="NavBarFont1">Class</FONT>&nbsp;</TD>
  <TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1">    <A HREF="package-use.html"><FONT CLASS="NavBarFont1"><B>Use</B></FONT></A>&nbsp;</TD>
  <TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1">    <A HREF="package-tree.html"><FONT CLASS="NavBarFont1"><B>Tree</B></FONT></A>&nbsp;</TD>
  <TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1">    <A HREF="../../../../deprecated-list.html"><FONT CLASS="NavBarFont1"><B>Deprecated</B></FONT></A>&nbsp;</TD>
  <TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1">    <A HREF="../../../../index-all.html"><FONT CLASS="NavBarFont1"><B>Index</B></FONT></A>&nbsp;</TD>
  <TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1">    <A HREF="../../../../help-doc.html"><FONT CLASS="NavBarFont1"><B>Help</B></FONT></A>&nbsp;</TD>
  </TR>
</TABLE>
</TD>
<TD ALIGN="right" VALIGN="top" ROWSPAN=3><EM>
</EM>
</TD>
</TR>

<TR>
<TD BGCOLOR="white" CLASS="NavBarCell2"><FONT SIZE="-2">
&nbsp;<A HREF="../../../../org/archive/crawler/datamodel/credential/package-summary.html"><B>PREV PACKAGE</B></A>&nbsp;
&nbsp;<A HREF="../../../../org/archive/crawler/event/package-summary.html"><B>NEXT PACKAGE</B></A></FONT></TD>
<TD BGCOLOR="white" CLASS="NavBarCell2"><FONT SIZE="-2">
  <A HREF="../../../../index.html?org/archive/crawler/deciderules/package-summary.html" target="_top"><B>FRAMES</B></A>  &nbsp;
&nbsp;<A HREF="package-summary.html" target="_top"><B>NO FRAMES</B></A>  &nbsp;
&nbsp;<SCRIPT type="text/javascript">
  <!--
  if(window==top) {
    document.writeln('<A HREF="../../../../allclasses-noframe.html"><B>All Classes</B></A>');
  }
  //-->
</SCRIPT>
<NOSCRIPT>
  <A HREF="../../../../allclasses-noframe.html"><B>All Classes</B></A>
</NOSCRIPT>


</FONT></TD>
</TR>
</TABLE>
<A NAME="skip-navbar_bottom"></A>
<!-- ======== END OF BOTTOM NAVBAR ======= -->

<HR>
Copyright &copy; 2003-2006 Internet Archive. All Rights Reserved.
</BODY>
</HTML>