robotua.html
来自「perl教程」· HTML 代码 · 共 173 行
HTML
173 行
<?xml version="1.0" ?>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<head>
<!-- saved from url=(0017)http://localhost/ -->
<script language="JavaScript" src="../../displayToc.js"></script>
<script language="JavaScript" src="../../tocParas.js"></script>
<script language="JavaScript" src="../../tocTab.js"></script>
<link rel="stylesheet" type="text/css" href="../../scineplex.css">
<title>LWP::RobotUA - a class for well-behaved Web robots</title>
<link rel="stylesheet" href="../../Active.css" type="text/css" />
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
<link rev="made" href="mailto:" />
</head>
<body>
<script>writelinks('__top__',2);</script>
<h1><a>LWP::RobotUA - a class for well-behaved Web robots</a></h1>
<p><a name="__index__"></a></p>
<!-- INDEX BEGIN -->
<ul>
<li><a href="#name">NAME</a></li>
<li><a href="#synopsis">SYNOPSIS</a></li>
<li><a href="#description">DESCRIPTION</a></li>
<li><a href="#methods">METHODS</a></li>
<li><a href="#see_also">SEE ALSO</a></li>
<li><a href="#copyright">COPYRIGHT</a></li>
</ul>
<!-- INDEX END -->
<hr />
<p>
</p>
<h1><a name="name">NAME</a></h1>
<p>LWP::RobotUA - a class for well-behaved Web robots</p>
<p>
</p>
<hr />
<h1><a name="synopsis">SYNOPSIS</a></h1>
<pre>
<span class="keyword">use</span> <span class="variable">LWP::RobotUA</span><span class="operator">;</span>
<span class="keyword">my</span> <span class="variable">$ua</span> <span class="operator">=</span> <span class="variable">LWP::RobotUA</span><span class="operator">-></span><span class="variable">new</span><span class="operator">(</span><span class="string">'my-robot/0.1'</span><span class="operator">,</span> <span class="string">'me@foo.com'</span><span class="operator">);</span>
<span class="variable">$ua</span><span class="operator">-></span><span class="variable">delay</span><span class="operator">(</span><span class="number">10</span><span class="operator">);</span> <span class="comment"># be very nice -- max one hit every ten minutes!</span>
<span class="operator">...</span>
</pre>
<pre>
<span class="comment"># Then just use it just like a normal LWP::UserAgent:</span>
<span class="keyword">my</span> <span class="variable">$response</span> <span class="operator">=</span> <span class="variable">$ua</span><span class="operator">-></span><span class="variable">get</span><span class="operator">(</span><span class="string">'http://whatever.int/...'</span><span class="operator">);</span>
<span class="operator">...</span>
</pre>
<p>
</p>
<hr />
<h1><a name="description">DESCRIPTION</a></h1>
<p>This class implements a user agent that is suitable for robot
applications. Robots should be nice to the servers they visit. They
should consult the <em>/robots.txt</em> file to ensure that they are welcomed
and they should not make requests too frequently.</p>
<p>But before you consider writing a robot, take a look at
<URL:http://www.robotstxt.org/>.</p>
<p>When you use a <em>LWP::RobotUA</em> object as your user agent, then you do not
really have to think about these things yourself; <code>robots.txt</code> files
are automatically consulted and obeyed, the server isn't queried
too rapidly, and so on. Just send requests
as you do when you are using a normal <em>LWP::UserAgent</em>
object (using <code>$ua->get(...)</code>, <code>$ua->head(...)</code>,
<code>$ua->request(...)</code>, etc.), and this
special agent will make sure you are nice.</p>
<p>
</p>
<hr />
<h1><a name="methods">METHODS</a></h1>
<p>The LWP::RobotUA is a sub-class of LWP::UserAgent and implements the
same methods. In addition the following methods are provided:</p>
<dl>
<dt><strong><a name="item_new">$ua = LWP::RobotUA->new( %options )</a></strong>
<dt><strong>$ua = LWP::RobotUA->new( $agent, $from )</strong>
<dt><strong>$ua = LWP::RobotUA->new( $agent, $from, $rules )</strong>
<dd>
<p>The LWP::UserAgent options <code>agent</code> and <code>from</code> are mandatory. The
options <a href="#item_delay"><code>delay</code></a>, <a href="#item_use_sleep"><code>use_sleep</code></a> and <a href="#item_rules"><code>rules</code></a> initialize attributes
private to the RobotUA. If <a href="#item_rules"><code>rules</code></a> are not provided, then
<code>WWW::RobotRules</code> is instantiated providing an internal database of
<em>robots.txt</em>.</p>
</dd>
<dd>
<p>It is also possible to just pass the value of <code>agent</code>, <code>from</code> and
optionally <a href="#item_rules"><code>rules</code></a> as plain positional arguments.</p>
</dd>
</li>
<dt><strong><a name="item_delay">$ua->delay</a></strong>
<dt><strong>$ua->delay( $minutes )</strong>
<dd>
<p>Get/set the minimum delay between requests to the same server, in
<em>minutes</em>. The default is 1 minute. Note that this number doesn't
have to be an integer; for example, this sets the delay to 10 seconds:</p>
</dd>
<dd>
<pre>
<span class="variable">$ua</span><span class="operator">-></span><span class="variable">delay</span><span class="operator">(</span><span class="number">10</span><span class="operator">/</span><span class="number">60</span><span class="operator">);</span>
</pre>
</dd>
</li>
<dt><strong><a name="item_use_sleep">$ua->use_sleep</a></strong>
<dt><strong>$ua->use_sleep( $boolean )</strong>
<dd>
<p>Get/set a value indicating whether the UA should <a href="../../lib/Pod/perlfunc.html#item_sleep"><code>sleep()</code></a> if requests
arrive too fast, defined as $ua->delay minutes not passed since
last request to the given server. The default is TRUE. If this value is
FALSE then an internal SERVICE_UNAVAILABLE response will be generated.
It will have an Retry-After header that indicates when it is OK to
send another request to this server.</p>
</dd>
</li>
<dt><strong><a name="item_rules">$ua->rules</a></strong>
<dt><strong>$ua->rules( $rules )</strong>
<dd>
<p>Set/get which <em>WWW::RobotRules</em> object to use.</p>
</dd>
</li>
<dt><strong><a name="item_no_visits">$ua->no_visits( $netloc )</a></strong>
<dd>
<p>Returns the number of documents fetched from this server host. Yeah I
know, this method should probably have been named <code>num_visits()</code> or
something like that. :-(</p>
</dd>
</li>
<dt><strong><a name="item_host_wait">$ua->host_wait( $netloc )</a></strong>
<dd>
<p>Returns the number of <em>seconds</em> (from now) you must wait before you can
make a new request to this host.</p>
</dd>
</li>
<dt><strong><a name="item_as_string">$ua->as_string</a></strong>
<dd>
<p>Returns a string that describes the state of the UA.
Mainly useful for debugging.</p>
</dd>
</li>
</dl>
<p>
</p>
<hr />
<h1><a name="see_also">SEE ALSO</a></h1>
<p><a href="../../lib/LWP/UserAgent.html">the LWP::UserAgent manpage</a>, <a href="../../lib/WWW/RobotRules.html">the WWW::RobotRules manpage</a></p>
<p>
</p>
<hr />
<h1><a name="copyright">COPYRIGHT</a></h1>
<p>Copyright 1996-2004 Gisle Aas.</p>
<p>This library is free software; you can redistribute it and/or
modify it under the same terms as Perl itself.</p>
</body>
</html>
⌨️ 快捷键说明
复制代码Ctrl + C
搜索代码Ctrl + F
全屏模式F11
增大字号Ctrl + =
减小字号Ctrl + -
显示快捷键?