scanning.html
来自「perl教程」· HTML 代码 · 共 687 行 · 第 1/4 页
HTML
687 行
the single scalar "bar", and <a href="#item_parent"><code>$e->parent</code></a> returns the object
that's this node's parent -- which may be, for example, a "p" element.</p>
<p>And that's all that there is to it -- you throw HTML
source at TreeBuilder, and it returns a tree built of HTML::Element
objects and some text strings.</p>
<p>However, what do you <em>do</em> with a tree of objects? People code
information into HTML trees not for the fun of arranging elements, but
to represent the structure of specific text and images -- some text is
in this "li" element, some other text is in that heading, some
images are in that other table cell that has those attributes, and so on.</p>
<p>Now, it may happen that you're rendering that whole HTML tree into some
layout format. Or you could be trying to make some systematic change to
the HTML tree before dumping it out as HTML source again. But, in my
experience, by far the most common programming task that Perl
programmers face with HTML is in trying to extract some piece
of information from a larger document. Since that's so common (and
also since it involves concepts that are basic to more complex tasks),
that is what the rest of this article will be about.</p>
<p>
</p>
<h2><a name="scanning_html_trees">Scanning HTML trees</a></h2>
<p>Suppose you have a thousand HTML documents, each of them a press
release. They all start out:</p>
<pre>
[...lots of leading images and junk...]
<h1>ConGlomCo to Open New Corporate Office in Ougadougou</h1>
BAKERSFIELD, CA, 2000-04-24 -- ConGlomCo's vice president in charge
of world conquest, Rock Feldspar, announced today the opening of a
new office in Ougadougou, the capital city of Burkino Faso, gateway
to the bustling "Silicon Sahara" of Africa...
[...etc...]</pre>
<p>...and what you've got to do is, for each document, copy whatever text
is in the "h1" element, so that you can, for example, make a table of
contents of it. Now, there are three ways to do this:</p>
<ul>
<li><strong><a name="item_you_can_just_use_a_regexp_to_scan_the_file_for_a_t">You can just use a regexp to scan the file for a text pattern.</a></strong>
<p>For many very simple tasks, this will do fine. Many HTML documents are,
in practice, very consistently formatted as far as placement of
linebreaks and whitespace, so you could just get away with scanning the
file like so:</p>
<pre>
<span class="keyword">sub</span><span class="variable"> get_heading </span><span class="operator">{</span>
<span class="keyword">my</span> <span class="variable">$filename</span> <span class="operator">=</span> <span class="variable">$_</span><span class="operator">[</span><span class="number">0</span><span class="operator">]</span><span class="operator">;</span>
<span class="keyword">local</span> <span class="variable">*HTML</span><span class="operator">;</span>
<span class="keyword">open</span><span class="operator">(</span><span class="variable">HTML</span><span class="operator">,</span> <span class="variable">$filename</span><span class="operator">)</span>
<span class="keyword">or</span> <span class="keyword">die</span> <span class="string">"Couldn't open $filename);
my $heading;
Line:
while(<HTML>) {
if( m{<h1>(.*?)</h1>}i ) { # match it!
$heading = $1;
last Line;
}
}
close(HTML);
warn "</span><span class="variable">No</span> <span class="variable">heading</span> <span class="variable">in</span> <span class="variable">$filename</span><span class="operator">?</span><span class="string">"
unless defined $heading;
return $heading;
}
</span>
</pre>
<p>This is quick and fast, but awfully fragile -- if there's a newline in
the middle of a heading's text, it won't match the above regexp, and
you'll get an error. The regexp will also fail if the "h1" element's
start-tag has any attributes. If you have to adapt your code to fit
more kinds of start-tags, you'll end up basically reinventing part of
HTML::Parser, at which point you should probably just stop, and use
HTML::Parser itself:</p>
</li>
<li><strong><a name="item_you_can_use_html_3a_3aparser_to_scan_the_file_for_">You can use HTML::Parser to scan the file for an "h1" start-tag
token, then capture all the text tokens until the "h1" close-tag. This
approach is extensively covered in the Ken MacFarlane's TPJ17 article
"Parsing HTML with HTML::Parser". (A variant of this approach is to use
HTML::TokeParser, which presents a different and rather handier
interface to the tokens that HTML::Parser picks out.)</a></strong>
<p>Using HTML::Parser is less fragile than our first approach, since it's
not sensitive to the exact internal formatting of the start-tag (much
less whether it's split across two lines). However, when you need more
information about the context of the "h1" element, or if you're having
to deal with any of the tricky bits of HTML, such as parsing of tables,
you'll find out the flat list of tokens that HTML::Parser returns
isn't immediately useful. To get something useful out of those tokens,
you'll need to write code that knows some things about what elements
take no content (as with "hr" elements), and that a "</p>" end-tags
are omissible, so a "<p>" will end any currently
open paragraph -- and you're well on your way to pointlessly
reinventing much of the code in HTML::TreeBuilder</p>
<p>Footnote:
And, as the person who last rewrote that module, I can attest that it
wasn't terribly easy to get right! Never underestimate the perversity
of people coding HTML.</p>
<p>, at which point you should probably just stop, and use
HTML::TreeBuilder itself:</p>
<li><strong><a name="item_you_can_use_html_3a_3atreebuilder_2c_and_scan_the_">You can use HTML::Treebuilder, and scan the tree of element
objects that you get back.</a></strong>
</ul>
<p>The last approach, using HTML::TreeBuilder, is the diametric opposite of
first approach: The first approach involves just elementary Perl and one
regexp, whereas the TreeBuilder approach involves being at home with
the concept of tree-shaped data structures and modules with
object-oriented interfaces, as well as with the particular interfaces
that HTML::TreeBuilder and HTML::Element provide.</p>
<p>However, what the TreeBuilder approach has going for it is that it's
the most robust, because it involves dealing with HTML in its "native"
format -- it deals with the tree structure that HTML code represents,
without any consideration of how the source is coded and with what
tags omitted.</p>
<p>So, to extract the text from the "h1" elements of an HTML document:</p>
<pre>
<span class="keyword">sub</span><span class="variable"> get_heading </span><span class="operator">{</span>
<span class="keyword">my</span> <span class="variable">$tree</span> <span class="operator">=</span> <span class="variable">HTML::TreeBuilder</span><span class="operator">-></span><span class="variable">new</span><span class="operator">;</span>
<span class="variable">$tree</span><span class="operator">-></span><span class="variable">parse_file</span><span class="operator">(</span><span class="variable">$_</span><span class="operator">[</span><span class="number">0</span><span class="operator">]</span><span class="operator">);</span> <span class="comment"># !</span>
<span class="keyword">my</span> <span class="variable">$heading</span><span class="operator">;</span>
<span class="keyword">my</span> <span class="variable">$h1</span> <span class="operator">=</span> <span class="variable">$tree</span><span class="operator">-></span><span class="variable">look_down</span><span class="operator">(</span><span class="string">'_tag'</span><span class="operator">,</span> <span class="string">'h1'</span><span class="operator">);</span> <span class="comment"># !</span>
<span class="keyword">if</span><span class="operator">(</span><span class="variable">$h1</span><span class="operator">)</span> <span class="operator">{</span>
<span class="variable">$heading</span> <span class="operator">=</span> <span class="variable">$h1</span><span class="operator">-></span><span class="variable">as_text</span><span class="operator">;</span> <span class="comment"># !</span>
<span class="operator">}</span> <span class="keyword">else</span> <span class="operator">{</span>
<span class="keyword">warn</span> <span class="string">"No heading in $_[0]?"</span><span class="operator">;</span>
<span class="operator">}</span>
<span class="variable">$tree</span><span class="operator">-></span><span class="keyword">delete</span><span class="operator">;</span> <span class="comment"># clear memory!</span>
<span class="keyword">return</span> <span class="variable">$heading</span><span class="operator">;</span>
<span class="operator">}</span>
</pre>
<p>This uses some unfamiliar methods that need explaining. The
<code>parse_file</code> method that we've seen before, builds a tree based on
source from the file given. The <a href="../../../lib/Pod/perlfunc.html#item_delete"><code>delete</code></a> method is for marking a
tree's contents as available for garbage collection, when you're done
with the tree. The <code>as_text</code> method returns a string that contains
all the text bits that are children (or otherwise descendants) of the
given node -- to get the text content of the <code>$h1</code> object, we could
just say:</p>
<pre>
<span class="variable">$heading</span> <span class="operator">=</span> <span class="keyword">join</span> <span class="string">''</span><span class="operator">,</span> <span class="variable">$h1</span><span class="operator">-></span><span class="variable">content_list</span><span class="operator">;</span>
</pre>
<p>but that will work only if we're sure that the "h1" element's children
will be only text bits -- if the document contained:</p>
<pre>
<h1>Local Man Sees <cite>Blade</cite> Again</h1></pre>
<p>then the sub-tree would be:</p>
<pre>
. h1
. "Local Man Sees "
. cite
. "Blade"
. " Again'</pre>
<p>so <code>join '', $h1->content_list</code> will be something like:</p>
<pre>
Local Man Sees HTML::Element=HASH(0x15424040) Again</pre>
<p>whereas <code>$h1->as_text</code> would yield:</p>
<pre>
Local Man Sees Blade Again</pre>
<p>and depending on what you're doing with the heading text, you might
want the <code>as_HTML</code> method instead. It returns the (sub)tree
represented as HTML source. <code>$h1->as_HTML</code> would yield:</p>
<pre>
<h1>Local Man Sees <cite>Blade</cite> Again</h1></pre>
<p>However, if you wanted the contents of <code>$h1</code> as HTML, but not the
<code>$h1</code> itself, you could say:</p>
<pre>
join '',
map(
ref($_) ? $_->as_HTML : $_,
$h1->content_list
)</pre>
<p>This <a href="../../../lib/Pod/perlfunc.html#item_map"><code>map</code></a> iterates over the nodes in <code>$h1</code>'s list of children; and
for each node that's just a text bit (as "Local Man Sees " is), it just
passes through that string value, and for each node that's an actual
object (causing <a href="../../../lib/Pod/perlfunc.html#item_ref"><code>ref</code></a> to be true), <code>as_HTML</code> will used instead of the
string value of the object itself (which would be something quite
⌨️ 快捷键说明
复制代码Ctrl + C
搜索代码Ctrl + F
全屏模式F11
增大字号Ctrl + =
减小字号Ctrl + -
显示快捷键?