📄 ch20_13.htm
字号:
<HTML><HEAD><TITLE>Recipe 20.12. Parsing a Web Server Log File (Perl Cookbook)</TITLE><METANAME="DC.title"CONTENT="Perl Cookbook"><METANAME="DC.creator"CONTENT="Tom Christiansen & Nathan Torkington"><METANAME="DC.publisher"CONTENT="O'Reilly & Associates, Inc."><METANAME="DC.date"CONTENT="1999-07-02T01:46:02Z"><METANAME="DC.type"CONTENT="Text.Monograph"><METANAME="DC.format"CONTENT="text/html"SCHEME="MIME"><METANAME="DC.source"CONTENT="1-56592-243-3"SCHEME="ISBN"><METANAME="DC.language"CONTENT="en-US"><METANAME="generator"CONTENT="Jade 1.1/O'Reilly DocBook 3.0 to HTML 4.0"><LINKREV="made"HREF="mailto:online-books@oreilly.com"TITLE="Online Books Comments"><LINKREL="up"HREF="ch20_01.htm"TITLE="20. Web Automation"><LINKREL="prev"HREF="ch20_12.htm"TITLE="20.11. Creating a Robot"><LINKREL="next"HREF="ch20_14.htm"TITLE="20.13. Processing Server Logs"></HEAD><BODYBGCOLOR="#FFFFFF"><img alt="Book Home" border="0" src="gifs/smbanner.gif" usemap="#banner-map" /><map name="banner-map"><area shape="rect" coords="1,-2,616,66" href="index.htm" alt="Perl Cookbook"><area shape="rect" coords="629,-11,726,25" href="jobjects/fsearch.htm" alt="Search this book" /></map><div class="navbar"><p><TABLEWIDTH="684"BORDER="0"CELLSPACING="0"CELLPADDING="0"><TR><TDALIGN="LEFT"VALIGN="TOP"WIDTH="228"><ACLASS="sect1"HREF="ch20_12.htm"TITLE="20.11. Creating a Robot"><IMGSRC="../gifs/txtpreva.gif"ALT="Previous: 20.11. Creating a Robot"BORDER="0"></A></TD><TDALIGN="CENTER"VALIGN="TOP"WIDTH="228"><B><FONTFACE="ARIEL,HELVETICA,HELV,SANSERIF"SIZE="-1"><ACLASS="chapter"REL="up"HREF="ch20_01.htm"TITLE="20. Web Automation"></A></FONT></B></TD><TDALIGN="RIGHT"VALIGN="TOP"WIDTH="228"><ACLASS="sect1"HREF="ch20_14.htm"TITLE="20.13. Processing Server Logs"><IMGSRC="../gifs/txtnexta.gif"ALT="Next: 20.13. Processing Server Logs"BORDER="0"></A></TD></TR></TABLE></DIV><DIVCLASS="sect1"><H2CLASS="sect1"><ACLASS="title"NAME="ch20-30943">20.12. Parsing a Web Server Log File</A></H2><DIVCLASS="sect2"><H3CLASS="sect2"><ACLASS="title"NAME="ch20-pgfId-1327">Problem<ACLASS="indexterm"NAME="ch20-idx-1000002674-0"></A><ACLASS="indexterm"NAME="ch20-idx-1000002674-1"></A><ACLASS="indexterm"NAME="ch20-idx-1000002674-2"></A><ACLASS="indexterm"NAME="ch20-idx-1000002674-3"></A><ACLASS="indexterm"NAME="ch20-idx-1000002674-4"></A><ACLASS="indexterm"NAME="ch20-idx-1000002674-5"></A></A></H3><PCLASS="para">You want to extract from a web server log file only the information you're interested in.</P></DIV><DIVCLASS="sect2"><H3CLASS="sect2"><ACLASS="title"NAME="ch20-pgfId-1333">Solution</A></H3><PCLASS="para">Pull apart the log file as follows:</P><PRECLASS="programlisting">while (<LOGFILE>) { my ($client, $identuser, $authuser, $date, $time, $tz, $method, $url, $protocol, $status, $bytes) = /^(\S+) (\S+) (\S+) \[([^:]+):(\d+:\d+:\d+) ([^\]]+) "(\S+) (.*?) (\S+)" (\S+) (\S+)$/; # ...}</PRE></DIV><DIVCLASS="sect2"><H3CLASS="sect2"><ACLASS="title"NAME="ch20-pgfId-1351">Discussion</A></H3><PCLASS="para">This regular expression pulls apart entries in <ACLASS="indexterm"NAME="ch20-idx-1000002682-0"></A>Common Log Format, an informal standard that most web servers adhere to. The fields are:</P><DLCLASS="variablelist"><DTCLASS="term">client</DT><DDCLASS="listitem"><PCLASS="para">IP address or domain name of browser's machine</P></DD><DTCLASS="term">identuser</DT><DDCLASS="listitem"><PCLASS="para">If IDENT (RFC 1413) was used, what it returned</P></DD><DTCLASS="term">authuser</DT><DDCLASS="listitem"><PCLASS="para">If username/password authentication was used, whom they logged in as</P></DD><DTCLASS="term">date</DT><DDCLASS="listitem"><PCLASS="para">Date of request (e.g., 01/Mar/1997)</P></DD><DTCLASS="term">time</DT><DDCLASS="listitem"><PCLASS="para">Time of request (e.g., 12:55:36)</P></DD><DTCLASS="term">tz</DT><DDCLASS="listitem"><PCLASS="para">Time zone (e.g., -0700)</P></DD><DTCLASS="term">method</DT><DDCLASS="listitem"><PCLASS="para">Method of request (e.g., GET, POST, or PUT)</P></DD><DTCLASS="term">url</DT><DDCLASS="listitem"><PCLASS="para">URL in request (e.g., <EMCLASS="emphasis">/~user/index.html</EM>)</P></DD><DTCLASS="term">protocol</DT><DDCLASS="listitem"><PCLASS="para">HTTP/1.0 or HTTP/1.1</P></DD><DTCLASS="term">status</DT><DDCLASS="listitem"><PCLASS="para">Returned status (200 is okay, 500 is server error)</P></DD><DTCLASS="term">bytes</DT><DDCLASS="listitem"><PCLASS="para">Number of bytes returned (could be <CODECLASS="literal">"-"</CODE> for errors, redirects, and other non-document transfers)</P></DD></DL><PCLASS="para">Other formats include the referrer and agent information. The pattern needs only minor changes for it to work with other log file formats. Watch out that spaces in the URL field are not escaped. This means that we can't use <CODECLASS="literal">\S*</CODE> to extract the URL. <CODECLASS="literal">.*</CODE> would cause the regex to match the entire string and then backtrack until it could satisfy the rest of the pattern. We use <CODECLASS="literal">.*?</CODE> and anchor the pattern to the end of the string with <CODECLASS="literal">$</CODE> to make the regular expression engine match nothing and then add characters until the entire pattern is satisfied.<ACLASS="indexterm"NAME="ch20-idx-1000002684-0"></A><ACLASS="indexterm"NAME="ch20-idx-1000002684-1"></A><ACLASS="indexterm"NAME="ch20-idx-1000002684-2"></A><ACLASS="indexterm"NAME="ch20-idx-1000002684-3"></A></P></DIV><DIVCLASS="sect2"><H3CLASS="sect2"><ACLASS="title"NAME="ch20-pgfId-1385">See Also</A></H3><PCLASS="para">The CLF spec at <ACLASS="systemitem.url"HREF="http://www.w3.org/Daemon/User/Config/Logging.html">http://www.w3.org/Daemon/User/Config/Logging.html</A></P></DIV></DIV><DIVCLASS="htmlnav"><P></P><HRALIGN="LEFT"WIDTH="684"TITLE="footer"><TABLEWIDTH="684"BORDER="0"CELLSPACING="0"CELLPADDING="0"><TR><TDALIGN="LEFT"VALIGN="TOP"WIDTH="228"><ACLASS="sect1"HREF="ch20_12.htm"TITLE="20.11. Creating a Robot"><IMGSRC="../gifs/txtpreva.gif"ALT="Previous: 20.11. Creating a Robot"BORDER="0"></A></TD><TDALIGN="CENTER"VALIGN="TOP"WIDTH="228"><ACLASS="book"HREF="index.htm"TITLE="Perl Cookbook"><IMGSRC="../gifs/txthome.gif"ALT="Perl Cookbook"BORDER="0"></A></TD><TDALIGN="RIGHT"VALIGN="TOP"WIDTH="228"><ACLASS="sect1"HREF="ch20_14.htm"TITLE="20.13. Processing Server Logs"><IMGSRC="../gifs/txtnexta.gif"ALT="Next: 20.13. Processing Server Logs"BORDER="0"></A></TD></TR><TR><TDALIGN="LEFT"VALIGN="TOP"WIDTH="228">20.11. Creating a Robot</TD><TDALIGN="CENTER"VALIGN="TOP"WIDTH="228"><ACLASS="index"HREF="index/index.htm"TITLE="Book Index"><IMGSRC="../gifs/index.gif"ALT="Book Index"BORDER="0"></A></TD><TDALIGN="RIGHT"VALIGN="TOP"WIDTH="228">20.13. Processing Server Logs</TD></TR></TABLE><HRALIGN="LEFT"WIDTH="684"TITLE="footer"><FONTSIZE="-1"></DIV<!-- LIBRARY NAV BAR --> <img src="../gifs/smnavbar.gif" usemap="#library-map" border="0" alt="Library Navigation Links"><p> <a href="copyrght.htm">Copyright © 2002</a> O'Reilly & Associates. All rights reserved.</font> </p> <map name="library-map"> <area shape="rect" coords="1,0,85,94" href="../index.htm"><area shape="rect" coords="86,1,178,103" href="../lwp/index.htm"><area shape="rect" coords="180,0,265,103" href="../lperl/index.htm"><area shape="rect" coords="267,0,353,105" href="../perlnut/index.htm"><area shape="rect" coords="354,1,446,115" href="../prog/index.htm"><area shape="rect" coords="448,0,526,132" href="../tk/index.htm"><area shape="rect" coords="528,1,615,119" href="../cookbook/index.htm"><area shape="rect" coords="617,0,690,135" href="../pxml/index.htm"></map> </BODY></HTML>⌨️ 快捷键说明
复制代码
Ctrl + C
搜索代码
Ctrl + F
全屏模式
F11
切换主题
Ctrl + Shift + D
显示快捷键
?
增大字号
Ctrl + =
减小字号
Ctrl + -