⭐ 欢迎来到虫虫下载站! | 📦 资源下载 📁 资源专辑 ℹ️ 关于我们
⭐ 虫虫下载站

📄 writerpoolmember.html

📁 一个开源的网页爬虫一个开源的网页爬虫一个开源的网页爬虫一个开源的网页爬虫一个开源的网页爬虫一个开源的网页爬虫
💻 HTML
📖 第 1 页 / 共 3 页
字号:
</TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected &nbsp;<A HREF="../../../org/archive/util/TimestampSerialno.html" title="class in org.archive.util">TimestampSerialno</A></CODE></FONT></TD><TD><CODE><B><A HREF="../../../org/archive/io/WriterPoolMember.html#getTimestampSerialNo(java.lang.String)">getTimestampSerialNo</A></B>(java.lang.String&nbsp;timestamp)</CODE><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Do static synchronization around getting of counter and timestamp so no chance of a thread getting in between the getting of timestamp and allocation of serial number throwing the two out of alignment.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>&nbsp;boolean</CODE></FONT></TD><TD><CODE><B><A HREF="../../../org/archive/io/WriterPoolMember.html#isCompressed()">isCompressed</A></B>()</CODE><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected &nbsp;void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../org/archive/io/WriterPoolMember.html#postWriteRecordTasks()">postWriteRecordTasks</A></B>()</CODE><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Post file write tasks.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected &nbsp;void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../org/archive/io/WriterPoolMember.html#preWriteRecordTasks()">preWriteRecordTasks</A></B>()</CODE><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Post write tasks.</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected &nbsp;void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../org/archive/io/WriterPoolMember.html#readFullyFrom(java.io.InputStream, long, byte[])">readFullyFrom</A></B>(java.io.InputStream&nbsp;is,              long&nbsp;recordLength,              byte[]&nbsp;b)</CODE><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected &nbsp;void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../org/archive/io/WriterPoolMember.html#write(byte[])">write</A></B>(byte[]&nbsp;b)</CODE><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected &nbsp;void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../org/archive/io/WriterPoolMember.html#write(byte[], int, int)">write</A></B>(byte[]&nbsp;b,      int&nbsp;off,      int&nbsp;len)</CODE><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD ALIGN="right" VALIGN="top" WIDTH="1%"><FONT SIZE="-1"><CODE>protected &nbsp;void</CODE></FONT></TD><TD><CODE><B><A HREF="../../../org/archive/io/WriterPoolMember.html#write(int)">write</A></B>(int&nbsp;b)</CODE><BR>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</TD></TR></TABLE>&nbsp;<A NAME="methods_inherited_from_class_java.lang.Object"><!-- --></A><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY=""><TR BGCOLOR="#EEEEFF" CLASS="TableSubHeadingColor"><TH ALIGN="left"><B>Methods inherited from class java.lang.Object</B></TH></TR><TR BGCOLOR="white" CLASS="TableRowColor"><TD><CODE>clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait</CODE></TD></TR></TABLE>&nbsp;<P><!-- ============ FIELD DETAIL =========== --><A NAME="field_detail"><!-- --></A><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY=""><TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor"><TH ALIGN="left" COLSPAN="1"><FONT SIZE="+2"><B>Field Detail</B></FONT></TH></TR></TABLE><A NAME="UTF8"><!-- --></A><H3>UTF8</H3><PRE>public static final java.lang.String <B>UTF8</B></PRE><DL><DL><DT><B>See Also:</B><DD><A HREF="../../../constant-values.html#org.archive.io.WriterPoolMember.UTF8">Constant Field Values</A></DL></DL><HR><A NAME="DEFAULT_PREFIX"><!-- --></A><H3>DEFAULT_PREFIX</H3><PRE>public static final java.lang.String <B>DEFAULT_PREFIX</B></PRE><DL><DD>Default file prefix.  Stands for Internet Archive Heritrix.<P><DL><DT><B>See Also:</B><DD><A HREF="../../../constant-values.html#org.archive.io.WriterPoolMember.DEFAULT_PREFIX">Constant Field Values</A></DL></DL><HR><A NAME="HOSTNAME_VARIABLE"><!-- --></A><H3>HOSTNAME_VARIABLE</H3><PRE>public static final java.lang.String <B>HOSTNAME_VARIABLE</B></PRE><DL><DD>Value to interpolate with actual hostname.<P><DL><DT><B>See Also:</B><DD><A HREF="../../../constant-values.html#org.archive.io.WriterPoolMember.HOSTNAME_VARIABLE">Constant Field Values</A></DL></DL><HR><A NAME="DEFAULT_SUFFIX"><!-- --></A><H3>DEFAULT_SUFFIX</H3><PRE>public static final java.lang.String <B>DEFAULT_SUFFIX</B></PRE><DL><DD>Default for file suffix.<P><DL><DT><B>See Also:</B><DD><A HREF="../../../constant-values.html#org.archive.io.WriterPoolMember.DEFAULT_SUFFIX">Constant Field Values</A></DL></DL><!-- ========= CONSTRUCTOR DETAIL ======== --><A NAME="constructor_detail"><!-- --></A><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY=""><TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor"><TH ALIGN="left" COLSPAN="1"><FONT SIZE="+2"><B>Constructor Detail</B></FONT></TH></TR></TABLE><A NAME="WriterPoolMember(java.util.concurrent.atomic.AtomicInteger, java.io.OutputStream, java.io.File, boolean, java.lang.String)"><!-- --></A><H3>WriterPoolMember</H3><PRE>protected <B>WriterPoolMember</B>(java.util.concurrent.atomic.AtomicInteger&nbsp;serialNo,                           java.io.OutputStream&nbsp;out,                           java.io.File&nbsp;file,                           boolean&nbsp;cmprs,                           java.lang.String&nbsp;a14DigitDate)                    throws java.io.IOException</PRE><DL><DD>Constructor. Takes a stream. Use with caution. There is no upperbound check on size. Will just keep writing.<P><DL><DT><B>Parameters:</B><DD><CODE>serialNo</CODE> - used to create unique filename sequences<DD><CODE>out</CODE> - Where to write.<DD><CODE>file</CODE> - File the <code>out</code> is connected to.<DD><CODE>cmprs</CODE> - Compress the content written.<DD><CODE>a14DigitDate</CODE> - If null, we'll write current time.<DT><B>Throws:</B><DD><CODE>java.io.IOException</CODE></DL></DL><HR><A NAME="WriterPoolMember(java.util.concurrent.atomic.AtomicInteger, java.util.List, java.lang.String, boolean, int, java.lang.String)"><!-- --></A><H3>WriterPoolMember</H3><PRE>public <B>WriterPoolMember</B>(java.util.concurrent.atomic.AtomicInteger&nbsp;serialNo,                        java.util.List&lt;java.io.File&gt;&nbsp;dirs,                        java.lang.String&nbsp;prefix,                        boolean&nbsp;cmprs,                        int&nbsp;maxSize,                        java.lang.String&nbsp;extension)</PRE><DL><DD>Constructor.<P><DL><DT><B>Parameters:</B><DD><CODE>serialNo</CODE> - used to create unique filename sequences<DD><CODE>dirs</CODE> - Where to drop files.<DD><CODE>prefix</CODE> - File prefix to use.<DD><CODE>cmprs</CODE> - Compress the records written.<DD><CODE>maxSize</CODE> - Maximum size for ARC files written.<DD><CODE>extension</CODE> - Extension to give file.</DL></DL><HR><A NAME="WriterPoolMember(java.util.concurrent.atomic.AtomicInteger, java.util.List, java.lang.String, java.lang.String, boolean, int, java.lang.String)"><!-- --></A><H3>WriterPoolMember</H3><PRE>public <B>WriterPoolMember</B>(java.util.concurrent.atomic.AtomicInteger&nbsp;serialNo,                        java.util.List&lt;java.io.File&gt;&nbsp;dirs,                        java.lang.String&nbsp;prefix,                        java.lang.String&nbsp;suffix,                        boolean&nbsp;cmprs,                        int&nbsp;maxSize,                        java.lang.String&nbsp;extension)</PRE><DL><DD>Constructor.<P><DL><DT><B>Parameters:</B><DD><CODE>serialNo</CODE> - used to create unique filename sequences<DD><CODE>dirs</CODE> - Where to drop files.<DD><CODE>prefix</CODE> - File prefix to use.<DD><CODE>cmprs</CODE> - Compress the records written.<DD><CODE>maxSize</CODE> - Maximum size for ARC files written.<DD><CODE>suffix</CODE> - File tail to use.  If null, unused.<DD><CODE>extension</CODE> - Extension to give file.</DL></DL><!-- ============ METHOD DETAIL ========== --><A NAME="method_detail"><!-- --></A><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY=""><TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor"><TH ALIGN="left" COLSPAN="1"><FONT SIZE="+2"><B>Method Detail</B></FONT></TH></TR></TABLE><A NAME="checkSize()"><!-- --></A><H3>checkSize</H3><PRE>public void <B>checkSize</B>()               throws java.io.IOException</PRE><DL><DD>Call this method just before/after any significant write. Call at the end of the writing of a record or just before we start writing a new record.  Will close current file and open a new file if file size has passed out maxSize.  <p>Creates and opens a file if none already open.  One use of this method then is after construction, call this method to add the metadata, then call <A HREF="../../../org/archive/io/WriterPoolMember.html#getPosition()"><CODE>getPosition()</CODE></A> to find offset of first record.<P><DD><DL></DL></DD><DD><DL><DT><B>Throws:</B><DD><CODE>java.io.IOException</CODE></DL></DD></DL><HR><A NAME="createFile()"><!-- --></A><H3>createFile</H3><PRE>protected java.lang.String <B>createFile</B>()                               throws java.io.IOException</PRE><DL><DD>Create a new file. Rotates off the current Writer and creates a new in its place to take subsequent writes.  Usually called from <A HREF="../../../org/archive/io/WriterPoolMember.html#checkSize()"><CODE>checkSize()</CODE></A>.<P><DD><DL></DL></DD><DD><DL><DT><B>Returns:</B><DD>Name of file created.<DT><B>Throws:</B><DD><CODE>java.io.IOException</CODE></DL></DD></DL><HR><A NAME="createFile(java.io.File)"><!-- --></A><H3>createFile</H3><PRE>protected java.lang.String <B>createFile</B>(java.io.File&nbsp;file)                               throws java.io.IOException</PRE><DL><DD><DL></DL></DD><DD><DL><DT><B>Throws:</B><DD><CODE>java.io.IOException</CODE></DL></DD></DL><HR><A NAME="getNextDirectory(java.util.List)"><!-- --></A><H3>getNextDirectory</H3><PRE>protected java.io.File <B>getNextDirectory</B>(java.util.List&lt;java.io.File&gt;&nbsp;dirs)                                 throws java.io.IOException</PRE><DL><DD><DL></DL></DD><DD><DL><DT><B>Parameters:</B><DD><CODE>dirs</CODE> - List of File objects that point at directories.<DT><B>Returns:</B><DD>Find next directory to write an arc too.  If more than one, it tries to round-robin through each in turn.<DT><B>Throws:</B><DD><CODE>java.io.IOException</CODE></DL></DD></DL><HR><A NAME="checkWriteable(java.io.File)"><!-- --></A><H3>checkWriteable</H3><PRE>protected java.io.File <B>checkWriteable</B>(java.io.File&nbsp;d)</PRE><DL><DD><DL></DL></DD><DD><DL></DL></DD></DL><HR><A NAME="getTimestampSerialNo()"><!-- --></A><H3>getTimestampSerialNo</H3><PRE>protected <A HREF="../../../org/archive/util/TimestampSerialno.html" title="class in org.archive.util">TimestampSerialno</A> <B>getTimestampSerialNo</B>()</PRE><DL><DD><DL></DL></DD><DD><DL></DL></DD></DL><HR><A NAME="getTimestampSerialNo(java.lang.String)"><!-- --></A><H3>getTimestampSerialNo</H3><PRE>protected <A HREF="../../../org/archive/util/TimestampSerialno.html" title="class in org.archive.util">TimestampSerialno</A> <B>getTimestampSerialNo</B>(java.lang.String&nbsp;timestamp)</PRE>

⌨️ 快捷键说明

复制代码 Ctrl + C
搜索代码 Ctrl + F
全屏模式 F11
切换主题 Ctrl + Shift + D
显示快捷键 ?
增大字号 Ctrl + =
减小字号 Ctrl + -