tokensources.html
来自「 Lucene是apache软件基金会[4] jakarta项目组的一个子项目」· HTML 代码 · 共 393 行 · 第 1/2 页
HTML
393 行
</TABLE>
<P>
<!-- ============ FIELD DETAIL =========== -->
<!-- ========= CONSTRUCTOR DETAIL ======== -->
<A NAME="constructor_detail"><!-- --></A><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor">
<TD COLSPAN=1><FONT SIZE="+2">
<B>Constructor Detail</B></FONT></TD>
</TR>
</TABLE>
<A NAME="TokenSources()"><!-- --></A><H3>
TokenSources</H3>
<PRE>
public <B>TokenSources</B>()</PRE>
<DL>
</DL>
<!-- ============ METHOD DETAIL ========== -->
<A NAME="method_detail"><!-- --></A><TABLE BORDER="1" WIDTH="100%" CELLPADDING="3" CELLSPACING="0" SUMMARY="">
<TR BGCOLOR="#CCCCFF" CLASS="TableHeadingColor">
<TD COLSPAN=1><FONT SIZE="+2">
<B>Method Detail</B></FONT></TD>
</TR>
</TABLE>
<A NAME="getAnyTokenStream(org.apache.lucene.index.IndexReader, int, java.lang.String, org.apache.lucene.analysis.Analyzer)"><!-- --></A><H3>
getAnyTokenStream</H3>
<PRE>
public static <A HREF="../../../../../org/apache/lucene/analysis/TokenStream.html" title="class in org.apache.lucene.analysis">TokenStream</A> <B>getAnyTokenStream</B>(<A HREF="../../../../../org/apache/lucene/index/IndexReader.html" title="class in org.apache.lucene.index">IndexReader</A> reader, int docId, <A HREF="http://java.sun.com/j2se/1.4/docs/api/java/lang/String.html" title="class or interface in java.lang">String</A> field, <A HREF="../../../../../org/apache/lucene/analysis/Analyzer.html" title="class in org.apache.lucene.analysis">Analyzer</A> analyzer) throws <A HREF="http://java.sun.com/j2se/1.4/docs/api/java/io/IOException.html" title="class or interface in java.io">IOException</A></PRE>
<DL>
<DD>A convenience method that tries a number of approaches to getting a token stream. The cost of finding there are no termVectors in the index is minimal (1000 invocations still registers 0 ms). So this "lazy" (flexible?) approach to coding is probably acceptable
<P>
<DD><DL>
<DT><B>Parameters:</B><DD><CODE>reader</CODE> - <DD><CODE>docId</CODE> - <DD><CODE>field</CODE> - <DD><CODE>analyzer</CODE> - <DT><B>Returns:</B><DD>null if field not stored correctly<DT><B>Throws:</B><DD><CODE><A HREF="http://java.sun.com/j2se/1.4/docs/api/java/io/IOException.html" title="class or interface in java.io">IOException</A></CODE></DL>
</DD>
</DL>
<HR>
<A NAME="getTokenStream(org.apache.lucene.index.TermPositionVector)"><!-- --></A><H3>
getTokenStream</H3>
<PRE>
public static <A HREF="../../../../../org/apache/lucene/analysis/TokenStream.html" title="class in org.apache.lucene.analysis">TokenStream</A> <B>getTokenStream</B>(<A HREF="../../../../../org/apache/lucene/index/TermPositionVector.html" title="interface in org.apache.lucene.index">TermPositionVector</A> tpv)</PRE>
<DL>
<DD><DL>
</DL>
</DD>
</DL>
<HR>
<A NAME="getTokenStream(org.apache.lucene.index.TermPositionVector, boolean)"><!-- --></A><H3>
getTokenStream</H3>
<PRE>
public static <A HREF="../../../../../org/apache/lucene/analysis/TokenStream.html" title="class in org.apache.lucene.analysis">TokenStream</A> <B>getTokenStream</B>(<A HREF="../../../../../org/apache/lucene/index/TermPositionVector.html" title="interface in org.apache.lucene.index">TermPositionVector</A> tpv, boolean tokenPositionsGuaranteedContiguous)</PRE>
<DL>
<DD>Low level api. Returns a token stream or null if no offset info available in index. This can be used to feed the highlighter with a pre-parsed token stream In my tests the speeds to recreate 1000 token streams using this method are: - with TermVector offset only data stored - 420 milliseconds - with TermVector offset AND position data stored - 271 milliseconds (nb timings for TermVector with position data are based on a tokenizer with contiguous positions - no overlaps or gaps) The cost of not using TermPositionVector to store pre-parsed content and using an analyzer to re-parse the original content: - reanalyzing the original content - 980 milliseconds The re-analyze timings will typically vary depending on - 1) The complexity of the analyzer code (timings above were using a stemmer/lowercaser/stopword combo) 2) The number of other fields (Lucene reads ALL fields off the disk when accessing just one document field - can cost dear!) 3) Use of compression on field storage - could be faster cos of compression (less disk IO) or slower (more CPU burn) depending on the content.
<P>
<DD><DL>
<DT><B>Parameters:</B><DD><CODE>tpv</CODE> - <DD><CODE>tokenPositionsGuaranteedContiguous</CODE> - true if the token position numbers have no overlaps or gaps. If looking to eek out the last drops of performance, set to true. If in doubt, set to false.</DL>
</DD>
</DL>
<HR>
<A NAME="getTokenStream(org.apache.lucene.index.IndexReader, int, java.lang.String)"><!-- --></A><H3>
getTokenStream</H3>
<PRE>
public static <A HREF="../../../../../org/apache/lucene/analysis/TokenStream.html" title="class in org.apache.lucene.analysis">TokenStream</A> <B>getTokenStream</B>(<A HREF="../../../../../org/apache/lucene/index/IndexReader.html" title="class in org.apache.lucene.index">IndexReader</A> reader, int docId, <A HREF="http://java.sun.com/j2se/1.4/docs/api/java/lang/String.html" title="class or interface in java.lang">String</A> field) throws <A HREF="http://java.sun.com/j2se/1.4/docs/api/java/io/IOException.html" title="class or interface in java.io">IOException</A></PRE>
<DL>
<DD><DL>
<DT><B>Throws:</B><DD><CODE><A HREF="http://java.sun.com/j2se/1.4/docs/api/java/io/IOException.html" title="class or interface in java.io">IOException</A></CODE></DL>
</DD>
</DL>
<HR>
<A NAME="getTokenStream(org.apache.lucene.index.IndexReader, int, java.lang.String, org.apache.lucene.analysis.Analyzer)"><!-- --></A><H3>
getTokenStream</H3>
<PRE>
public static <A HREF="../../../../../org/apache/lucene/analysis/TokenStream.html" title="class in org.apache.lucene.analysis">TokenStream</A> <B>getTokenStream</B>(<A HREF="../../../../../org/apache/lucene/index/IndexReader.html" title="class in org.apache.lucene.index">IndexReader</A> reader, int docId, <A HREF="http://java.sun.com/j2se/1.4/docs/api/java/lang/String.html" title="class or interface in java.lang">String</A> field, <A HREF="../../../../../org/apache/lucene/analysis/Analyzer.html" title="class in org.apache.lucene.analysis">Analyzer</A> analyzer) throws <A HREF="http://java.sun.com/j2se/1.4/docs/api/java/io/IOException.html" title="class or interface in java.io">IOException</A></PRE>
<DL>
<DD><DL>
<DT><B>Throws:</B><DD><CODE><A HREF="http://java.sun.com/j2se/1.4/docs/api/java/io/IOException.html" title="class or interface in java.io">IOException</A></CODE></DL>
</DD>
</DL>
<!-- ========= END OF CLASS DATA ========= -->
<HR>
<!-- ======= START OF BOTTOM NAVBAR ====== -->
<A NAME="navbar_bottom"><!-- --></A><A HREF="#skip-navbar_bottom" title="Skip navigation links"></A><TABLE BORDER="0" WIDTH="100%" CELLPADDING="1" CELLSPACING="0" SUMMARY="">
<TR>
<TD COLSPAN=3 BGCOLOR="#EEEEFF" CLASS="NavBarCell1">
<A NAME="navbar_bottom_firstrow"><!-- --></A><TABLE BORDER="0" CELLPADDING="0" CELLSPACING="3" SUMMARY="">
<TR ALIGN="center" VALIGN="top">
<TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1"> <A HREF="../../../../../overview-summary.html"><FONT CLASS="NavBarFont1"><B>Overview</B></FONT></A> </TD>
<TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1"> <A HREF="package-summary.html"><FONT CLASS="NavBarFont1"><B>Package</B></FONT></A> </TD>
<TD BGCOLOR="#FFFFFF" CLASS="NavBarCell1Rev"> <FONT CLASS="NavBarFont1Rev"><B>Class</B></FONT> </TD>
<TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1"> <A HREF="class-use/TokenSources.html"><FONT CLASS="NavBarFont1"><B>Use</B></FONT></A> </TD>
<TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1"> <A HREF="package-tree.html"><FONT CLASS="NavBarFont1"><B>Tree</B></FONT></A> </TD>
<TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1"> <A HREF="../../../../../deprecated-list.html"><FONT CLASS="NavBarFont1"><B>Deprecated</B></FONT></A> </TD>
<TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1"> <A HREF="../../../../../index-all.html"><FONT CLASS="NavBarFont1"><B>Index</B></FONT></A> </TD>
<TD BGCOLOR="#EEEEFF" CLASS="NavBarCell1"> <A HREF="../../../../../help-doc.html"><FONT CLASS="NavBarFont1"><B>Help</B></FONT></A> </TD>
</TR>
</TABLE>
</TD>
<TD ALIGN="right" VALIGN="top" ROWSPAN=3><EM>
</EM>
</TD>
</TR>
<TR>
<TD BGCOLOR="white" CLASS="NavBarCell2"><FONT SIZE="-2">
<A HREF="../../../../../org/apache/lucene/search/highlight/TokenGroup.html" title="class in org.apache.lucene.search.highlight"><B>PREV CLASS</B></A>
<A HREF="../../../../../org/apache/lucene/search/highlight/WeightedTerm.html" title="class in org.apache.lucene.search.highlight"><B>NEXT CLASS</B></A></FONT></TD>
<TD BGCOLOR="white" CLASS="NavBarCell2"><FONT SIZE="-2">
<A HREF="../../../../../index.html" target="_top"><B>FRAMES</B></A>
<A HREF="TokenSources.html" target="_top"><B>NO FRAMES</B></A>
<SCRIPT type="text/javascript">
<!--
if(window==top) {
document.writeln('<A HREF="../../../../../allclasses-noframe.html"><B>All Classes</B></A>');
}
//-->
</SCRIPT>
<NOSCRIPT>
<A HREF="../../../../../allclasses-noframe.html"><B>All Classes</B></A>
</NOSCRIPT>
</FONT></TD>
</TR>
<TR>
<TD VALIGN="top" CLASS="NavBarCell3"><FONT SIZE="-2">
SUMMARY: NESTED | FIELD | <A HREF="#constructor_summary">CONSTR</A> | <A HREF="#method_summary">METHOD</A></FONT></TD>
<TD VALIGN="top" CLASS="NavBarCell3"><FONT SIZE="-2">
DETAIL: FIELD | <A HREF="#constructor_detail">CONSTR</A> | <A HREF="#method_detail">METHOD</A></FONT></TD>
</TR>
</TABLE>
<A NAME="skip-navbar_bottom"></A><!-- ======== END OF BOTTOM NAVBAR ======= -->
<HR>
Copyright © 2000-2007 Apache Software Foundation. All Rights Reserved.
</BODY>
</HTML>
⌨️ 快捷键说明
复制代码Ctrl + C
搜索代码Ctrl + F
全屏模式F11
增大字号Ctrl + =
减小字号Ctrl + -
显示快捷键?