Package org.apache.lucene.index

Class IndexingChain

java.lang.Object

org.apache.lucene.index.IndexingChain

All Implemented Interfaces:

Accountable

final class IndexingChain
extends java.lang.Object
implements Accountable

Default general purpose indexing chain, which handles indexing all types of fields.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private static class	IndexingChain.FieldSchema	A schema of the field in the current document.
private static class	IndexingChain.IntBlockAllocator
private class	IndexingChain.PerField	NOTE: not static: accesses at least docState, termsHash.

Field Summary

Fields

Modifier and Type	Field	Description
private java.util.function.Consumer<java.lang.Throwable>	abortingExceptionConsumer
private ByteBlockPool.Allocator	byteBlockAllocator
(package private) Counter	bytesUsed
private IndexingChain.PerField[]	docFields
(package private) ByteBlockPool	docValuesBytePool
private IndexingChain.PerField[]	fieldHash
(package private) FieldInfos.Builder	fieldInfos
private IndexingChain.PerField[]	fields
private boolean	hasHitAbortingException
private int	hashMask
private int	indexCreatedVersionMajor
private LiveIndexWriterConfig	indexWriterConfig
private InfoStream	infoStream
private long	nextFieldGen
(package private) StoredFieldsConsumer	storedFieldsConsumer
(package private) TermsHash	termsHash
(package private) TermVectorsConsumer	termVectorsWriter
private int	totalFieldCount

Fields inherited from interface org.apache.lucene.util.Accountable

NULL_ACCOUNTABLE

Constructor Summary

Constructors

Constructor	Description
IndexingChain(int indexCreatedVersionMajor, SegmentInfo segmentInfo, Directory directory, FieldInfos.Builder fieldInfos, LiveIndexWriterConfig indexWriterConfig, java.util.function.Consumer<java.lang.Throwable> abortingExceptionConsumer)

Method Summary

Modifier and Type	Method	Description
(package private) void	abort()
private void	finishStoredFields()	Calls StoredFieldsWriter.finishDocument, aborting the segment if it hits any exception.
(package private) Sorter.DocMap	flush(SegmentWriteState state)
java.util.Collection<Accountable>	getChildResources()	Returns nested resources of this class.
private LeafReader	getDocValuesLeafReader()
(package private) DocIdSetIterator	getHasDocValues(java.lang.String field)
private IndexingChain.PerField	getOrAddPerField(java.lang.String fieldName, IndexableFieldType fieldType)	Returns a previously created IndexingChain.PerField, absorbing the type information from FieldType, and creates a new IndexingChain.PerField if this field name wasn't seen yet.
private IndexingChain.PerField	getPerField(java.lang.String name)	Returns a previously created IndexingChain.PerField, or null if this field name wasn't seen yet.
private void	indexDocValue(int docID, IndexingChain.PerField fp, DocValuesType dvType, IndexableField field)	Called from processDocument to index one field's doc value
private void	initializeFieldInfo(IndexingChain.PerField pf)
private Sorter.DocMap	maybeSortSegment(SegmentWriteState state)
private void	onAbortingException(java.lang.Throwable th)
private void	oversizeDocFields()
(package private) void	processDocument(int docID, java.lang.Iterable<? extends IndexableField> document)
private boolean	processField(int docID, IndexableField field, IndexingChain.PerField pf)	Index each field Returns true, if we are indexing a unique field with postings
long	ramBytesUsed()	Return the memory usage of this object in bytes.
private void	rehash()
private void	startStoredFields(int docID)	Calls StoredFieldsWriter.startDocument, aborting the segment if it hits any exception.
private static void	updateDocFieldSchema(java.lang.String fieldName, IndexingChain.FieldSchema schema, IndexableFieldType fieldType)
private void	validateIndexSortDVType(Sort indexSort, java.lang.String fieldToValidate, DocValuesType dvType)
private static void	verifyUnIndexedFieldType(java.lang.String name, IndexableFieldType ft)
private void	writeDocValues(SegmentWriteState state, Sorter.DocMap sortMap)	Writes all buffered doc values (called from flush(org.apache.lucene.index.SegmentWriteState)).
private void	writeNorms(SegmentWriteState state, Sorter.DocMap sortMap)
private void	writePoints(SegmentWriteState state, Sorter.DocMap sortMap)	Writes all buffered points.
private void	writeVectors(SegmentWriteState state, Sorter.DocMap sortMap)	Writes all buffered vectors.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

bytesUsed

final Counter bytesUsed

fieldInfos

final FieldInfos.Builder fieldInfos

termsHash

final TermsHash termsHash

docValuesBytePool

final ByteBlockPool docValuesBytePool

storedFieldsConsumer

final StoredFieldsConsumer storedFieldsConsumer

termVectorsWriter

final TermVectorsConsumer termVectorsWriter

fieldHash

private IndexingChain.PerField[] fieldHash

hashMask

private int hashMask

totalFieldCount

private int totalFieldCount

nextFieldGen

private long nextFieldGen

fields

private IndexingChain.PerField[] fields

docFields

private IndexingChain.PerField[] docFields

infoStream

private final InfoStream infoStream

byteBlockAllocator

private final ByteBlockPool.Allocator byteBlockAllocator

indexWriterConfig

private final LiveIndexWriterConfig indexWriterConfig

indexCreatedVersionMajor

private final int indexCreatedVersionMajor

abortingExceptionConsumer

private final java.util.function.Consumer<java.lang.Throwable> abortingExceptionConsumer

hasHitAbortingException

private boolean hasHitAbortingException

Constructor Detail

IndexingChain

IndexingChain(int indexCreatedVersionMajor,
              SegmentInfo segmentInfo,
              Directory directory,
              FieldInfos.Builder fieldInfos,
              LiveIndexWriterConfig indexWriterConfig,
              java.util.function.Consumer<java.lang.Throwable> abortingExceptionConsumer)

Method Detail

onAbortingException

private void onAbortingException(java.lang.Throwable th)

getDocValuesLeafReader

private LeafReader getDocValuesLeafReader()

maybeSortSegment

private Sorter.DocMap maybeSortSegment(SegmentWriteState state)
                                throws java.io.IOException

Throws:

java.io.IOException

flush

Sorter.DocMap flush(SegmentWriteState state)
             throws java.io.IOException

Throws:

java.io.IOException

writePoints

private void writePoints(SegmentWriteState state,
                         Sorter.DocMap sortMap)
                  throws java.io.IOException

Writes all buffered points.

Throws:

java.io.IOException

writeDocValues

private void writeDocValues(SegmentWriteState state,
                            Sorter.DocMap sortMap)
                     throws java.io.IOException

Writes all buffered doc values (called from flush(org.apache.lucene.index.SegmentWriteState)).

Throws:

java.io.IOException

writeVectors

private void writeVectors(SegmentWriteState state,
                          Sorter.DocMap sortMap)
                   throws java.io.IOException

Writes all buffered vectors.

Throws:

java.io.IOException

writeNorms

private void writeNorms(SegmentWriteState state,
                        Sorter.DocMap sortMap)
                 throws java.io.IOException

Throws:

java.io.IOException

abort

void abort()
    throws java.io.IOException

Throws:

java.io.IOException

rehash

private void rehash()

startStoredFields

private void startStoredFields(int docID)
                        throws java.io.IOException

Calls StoredFieldsWriter.startDocument, aborting the segment if it hits any exception.

Throws:

java.io.IOException

finishStoredFields

private void finishStoredFields()
                         throws java.io.IOException

Calls StoredFieldsWriter.finishDocument, aborting the segment if it hits any exception.

Throws:

java.io.IOException

processDocument

void processDocument(int docID,
                     java.lang.Iterable<? extends IndexableField> document)
              throws java.io.IOException

Throws:

java.io.IOException

oversizeDocFields

private void oversizeDocFields()

initializeFieldInfo

private void initializeFieldInfo(IndexingChain.PerField pf)
                          throws java.io.IOException

Throws:

java.io.IOException

processField

private boolean processField(int docID,
                             IndexableField field,
                             IndexingChain.PerField pf)
                      throws java.io.IOException

Index each field Returns true, if we are indexing a unique field with postings

Throws:

java.io.IOException

getOrAddPerField

private IndexingChain.PerField getOrAddPerField(java.lang.String fieldName,
                                                IndexableFieldType fieldType)

Returns a previously created IndexingChain.PerField, absorbing the type information from FieldType, and creates a new IndexingChain.PerField if this field name wasn't seen yet.

updateDocFieldSchema

private static void updateDocFieldSchema(java.lang.String fieldName,
                                         IndexingChain.FieldSchema schema,
                                         IndexableFieldType fieldType)

verifyUnIndexedFieldType

private static void verifyUnIndexedFieldType(java.lang.String name,
                                             IndexableFieldType ft)

validateIndexSortDVType

private void validateIndexSortDVType(Sort indexSort,
                                     java.lang.String fieldToValidate,
                                     DocValuesType dvType)
                              throws java.io.IOException

Throws:

java.io.IOException

indexDocValue

private void indexDocValue(int docID,
                           IndexingChain.PerField fp,
                           DocValuesType dvType,
                           IndexableField field)

Called from processDocument to index one field's doc value

getPerField

private IndexingChain.PerField getPerField(java.lang.String name)

Returns a previously created IndexingChain.PerField, or null if this field name wasn't seen yet.

ramBytesUsed

public long ramBytesUsed()

Description copied from interface: Accountable

Return the memory usage of this object in bytes. Negative values are illegal.

Specified by:

ramBytesUsed in interface Accountable

getChildResources

public java.util.Collection<Accountable> getChildResources()

Description copied from interface: Accountable

Returns nested resources of this class. The result should be a point-in-time snapshot (to avoid race conditions).

Specified by:

getChildResources in interface Accountable

See Also:

Accountables

getHasDocValues

DocIdSetIterator getHasDocValues(java.lang.String field)