Module org.apache.lucene.facet

Package org.apache.lucene.facet.taxonomy.directory

Class DirectoryTaxonomyReader

java.lang.Object

org.apache.lucene.facet.taxonomy.TaxonomyReader

org.apache.lucene.facet.taxonomy.directory.DirectoryTaxonomyReader

All Implemented Interfaces:

Closeable, AutoCloseable, Accountable

public class DirectoryTaxonomyReader
extends TaxonomyReader
implements Accountable

A TaxonomyReader which retrieves stored taxonomy information from a Directory.

Reading from the on-disk index on every method call is too slow, so this implementation employs caching: Some methods cache recent requests and their results, while other methods prefetch all the data into memory and then provide answers directly from in-memory tables. See the documentation of individual methods for comments on their performance.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.facet.taxonomy.TaxonomyReader

TaxonomyReader.ChildrenIterator

Field Summary

Fields

Modifier and Type	Field	Description
private static final long	BYTES_PER_CACHE_ENTRY
private LRUHashMap<Integer,FacetLabel>	categoryCache
private static final int	DEFAULT_CACHE_VALUE
private final DirectoryReader	indexReader
private LRUHashMap<FacetLabel,Integer>	ordinalCache
private TaxonomyIndexArrays	taxoArrays
private final long	taxoEpoch
private final DirectoryTaxonomyWriter	taxoWriter

Fields inherited from class org.apache.lucene.facet.taxonomy.TaxonomyReader

INVALID_ORDINAL, ROOT_ORDINAL

Fields inherited from interface org.apache.lucene.util.Accountable

NULL_ACCOUNTABLE

Constructor Summary

Constructors

Constructor	Description
DirectoryTaxonomyReader(DirectoryTaxonomyWriter taxoWriter)	Opens a DirectoryTaxonomyReader over the given DirectoryTaxonomyWriter (for NRT).
DirectoryTaxonomyReader(DirectoryReader indexReader, DirectoryTaxonomyWriter taxoWriter, LRUHashMap<FacetLabel,Integer> ordinalCache, LRUHashMap<Integer,FacetLabel> categoryCache, TaxonomyIndexArrays taxoArrays)	Expert: Use this method to explicitly force the DirectoryTaxonomyReader to use specific parent/children arrays and caches.
DirectoryTaxonomyReader(Directory directory)	Open for reading a taxonomy stored in a given Directory.

Method Summary

Modifier and Type	Method	Description
private void	checkOrdinalBounds(int... ordinals)	Checks if the ordinals in the array are >=0 and invalid input: '<' DirectoryTaxonomyReader#indexReader.maxDoc()
protected void	doClose()	performs the actual task of closing the resources that are used by the taxonomy reader.
protected DirectoryTaxonomyReader	doOpenIfChanged()	Implements the opening of a new DirectoryTaxonomyReader instance if the taxonomy has changed.
int[]	getBulkOrdinals(FacetLabel... categoryPaths)	Returns the ordinals of the categories given as a path.
FacetLabel[]	getBulkPath(int... ordinals)	Returns an array of FacetLabels for a given array of ordinals.
Collection<Accountable>	getChildResources()	Returns nested resources of this class.
Map<String,String>	getCommitUserData()	Retrieve user committed data.
DirectoryReader	getInternalIndexReader()	Expert: returns the underlying DirectoryReader instance that is used by this TaxonomyReader.
int	getOrdinal(FacetLabel cp)	Returns the ordinal of the category given as a path.
ParallelTaxonomyArrays	getParallelTaxonomyArrays()	Returns a ParallelTaxonomyArrays object which can be used to efficiently traverse the taxonomy tree.
FacetLabel	getPath(int ordinal)	Returns the path name of the category with the given ordinal.
private FacetLabel[]	getPathFromCache(int... ordinals)
int	getSize()	Returns the number of categories in the taxonomy.
protected DirectoryReader	openIndexReader(IndexWriter writer)	Open the DirectoryReader from this IndexWriter.
protected DirectoryReader	openIndexReader(Directory directory)	Open the DirectoryReader from this Directory.
long	ramBytesUsed()	Return the memory usage of this object in bytes.
void	setCacheSize(int size)	setCacheSize controls the maximum allowed size of each of the caches used by getPath(int) and getOrdinal(FacetLabel).
String	toString(int max)	Returns ordinal -> label mapping, up to the provided max ordinal or number of ordinals, whichever is smaller.

Methods inherited from class org.apache.lucene.facet.taxonomy.TaxonomyReader

close, decRef, ensureOpen, getChildren, getOrdinal, getRefCount, incRef, openIfChanged, tryIncRef

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

DEFAULT_CACHE_VALUE

private static final int DEFAULT_CACHE_VALUE

See Also:

• Constant Field Values

BYTES_PER_CACHE_ENTRY

private static final long BYTES_PER_CACHE_ENTRY

taxoWriter

private final DirectoryTaxonomyWriter taxoWriter

taxoEpoch

private final long taxoEpoch

indexReader

private final DirectoryReader indexReader

ordinalCache

private LRUHashMap<FacetLabel,Integer> ordinalCache

categoryCache

private LRUHashMap<Integer,FacetLabel> categoryCache

taxoArrays

private volatile TaxonomyIndexArrays taxoArrays

Constructor Details

DirectoryTaxonomyReader

DirectoryTaxonomyReader(DirectoryReader indexReader,
 DirectoryTaxonomyWriter taxoWriter,
 LRUHashMap<FacetLabel,Integer> ordinalCache,
 LRUHashMap<Integer,FacetLabel> categoryCache,
 TaxonomyIndexArrays taxoArrays)
                 throws IOException

Expert: Use this method to explicitly force the DirectoryTaxonomyReader to use specific parent/children arrays and caches.

Called from doOpenIfChanged(). If the taxonomy has been recreated, you should pass null as the caches and parent/children arrays.

Parameters:

indexReader - An indexReader that is opened in the desired Directory

taxoWriter - The DirectoryTaxonomyWriter from which to obtain newly added categories, in real-time.

ordinalCache - a FacetLabel to Integer ordinal mapping if it already exists

categoryCache - an ordinal to FacetLabel mapping if it already exists

taxoArrays - taxonomy arrays that store the parent, siblings, children information

Throws:

IOException

DirectoryTaxonomyReader

public DirectoryTaxonomyReader(Directory directory)
                        throws IOException

Open for reading a taxonomy stored in a given Directory.

Parameters:

directory - The Directory in which the taxonomy resides.

Throws:

CorruptIndexException - if the Taxonomy is corrupt.

IOException - if another error occurred.

DirectoryTaxonomyReader

public DirectoryTaxonomyReader(DirectoryTaxonomyWriter taxoWriter)
                        throws IOException

Opens a DirectoryTaxonomyReader over the given DirectoryTaxonomyWriter (for NRT).

Parameters:

taxoWriter - The DirectoryTaxonomyWriter from which to obtain newly added categories, in real-time.

Throws:

IOException

Method Details

doClose

protected void doClose()
                throws IOException

Description copied from class: TaxonomyReader

performs the actual task of closing the resources that are used by the taxonomy reader.

Specified by:

doClose in class TaxonomyReader

Throws:

IOException

doOpenIfChanged

protected DirectoryTaxonomyReader doOpenIfChanged()
                                           throws IOException

Implements the opening of a new DirectoryTaxonomyReader instance if the taxonomy has changed.

NOTE: the returned DirectoryTaxonomyReader shares the ordinal and category caches with this reader. This is not expected to cause any issues, unless the two instances continue to live. The reader guarantees that the two instances cannot affect each other in terms of correctness of the caches, however if the size of the cache is changed through setCacheSize(int), it will affect both reader instances.

Specified by:

doOpenIfChanged in class TaxonomyReader

Throws:

IOException

See Also:

• TaxonomyReader.openIfChanged(TaxonomyReader)

openIndexReader

protected DirectoryReader openIndexReader(Directory directory)
                                   throws IOException

Open the DirectoryReader from this Directory.

Throws:

IOException

openIndexReader

protected DirectoryReader openIndexReader(IndexWriter writer)
                                   throws IOException

Open the DirectoryReader from this IndexWriter.

Throws:

IOException

getInternalIndexReader

public DirectoryReader getInternalIndexReader()

Expert: returns the underlying DirectoryReader instance that is used by this TaxonomyReader.

getParallelTaxonomyArrays

public ParallelTaxonomyArrays getParallelTaxonomyArrays()
                                                 throws IOException

Description copied from class: TaxonomyReader

Returns a ParallelTaxonomyArrays object which can be used to efficiently traverse the taxonomy tree.

Specified by:

getParallelTaxonomyArrays in class TaxonomyReader

Throws:

IOException

getCommitUserData

public Map<String,String> getCommitUserData()
                                     throws IOException

Description copied from class: TaxonomyReader

Retrieve user committed data.

Specified by:

getCommitUserData in class TaxonomyReader

Throws:

IOException

See Also:

• TaxonomyWriter.setLiveCommitData(Iterable)

getOrdinal

public int getOrdinal(FacetLabel cp)
               throws IOException

Description copied from class: TaxonomyReader

Returns the ordinal of the category given as a path. The ordinal is the category's serial number, an integer which starts with 0 and grows as more categories are added (note that once a category is added, it can never be deleted).

Specified by:

getOrdinal in class TaxonomyReader

Returns:

the category's ordinal or TaxonomyReader.INVALID_ORDINAL if the category wasn't found.

Throws:

IOException

getBulkOrdinals

public int[] getBulkOrdinals(FacetLabel... categoryPaths)
                      throws IOException

Description copied from class: TaxonomyReader

Returns the ordinals of the categories given as a path. The ordinal is the category's serial number, an integer which starts with 0 and grows as more categories are added (note that once a category is added, it can never be deleted).

The implementation in DirectoryTaxonomyReader is generally faster than iteratively calling TaxonomyReader.getOrdinal(FacetLabel)

Overrides:

getBulkOrdinals in class TaxonomyReader

Returns:

array of the category's' ordinals or TaxonomyReader.INVALID_ORDINAL if the category wasn't found.

Throws:

IOException

getPath

public FacetLabel getPath(int ordinal)
                   throws IOException

Description copied from class: TaxonomyReader

Returns the path name of the category with the given ordinal.

Specified by:

getPath in class TaxonomyReader

Throws:

IOException

getPathFromCache

private FacetLabel[] getPathFromCache(int... ordinals)

checkOrdinalBounds

private void checkOrdinalBounds(int... ordinals)
                         throws IllegalArgumentException

Checks if the ordinals in the array are >=0 and invalid input: '<' DirectoryTaxonomyReader#indexReader.maxDoc()

Parameters:

ordinals - Integer array of ordinals

Throws:

IllegalArgumentException - Throw an IllegalArgumentException if one of the ordinals is out of bounds

getBulkPath

public FacetLabel[] getBulkPath(int... ordinals)
                         throws IOException

Returns an array of FacetLabels for a given array of ordinals.

This API is generally faster than iteratively calling getPath(int) over an array of ordinals. It uses the getPath(int) method iteratively when it detects that the index was created using StoredFields (with no performance gains) and uses DocValues based iteration when the index is based on BinaryDocValues. Lucene switched to BinaryDocValues in version 9.0

Overrides:

getBulkPath in class TaxonomyReader

Parameters:

ordinals - Array of category ordinals that were added to the taxonomy index

Throws:

IOException

getSize

public int getSize()

Description copied from class: TaxonomyReader

Returns the number of categories in the taxonomy. Note that the number of categories returned is often slightly higher than the number of categories inserted into the taxonomy; This is because when a category is added to the taxonomy, its ancestors are also added automatically (including the root, which always get ordinal 0).

Specified by:

getSize in class TaxonomyReader

ramBytesUsed

public long ramBytesUsed()

Description copied from interface: Accountable

Return the memory usage of this object in bytes. Negative values are illegal.

Specified by:

ramBytesUsed in interface Accountable

getChildResources

public Collection<Accountable> getChildResources()

Description copied from interface: Accountable

Returns nested resources of this class. The result should be a point-in-time snapshot (to avoid race conditions).

Specified by:

getChildResources in interface Accountable

See Also:

• Accountables

setCacheSize

public void setCacheSize(int size)

setCacheSize controls the maximum allowed size of each of the caches used by getPath(int) and getOrdinal(FacetLabel).

Currently, if the given size is smaller than the current size of a cache, it will not shrink, and rather we be limited to its current size.

Parameters:

size - the new maximum cache size, in number of entries.

toString

public String toString(int max)

Returns ordinal -> label mapping, up to the provided max ordinal or number of ordinals, whichever is smaller.