Module org.apache.lucene.analysis.nori

Package org.apache.lucene.analysis.ko.dict

Class TokenInfoDictionary

java.lang.Object

org.apache.lucene.analysis.morph.BinaryDictionary<TokenInfoMorphData>

org.apache.lucene.analysis.ko.dict.TokenInfoDictionary

All Implemented Interfaces:

Dictionary<TokenInfoMorphData>

public final class TokenInfoDictionary
extends BinaryDictionary<TokenInfoMorphData>

Binary dictionary implementation for a known-word dictionary model: Words are encoded into an FST mapping to a list of wordIDs.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private static class	TokenInfoDictionary.SingletonHolder

Field Summary

Fields

Modifier and Type	Field	Description
private final TokenInfoFST	fst
static final String	FST_FILENAME_SUFFIX
private final TokenInfoMorphData	morphAtts

Fields inherited from class org.apache.lucene.analysis.morph.BinaryDictionary

buffer, DICT_FILENAME_SUFFIX, POSDICT_FILENAME_SUFFIX, TARGETMAP_FILENAME_SUFFIX

Constructor Summary

Constructors

Modifier	Constructor	Description
private	TokenInfoDictionary()
	TokenInfoDictionary(URL targetMapUrl, URL posDictUrl, URL dictUrl, URL fstUrl)	Create a TokenInfoDictionary from an external resource URL (e.g.
	TokenInfoDictionary(Path targetMapFile, Path posDictFile, Path dictFile, Path fstFile)	Create a TokenInfoDictionary from an external resource path.
private	TokenInfoDictionary(IOSupplier<InputStream> targetMapResource, IOSupplier<InputStream> posResource, IOSupplier<InputStream> dictResource, IOSupplier<InputStream> fstResource)

Method Summary

Modifier and Type	Method	Description
(package private) static InputStream	getClassResource(String suffix)
TokenInfoFST	getFST()
static TokenInfoDictionary	getInstance()
TokenInfoMorphData	getMorphAttributes()

Methods inherited from class org.apache.lucene.analysis.morph.BinaryDictionary

lookupWordIds

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.lucene.analysis.morph.Dictionary

getLeftId, getRightId, getWordCost

Field Details

FST_FILENAME_SUFFIX

public static final String FST_FILENAME_SUFFIX

See Also:

• Constant Field Values

fst

private final TokenInfoFST fst

morphAtts

private final TokenInfoMorphData morphAtts

Constructor Details

TokenInfoDictionary

private TokenInfoDictionary()
                     throws IOException

Throws:

IOException

TokenInfoDictionary

public TokenInfoDictionary(Path targetMapFile,
 Path posDictFile,
 Path dictFile,
 Path fstFile)
                    throws IOException

Create a TokenInfoDictionary from an external resource path.

Parameters:

targetMapFile - where to load target map resource

posDictFile - where to load POS dictionary resource

dictFile - where to load dictionary entries resource

fstFile - where to load encoded FST data resource

Throws:

IOException - if resource was not found or broken

TokenInfoDictionary

public TokenInfoDictionary(URL targetMapUrl,
 URL posDictUrl,
 URL dictUrl,
 URL fstUrl)
                    throws IOException

Create a TokenInfoDictionary from an external resource URL (e.g. from Classpath with ClassLoader.getResource(String)).

Parameters:

targetMapUrl - where to load target map resource

posDictUrl - where to load POS dictionary resource

dictUrl - where to load dictionary entries resource

fstUrl - where to load encoded FST data resource

Throws:

IOException - if resource was not found or broken

TokenInfoDictionary

private TokenInfoDictionary(IOSupplier<InputStream> targetMapResource,
 IOSupplier<InputStream> posResource,
 IOSupplier<InputStream> dictResource,
 IOSupplier<InputStream> fstResource)
                     throws IOException

Throws:

IOException

Method Details

getClassResource

static InputStream getClassResource(String suffix)
                             throws IOException

Throws:

IOException

getFST

public TokenInfoFST getFST()

getInstance

public static TokenInfoDictionary getInstance()

getMorphAttributes

public TokenInfoMorphData getMorphAttributes()