Package org.apache.lucene.backward_codecs.lucene90

Class Lucene90HnswVectorsFormat

  • All Implemented Interfaces:
    NamedSPILoader.NamedSPI
    public class Lucene90HnswVectorsFormat
extends KnnVectorsFormat
    Lucene 9.0 vector format, which encodes numeric vector values and an optional associated graph connecting the documents having values. The graph is used to power HNSW search. The format consists of three files:

    .vec (vector data) file

    This file stores all the floating-point vector data ordered by field, document ordinal, and vector dimension. The floats are stored in little-endian byte order.

    .vex (vector index) file

    Stores graphs connecting the documents for each field. For each document having a vector for a given field, this is stored as:

    • [int32] the number of neighbor nodes
    • array[vint] the neighbor ordinals, delta-encoded (initially subtracting -1)

    .vem (vector metadata) file

    For each field:

    • [int32] field number
    • [int32] vector similarity function ordinal
    • [vlong] offset to this field's vectors in the .vec file
    • [vlong] length of this field's vectors, in bytes
    • [vlong] offset to this field's index in the .vex file
    • [vlong] length of this field's index data, in bytes
    • [int] dimension of this field's vectors
    • [int] the number of documents having values for this field
    • array[vint] the docids of documents having vectors, in order
    • array[vlong] for each document having a vector, the offset (delta-encoded relative to the previous document) of its entry in the .vex file

    • Field Detail

      • VECTOR_DATA_CODEC_NAME

        static final java.lang.String VECTOR_DATA_CODEC_NAME
        See Also:
        Constant Field Values

      • VECTOR_INDEX_CODEC_NAME

        static final java.lang.String VECTOR_INDEX_CODEC_NAME
        See Also:
        Constant Field Values

      • VECTOR_DATA_EXTENSION

        static final java.lang.String VECTOR_DATA_EXTENSION
        See Also:
        Constant Field Values

      • VECTOR_INDEX_EXTENSION

        static final java.lang.String VECTOR_INDEX_EXTENSION
        See Also:
        Constant Field Values

      • DEFAULT_MAX_CONN

        public static final int DEFAULT_MAX_CONN
        Default number of maximum connections per node
        See Also:
        Constant Field Values

      • DEFAULT_BEAM_WIDTH

        public static final int DEFAULT_BEAM_WIDTH
        Default number of the size of the queue maintained while searching and the number of random entry points to sample during a graph construction.
        See Also:
        Constant Field Values

      • maxConn

        final int maxConn
        Controls how many of the nearest neighbor candidates are connected to the new node. Defaults to DEFAULT_MAX_CONN. See HnswGraph for more details.

      • beamWidth

        final int beamWidth
        The number of candidate neighbors to track while searching the graph for each newly inserted node. Defaults to to DEFAULT_BEAM_WIDTH. See HnswGraph for details.

    • Constructor Detail

      • Lucene90HnswVectorsFormat

        public Lucene90HnswVectorsFormat()
        A constructor for vectors format with default parameters

      • Lucene90HnswVectorsFormat

        public Lucene90HnswVectorsFormat​(int maxConn,
                                 int beamWidth)
        A constructor for vectors format