Package org.apache.lucene.codecs.lucene90.compressing

Class Lucene90CompressingStoredFieldsWriter

    • Field Detail

      • FIELDS_EXTENSION

        public static final java.lang.String FIELDS_EXTENSION
        Extension of stored fields file
        See Also:
        Constant Field Values

      • INDEX_EXTENSION

        public static final java.lang.String INDEX_EXTENSION
        Extension of stored fields index
        See Also:
        Constant Field Values

      • META_EXTENSION

        public static final java.lang.String META_EXTENSION
        Extension of stored fields meta
        See Also:
        Constant Field Values

      • INDEX_CODEC_NAME

        public static final java.lang.String INDEX_CODEC_NAME
        Codec name for the index.
        See Also:
        Constant Field Values

      • TYPE_BITS

        static final int TYPE_BITS

      • TYPE_MASK

        static final int TYPE_MASK

      • segment

        private final java.lang.String segment

      • chunkSize

        private final int chunkSize

      • maxDocsPerChunk

        private final int maxDocsPerChunk

      • numStoredFields

        private int[] numStoredFields

      • endOffsets

        private int[] endOffsets

      • docBase

        private int docBase

      • numBufferedDocs

        private int numBufferedDocs

      • numChunks

        private long numChunks

      • numDirtyChunks

        private long numDirtyChunks

      • numDirtyDocs

        private long numDirtyDocs

      • numStoredFieldsInDoc

        private int numStoredFieldsInDoc

      • NEGATIVE_ZERO_FLOAT

        static final int NEGATIVE_ZERO_FLOAT

      • NEGATIVE_ZERO_DOUBLE

        static final long NEGATIVE_ZERO_DOUBLE

      • BULK_MERGE_ENABLED_SYSPROP

        static final java.lang.String BULK_MERGE_ENABLED_SYSPROP

      • BULK_MERGE_ENABLED

        static final boolean BULK_MERGE_ENABLED

    • Constructor Detail

      • Lucene90CompressingStoredFieldsWriter

        Lucene90CompressingStoredFieldsWriter​(Directory directory,
                                      SegmentInfo si,
                                      java.lang.String segmentSuffix,
                                      IOContext context,
                                      java.lang.String formatName,
                                      CompressionMode compressionMode,
                                      int chunkSize,
                                      int maxDocsPerChunk,
                                      int blockShift)
                               throws java.io.IOException
        Sole constructor.
        Throws:
        java.io.IOException

    • Method Detail

      • close

        public void close()
           throws java.io.IOException
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Specified by:
        close in class StoredFieldsWriter
        Throws:
        java.io.IOException

      • finishDocument

        public void finishDocument()
                    throws java.io.IOException
        Description copied from class: StoredFieldsWriter
        Called when a document and all its fields have been added.
        Overrides:
        finishDocument in class StoredFieldsWriter
        Throws:
        java.io.IOException

      • saveInts

        private static void saveInts​(int[] values,
                             int length,
                             DataOutput out)
                      throws java.io.IOException
        Throws:
        java.io.IOException

      • writeHeader

        private void writeHeader​(int docBase,
                         int numBufferedDocs,
                         int[] numStoredFields,
                         int[] lengths,
                         boolean sliced,
                         boolean dirtyChunk)
                  throws java.io.IOException
        Throws:
        java.io.IOException

      • triggerFlush

        private boolean triggerFlush()

      • flush

        private void flush​(boolean force)
            throws java.io.IOException
        Throws:
        java.io.IOException

      • writeZFloat

        static void writeZFloat​(DataOutput out,
                        float f)
                 throws java.io.IOException
        Writes a float in a variable-length format. Writes between one and five bytes. Small integral values typically take fewer bytes.

        ZFloat --> Header, Bytes*?

        • Header --> Uint8. When it is equal to 0xFF then the value is negative and stored in the next 4 bytes. Otherwise if the first bit is set then the other bits in the header encode the value plus one and no other bytes are read. Otherwise, the value is a positive float value whose first byte is the header, and 3 bytes need to be read to complete it.
        • Bytes --> Potential additional bytes to read depending on the header.
        Throws:
        java.io.IOException

      • writeZDouble

        static void writeZDouble​(DataOutput out,
                         double d)
                  throws java.io.IOException
        Writes a float in a variable-length format. Writes between one and five bytes. Small integral values typically take fewer bytes.

        ZFloat --> Header, Bytes*?

        • Header --> Uint8. When it is equal to 0xFF then the value is negative and stored in the next 8 bytes. When it is equal to 0xFE then the value is stored as a float in the next 4 bytes. Otherwise if the first bit is set then the other bits in the header encode the value plus one and no other bytes are read. Otherwise, the value is a positive float value whose first byte is the header, and 7 bytes need to be read to complete it.
        • Bytes --> Potential additional bytes to read depending on the header.
        Throws:
        java.io.IOException

      • writeTLong

        static void writeTLong​(DataOutput out,
                       long l)
                throws java.io.IOException
        Writes a long in a variable-length format. Writes between one and ten bytes. Small values or values representing timestamps with day, hour or second precision typically require fewer bytes.

        ZLong --> Header, Bytes*?

        • Header --> The first two bits indicate the compression scheme:
          • 00 - uncompressed
          • 01 - multiple of 1000 (second)
          • 10 - multiple of 3600000 (hour)
          • 11 - multiple of 86400000 (day)
          Then the next bit is a continuation bit, indicating whether more bytes need to be read, and the last 5 bits are the lower bits of the encoded value. In order to reconstruct the value, you need to combine the 5 lower bits of the header with a vLong in the next bytes (if the continuation bit is set to 1). Then zigzag-decode it and finally multiply by the multiple corresponding to the compression scheme.
        • Bytes --> Potential additional bytes to read depending on the header.
        Throws:
        java.io.IOException

      • finish

        public void finish​(int numDocs)
            throws java.io.IOException
        Description copied from class: StoredFieldsWriter
        Called before StoredFieldsWriter.close(), passing in the number of documents that were written. Note that this is intentionally redundant (equivalent to the number of calls to StoredFieldsWriter.startDocument(), but a Codec should check that this is the case to detect the JRE bug described in LUCENE-1282.
        Specified by:
        finish in class StoredFieldsWriter
        Throws:
        java.io.IOException

      • tooDirty

        boolean tooDirty​(Lucene90CompressingStoredFieldsReader candidate)
        Returns true if we should recompress this reader, even though we could bulk merge compressed data

        The last chunk written for a segment is typically incomplete, so without recompressing, in some worst-case situations (e.g. frequent reopen with tiny flushes), over time the compression ratio can degrade. This is a safety switch.

      • ramBytesUsed

        public long ramBytesUsed()
        Description copied from interface: Accountable
        Return the memory usage of this object in bytes. Negative values are illegal.