Package org.apache.lucene.misc.store

Class DirectIODirectory

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable
    public class DirectIODirectory
extends FilterDirectory
    A Directory implementation for all Unixes and Windows that uses DIRECT I/O to bypass OS level IO caching during merging. For all other cases (searching, writing) we delegate to the provided Directory instance.

    See Overview for more details.

    WARNING: this code is very new and quite easily could contain horrible bugs.

    This directory passes Solr and Lucene tests on Linux, OS X, and Windows; other systems should work but have not been tested! Use at your own risk.

    @throws UnsupportedOperationException if the operating system, file system or JDK does not support Direct I/O or a sufficient equivalent.

    • Field Detail

      • DEFAULT_MERGE_BUFFER_SIZE

        public static final int DEFAULT_MERGE_BUFFER_SIZE
        Default buffer size before writing to disk (256 KB); larger means less IO load but more RAM and direct buffer storage space consumed during merging.
        See Also:
        Constant Field Values

      • DEFAULT_MIN_BYTES_DIRECT

        public static final long DEFAULT_MIN_BYTES_DIRECT
        Default min expected merge size before direct IO is used (10 MB):
        See Also:
        Constant Field Values

      • blockSize

        private final int blockSize

      • mergeBufferSize

        private final int mergeBufferSize

      • minBytesDirect

        private final long minBytesDirect

      • isOpen

        volatile boolean isOpen

      • ExtendedOpenOption_DIRECT

        static final java.nio.file.OpenOption ExtendedOpenOption_DIRECT
        Reference to com.sun.nio.file.ExtendedOpenOption.DIRECT by reflective class and enum lookup. There are two reasons for using this instead of directly referencing ExtendedOpenOption.DIRECT:
        1. ExtendedOpenOption.DIRECT is OpenJDK's internal proprietary API. This API causes un-suppressible(?) warning to be emitted when compiling with --release flag and value N, where N is smaller than the the version of javac used for compilation. For details, please refer to https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8259039.
        2. It is possible that Lucene is run using JDK that does not support ExtendedOpenOption.DIRECT. In such a case, dynamic lookup allows us to bail out with UnsupportedOperationException with meaningful error message.

        This reference is null, if the JDK does not support direct I/O.

    • Constructor Detail

      • DirectIODirectory

        public DirectIODirectory​(FSDirectory delegate,
                         int mergeBufferSize,
                         long minBytesDirect)
                  throws java.io.IOException
        Create a new DirectIODirectory for the named location.
        Parameters:
        delegate - Directory for non-merges, also used as reference to file system path.
        mergeBufferSize - Size of buffer to use for merging.
        minBytesDirect - Merges, or files to be opened for reading, smaller than this will not use direct IO. See DEFAULT_MIN_BYTES_DIRECT and useDirectIO(java.lang.String, org.apache.lucene.store.IOContext, java.util.OptionalLong).
        Throws:
        java.io.IOException - If there is a low-level I/O error

      • DirectIODirectory

        public DirectIODirectory​(FSDirectory delegate)
                  throws java.io.IOException
        Create a new DirectIODirectory for the named location.
        Parameters:
        delegate - Directory for non-merges, also used as reference to file system path.
        Throws:
        java.io.IOException - If there is a low-level I/O error

    • Method Detail

      • getDirectory

        public java.nio.file.Path getDirectory()
        Returns:
        the underlying file system directory

      • useDirectIO

        protected boolean useDirectIO​(java.lang.String name,
                              IOContext context,
                              java.util.OptionalLong fileLength)
        Determines if direct IO should be used for a file. By default this tests if it is a merge context and if the merge or file length extends the minimum size (see DEFAULT_MIN_BYTES_DIRECT). Subclasses may override method to enforce direct IO for specific file types.
        Parameters:
        name - file name (unused by default implementation)
        context - information about merge size
        fileLength - if available, gives the file length. Will be empty when requesting an IndexOutput.
        Returns:
        true if direct IO should be used; false if input/output should be requested from delegate directory.

      • openInput

        public IndexInput openInput​(java.lang.String name,
                            IOContext context)
                     throws java.io.IOException
        Description copied from class: Directory
        Opens a stream for reading an existing file.

        This method must throw either NoSuchFileException or FileNotFoundException if name points to a non-existing file.

        Overrides:
        openInput in class FilterDirectory
        Parameters:
        name - the name of an existing file.
        Throws:
        java.io.IOException - in case of I/O error

      • createOutput

        public IndexOutput createOutput​(java.lang.String name,
                                IOContext context)
                         throws java.io.IOException
        Description copied from class: Directory
        Creates a new, empty file in the directory and returns an IndexOutput instance for appending data to this file.

        This method must throw FileAlreadyExistsException if the file already exists.

        Overrides:
        createOutput in class FilterDirectory
        Parameters:
        name - the name of the file to create.
        Throws:
        java.io.IOException - in case of I/O error

      • close

        public void close()
           throws java.io.IOException
        Description copied from class: Directory
        Closes the directory.
        Specified by:
        close in interface java.lang.AutoCloseable
        Specified by:
        close in interface java.io.Closeable
        Overrides:
        close in class FilterDirectory
        Throws:
        java.io.IOException

      • getDirectOpenOption

        private static java.nio.file.OpenOption getDirectOpenOption()