Pre-allocating drive space for file storage

Question

Is there a Java way to pre-allocate drive space for exclusive usage in the application?

There is no requirement for this space to be a separate filesystem or a part of

Answer 1

On Linux systems you can use fallocate() system call. It's extremely fast. Just run Bash command.

UPD:

fallocate -l 10G 10Gigfile

Answer 2

You could try using a RandomAccessFile object and use the setLength() method.

Example:

File file = ... //Create a temporary file on the filesystem your trying to reserve.
long bytes = ... //number of bytes you want to reserve.

RandomAccessFile rf = null;
try{
    rf = new RandomAccessFile(file, "rw"); //rw stands for open in read/write mode.
    rf.setLength(bytes); //This will cause java to "reserve" memory for your application by inflating/truncating the file to the specific size.

    //Do whatever you want with the space here...
}catch(IOException ex){
    //Handle this...
}finally{
    if(rf != null){
        try{
            rf.close(); //Lets be nice and tidy here.
        }catch(IOException ioex){
            //Handle this if you want...
        }
    }
}

Note: The file must exist before you create the RandomAccessFile object.

The RandomAccessFile object can then be used to read/write to the file. Make sure the target filesystem has enough free space. The space may not be "exclusive" per-say but you can always use File Locks to do that.

P.S: If you end up realizing hard drives are slow and useless (or meant to use RAM from the start) you can use the ByteBuffer object from java.nio. The allocate() and allocateDirect() methods should be more than enough. The byte buffer will be allocated into RAM (and possible SwapFile) and will be exclusive to this java program. Random access can be done by changing the position of the buffer. Since these buffers use signed integers to reference position, max sizes are limited to 2^31 - 1.

Read more on RandomAccessFile here.

Read more on FileLock (the java object) here.

Read more on ByteBuffer here.

Answer 3

You can pre-allocate space by writing a large file, but to be honest I wouldn't bother. Performance will be pretty good/ probably better than you need.

If you really needed performance, you'd be writing C++/C# and doing RAW I/O.

But that's typically only done when writing an RDBMS engine, high-volume media capture or similar.

Answer 4

Here's a stripped down version of my JNA-based fallocate solution. The main trick is obtaining the native file descriptor. I've only tested it on Linux so far, but it should work on all modern POSIX/non-Windows systems. It's not necessary on Windows, as Windows does not create sparse files by default (only with StandardOpenOption.SPARSE), so RandomAccessFile.setLength(size) or FileChannel.write(ByteBuffer.allocate(1), size - 1) are adequate there.

/**
 * Provides access to operating system-specific {@code fallocate} and
 * {@code posix_fallocate} functions.
 */
public final class Fallocate {

    private static final boolean IS_LINUX = Platform.isLinux();
    private static final boolean IS_POSIX = !Platform.isWindows();

    private static final int FALLOC_FL_KEEP_SIZE = 0x01;

    private final int fd;
    private int mode;
    private long offset;
    private final long length;

    private Fallocate(int fd, long length) {
        if (!isSupported()) {
            throwUnsupported("fallocate");
        }
        this.fd = fd;
        this.length = length;
    }

    public static boolean isSupported() {
        return IS_POSIX;
    }

    public static Fallocate forChannel(FileChannel channel, long length) {
        return new Fallocate(getDescriptor(channel), length);
    }

    public static Fallocate forDescriptor(FileDescriptor descriptor, long length) {
        return new Fallocate(getDescriptor(descriptor), length);
    }

    public Fallocate fromOffset(long offset) {
        this.offset = offset;
        return this;
    }

    public Fallocate keepSize() {
        requireLinux("fallocate keep size");
        mode |= FALLOC_FL_KEEP_SIZE;
        return this;
    }

    private void requireLinux(String feature) {
        if (!IS_LINUX) {
            throwUnsupported(feature);
        }
    }

    private void throwUnsupported(String feature) {
        throw new UnsupportedOperationException(feature +
                " is not supported on this operating system");
    }

    public void execute() throws IOException {
        final int errno;
        if (IS_LINUX) {
            final int result = FallocateHolder.fallocate(fd, mode, offset, length);
            errno = result == 0 ? 0 : Native.getLastError();
        } else {
            errno = PosixFallocateHolder.posix_fallocate(fd, offset, length);
        }
        if (errno != 0) {
            throw new IOException("fallocate returned " + errno);
        }
    }

    private static class FallocateHolder {

        static {
            Native.register(Platform.C_LIBRARY_NAME);
        }

        private static native int fallocate(int fd, int mode, long offset, long length);
    }

    private static class PosixFallocateHolder {

        static {
            Native.register(Platform.C_LIBRARY_NAME);
        }

        private static native int posix_fallocate(int fd, long offset, long length);
    }

    private static int getDescriptor(FileChannel channel) {
        try {
            // sun.nio.ch.FileChannelImpl declares private final java.io.FileDescriptor fd
            final Field field = channel.getClass().getDeclaredField("fd");
            field.setAccessible(true);
            return getDescriptor((FileDescriptor) field.get(channel));
        } catch (final Exception e) {
            throw new UnsupportedOperationException("unsupported FileChannel implementation", e);
        }
    }

    private static int getDescriptor(FileDescriptor descriptor) {
        try {
            // Oracle java.io.FileDescriptor declares private int fd
            final Field field = descriptor.getClass().getDeclaredField("fd");
            field.setAccessible(true);
            return (int) field.get(descriptor);
        } catch (final Exception e) {
            throw new UnsupportedOperationException("unsupported FileDescriptor implementation", e);
        }
    }
}