Convert InputStream to byte array in Java

Question

How do I read an entire InputStream into a byte array?

Answer 1

I use this.

public static byte[] toByteArray(InputStream is) throws IOException {
        ByteArrayOutputStream output = new ByteArrayOutputStream();
        try {
            byte[] b = new byte[4096];
            int n = 0;
            while ((n = is.read(b)) != -1) {
                output.write(b, 0, n);
            }
            return output.toByteArray();
        } finally {
            output.close();
        }
    }

Answer 2

You can use cactoos library with provides reusable object-oriented Java components. OOP is emphasized by this library, so no static methods, NULLs, and so on, only real objects and their contracts (interfaces). A simple operation like reading InputStream, can be performed like that

final InputStream input = ...;
final Bytes bytes = new BytesOf(input);
final byte[] array = bytes.asBytes();
Assert.assertArrayEquals(
    array,
    new byte[]{65, 66, 67}
);

Having a dedicated type Bytes for working with data structure byte[] enables us to use OOP tactics for solving tasks at hand. Something that a procedural "utility" method will forbid us to do. For example, you need to enconde bytes you've read from this InputStream to Base64. In this case you will use Decorator pattern and wrap Bytes object within implementation for Base64. cactoos already provides such implementation:

final Bytes encoded = new BytesBase64(
    new BytesOf(
        new InputStreamOf("XYZ")
    )
);
Assert.assertEquals(new TextOf(encoded).asString(), "WFla");

You can decode them in the same manner, by using Decorator pattern

final Bytes decoded = new Base64Bytes(
    new BytesBase64(
        new BytesOf(
            new InputStreamOf("XYZ")
        )
    )
);
Assert.assertEquals(new TextOf(decoded).asString(), "XYZ");

Whatever your task is you will be able to create own implementation of Bytes to solve it.

Answer 3

Use vanilla Java's DataInputStream and its readFully Method (exists since at least Java 1.4):

...
byte[] bytes = new byte[(int) file.length()];
DataInputStream dis = new DataInputStream(new FileInputStream(file));
dis.readFully(bytes);
...

There are some other flavors of this method, but I use this all the time for this use case.

Answer 4

We are seeing some delay for few AWS transaction, while converting S3 object to ByteArray.

Note: S3 Object is PDF document (max size is 3 mb).

We are using the option #1 (org.apache.commons.io.IOUtils) to convert the S3 object to ByteArray. We have noticed S3 provide the inbuild IOUtils method to convert the S3 object to ByteArray, we are request you to confirm what is the best way to convert the S3 object to ByteArray to avoid the delay.

Option #1:

import org.apache.commons.io.IOUtils;
is = s3object.getObjectContent();
content =IOUtils.toByteArray(is);

Option #2:

import com.amazonaws.util.IOUtils;
is = s3object.getObjectContent();
content =IOUtils.toByteArray(is);

Also let me know if we have any other better way to convert the s3 object to bytearray

Answer 5

Solution in Kotlin (will work in Java too, of course), which includes both cases of when you know the size or not:

    fun InputStream.readBytesWithSize(size: Long): ByteArray? {
        return when {
            size < 0L -> this.readBytes()
            size == 0L -> ByteArray(0)
            size > Int.MAX_VALUE -> null
            else -> {
                val sizeInt = size.toInt()
                val result = ByteArray(sizeInt)
                readBytesIntoByteArray(result, sizeInt)
                result
            }
        }
    }

    fun InputStream.readBytesIntoByteArray(byteArray: ByteArray,bytesToRead:Int=byteArray.size) {
        var offset = 0
        while (true) {
            val read = this.read(byteArray, offset, bytesToRead - offset)
            if (read == -1)
                break
            offset += read
            if (offset >= bytesToRead)
                break
        }
    }

If you know the size, it saves you on having double the memory used compared to the other solutions (in a brief moment, but still could be useful). That's because you have to read the entire stream to the end, and then convert it to a byte array (similar to ArrayList which you convert to just an array).

So, if you are on Android, for example, and you got some Uri to handle, you can try to get the size using this:

    fun getStreamLengthFromUri(context: Context, uri: Uri): Long {
        context.contentResolver.query(uri, arrayOf(MediaStore.MediaColumns.SIZE), null, null, null)?.use {
            if (!it.moveToNext())
                return@use
            val fileSize = it.getLong(it.getColumnIndex(MediaStore.MediaColumns.SIZE))
            if (fileSize > 0)
                return fileSize
        }
        //if you wish, you can also get the file-path from the uri here, and then try to get its size, using this: https://stackoverflow.com/a/61835665/878126
        FileUtilEx.getFilePathFromUri(context, uri, false)?.use {
            val file = it.file
            val fileSize = file.length()
            if (fileSize > 0)
                return fileSize
        }
        context.contentResolver.openInputStream(uri)?.use { inputStream ->
            if (inputStream is FileInputStream)
                return inputStream.channel.size()
            else {
                var bytesCount = 0L
                while (true) {
                    val available = inputStream.available()
                    if (available == 0)
                        break
                    val skip = inputStream.skip(available.toLong())
                    if (skip < 0)
                        break
                    bytesCount += skip
                }
                if (bytesCount > 0L)
                    return bytesCount
            }
        }
        return -1L
    }

Answer 6

In-case someone is still looking for a solution without dependency and If you have a file.

DataInputStream

 byte[] data = new byte[(int) file.length()];
 DataInputStream dis = new DataInputStream(new FileInputStream(file));
 dis.readFully(data);
 dis.close();

ByteArrayOutputStream

 InputStream is = new FileInputStream(file);
 ByteArrayOutputStream buffer = new ByteArrayOutputStream();
 int nRead;
 byte[] data = new byte[(int) file.length()];
 while ((nRead = is.read(data, 0, data.length)) != -1) {
     buffer.write(data, 0, nRead);
 }

RandomAccessFile

 RandomAccessFile raf = new RandomAccessFile(file, "r");
 byte[] data = new byte[(int) raf.length()];
 raf.readFully(data);