Faster way of copying data in Java?

Question

I have been given a task of copying data from a server. I am using BufferedInputStream and output stream to copy the data and I am doing it byte by byte. Even though it is running but It is taking ages to copy the data as some of them are in 100's MBs, so definitely it is not gonna work. Can anyone suggest me any alternate of Byte by Byte copy so that my code can copy file that are in few Hundred MBs. Buffer is 2048.

Here is how my code look like:

static void copyFiles(SmbFile[] files, String parent) throws IOException {

  SmbFileInputStream input = null;
  FileOutputStream output = null;
  BufferedInputStream buf_input = null;
  try {
    for (SmbFile f : files) {
      System.out.println("Working on files :" + f.getName());
      if (f.isDirectory()) {

        File folderToBeCreated = new File(parent+f.getName());
        if (!folderToBeCreated.exists()) {
          folderToBeCreated.mkdir();
          System.out.println("Folder name " + parent
                + f.getName() + "has been created");
        } else {
          System.out.println("exists");

        }

        copyFiles(f.listFiles(), parent +  f.getName());
      } else {

        input = (SmbFileInputStream) f.getInputStream();

        buf_input = new BufferedInputStream(input, BUFFER);

        File t = new File(parent + f.getName());
        if (!t.exists()) {
          t.createNewFile();
        }
        output = new FileOutputStream(t);

        int c;

        int count;
        byte data[] = new byte[BUFFER];

        while ((count = buf_input.read(data, 0, BUFFER)) != -1) {
          output.write(data, 0, count);
        }
      }
    }
  } catch (IOException e) {
    e.printStackTrace();

  } finally {
    if (input != null) {
      input.close();
    }
    if (output != null) {
      output.close();
    }
  }
}

Answer 1

Here is a link to an excellent post explaining how to use nio channels to make copies of streams. It introduces a helper method ChannelTools.fastChannelCopy that lets you copy streams like this:

final InputStream input = new FileInputStream(inputFile);
final OutputStream output = new FileOutputStream(outputFile);
final ReadableByteChannel inputChannel = Channels.newChannel(input);
final WriteableByteChannel outputChannel = Channels.newChannel(output);
ChannelTools.fastChannelCopy(inputChannel, outputChannel);
inputChannel.close();
outputChannel.close()

Answer 2

Well since you're using a BufferedInputStream, you aren't reading byte by byte, but rather the size of the buffer. You could just try increasing the buffer size.

Answer 3

Reading/writing byte-by-byte is definitely going to be slow, even though the actual reading/writing is done by chunks of the buffer size. One way to speed it up is to read/write by blocks. Have a look at read(byte[] b, int off, int len) method of BufferedInputStream. However it probably won't give you enough of the improvement.

What would be much better is to use nio package (new IO) to copy data using nio channels. Have a look at nio documentation for more info.

Answer 4

Take a look at this link that describes some things you can try to increase performance while working with SmbFileInputStream.

Answer 5

I would suggest to use FileUtils from org.apache.commons.io. It has enough utility methods to perform file operations.

org.apache.commons.io.FileUtils API Here