Sparse files are huge with io.Copy()

Question

I want to copy files from one place to another and the problem is I deal with a lot of sparse files.

Is there any (easy) way of copying sparse files without becoming hu

Answer 1

You don't need to resort to syscalls.

package main

import "os"

func main() {
    f, _ := os.Create("/tmp/sparse.dat")
    f.Write([]byte("start"))
    f.Seek(1024*1024*10, 0)
    f.Write([]byte("end"))
}

Then you'll see:

$ ls -l /tmp/sparse.dat
-rw-rw-r-- 1 soren soren 10485763 Jun 25 14:29 /tmp/sparse.dat
$ du /tmp/sparse.dat
8   /tmp/sparse.dat

It's true you can't use io.Copy as is. Instead you need to implement an alternative to io.Copy which reads a chunk from the src, checks if it's all '\0'. If it is, just dst.Seek(len(chunk), os.SEEK_CUR) to skip past that part in dst. That particular implementation is left as an exercise to the reader :)