golang speeding up response writing time?

问题

func grabPage(i int, wg *sync.WaitGroup, buf *[]byte) {
    defer wg.Done()
    res, err := http.Get("https://en.wikipedia.org/wiki/Immanuel_Kant")
    if err != nil {
        log.Fatal(err)
    }
    f, err := os.Create(fmt.Sprintf("./data/%d.txt", i))
    if err != nil {
        log.Fatal(err)
    }
    _, err = io.CopyBuffer(f, res.Body, *buf)
    if err != nil {
        log.Fatal(err)
    }
}

func main() {
    f, _ := os.Create("cpuprofile")

    pprof.StartCPUProfile(f)
    defer pprof.StopCPUProfile()
    runtime.GOMAXPROCS(4)
    start := time.Now()
    var wg sync.WaitGroup
    total := 800

    var buf []byte
    wg.Add(total)
    for index := 0; index < total; index++ {
        go grabPage(index, &wg, &buf)
    }
    wg.Wait()
    elapsed := time.Since(start)
    log.Printf("took %s", elapsed)
}

I have a toy program and am simply grabbing an http request and writing it to a file. wget shows a latency of .2s and I am wondering how to get closer to this sort of speed (if possible). I have tried io.copy,fasthttp, bufio stream readers etc. which all were slower than the above. I just want to make sure there is nothing obvious I am missing that could speed this up as I am very new to Go. I liked fasthttp's concept of reading directly to a buffer but I think I mimicked that via io.CopyBuffer. here are my proff results :/

     0     0%     0%      6.78s 68.00%  io.CopyBuffer /usr/local/opt/go/libexec/src/io/io.go
     0     0%     0%      6.78s 68.00%  io.copyBuffer /usr/local/opt/go/libexec/src/io/io.go
     0     0%     0%      5.44s 54.56%  net/http.(*http2gzipReader).Read /usr/local/opt/go/libexec/src/net/http/h2_bundle.go
     0     0%     0%      5.41s 54.26%  compress/gzip.(*Reader).Read /usr/local/opt/go/libexec/src/compress/gzip/gunzip.go
     0     0%     0%      5.35s 53.66%  compress/flate.(*decompressor).Read /usr/local/opt/go/libexec/src/compress/flate/inflate.go
 0.92s  9.23%  9.23%      5.20s 52.16%  compress/flate.(*decompressor).huffmanBlock /usr/local/opt/go/libexec/src/compress/flate/inflate.go
 2.97s 29.79% 39.02%      2.99s 29.99%  syscall.Syscall /usr/local/opt/go/libexec/src/syscall/asm_darwin_amd64.s
 0.01s   0.1% 39.12%      2.59s 25.98%  internal/poll.(*FD).Write /usr/local/opt/go/libexec/src/internal/poll/fd_unix.go
 1.02s 10.23% 49.35%      2.53s 25.38%  compress/flate.(*decompressor).huffSym /usr/local/opt/go/libexec/src/compress/flate/inflate.go

来源：https://stackoverflow.com/questions/46510272/golang-speeding-up-response-writing-time

标签

performance

optimization

get-request