golang http timeout and goroutines accumulation

随声附和 提交于 2019-12-30 05:23:04

问题


I use goroutines achieve http.Get timeout, and then I found that the number has been rising steadily goroutines, and when it reaches 1000 or so, the program will exit

Code:

package main

import (
        "errors"
        "io/ioutil"
        "log"
        "net"
        "net/http"
        "runtime"
        "time"
)

// timeout dialler
func timeoutDialler(timeout time.Duration) func(network, addr string) (net.Conn, error) {
        return func(network, addr string) (net.Conn, error) {
                return net.DialTimeout(network, addr, timeout)
        }
}

func timeoutHttpGet(url string) ([]byte, error) {
        // change dialler add timeout support && disable keep-alive
        tr := &http.Transport{
                Dial:              timeoutDialler(3 * time.Second),
                DisableKeepAlives: true,
        }

        client := &http.Client{Transport: tr}

        type Response struct {
                resp []byte
                err  error
        }

        ch := make(chan Response, 0)
        defer func() {
                close(ch)
                ch = nil
        }()

        go func() {
                resp, err := client.Get(url)
                if err != nil {
                        ch <- Response{[]byte{}, err}
                        return
                }
                defer resp.Body.Close()

                body, err := ioutil.ReadAll(resp.Body)
                if err != nil {
                        ch <- Response{[]byte{}, err}
                        return
                }

                tr.CloseIdleConnections()
                ch <- Response{body, err}
        }()

        select {
        case <-time.After(5 * time.Second):
                return []byte{}, errors.New("timeout")
        case response := <-ch:
                return response.resp, response.err
        }
}

func handler(w http.ResponseWriter, r *http.Request) {
        _, err := timeoutHttpGet("http://google.com")
        if err != nil {
                log.Println(err)
                return
        }
}

func main() {
        go func() {
                for {
                        log.Println(runtime.NumGoroutine())
                        time.Sleep(500 * time.Millisecond)
                }
        }()

        s := &http.Server{
                Addr:         ":8888",
                ReadTimeout:  15 * time.Second,
                WriteTimeout: 15 * time.Second,
        }

        http.HandleFunc("/", handler)
        log.Fatal(s.ListenAndServe())
}

http://play.golang.org/p/SzGTMMmZkI


回答1:


Init your chan with 1 instead of 0:

ch := make(chan Response, 1)

And remove the defer block that closes and nils ch.

See: http://blog.golang.org/go-concurrency-patterns-timing-out-and

Here is what I think is happening:

  1. after the 5s timeout, timeoutHttpGet returns
  2. the defer statement runs, closing ch and then setting it to nil
  3. the go routine it started to do the actual fetch finishes and attempts to send its data to ch
  4. but ch is nil, and so won't receive anything, preventing that statement from finishing, and thus preventing the go routine from finishing

I assume you are setting ch = nil because before you had that, you would get run-time panics because that's what happens when you attempt to write to a closed channel, as described by the spec.

Giving ch a buffer of 1 means that the fetch go routine can send to it without needing a receiver. If the handler has returned due to timeout, everything will just get garbage collected later on.



来源:https://stackoverflow.com/questions/20990332/golang-http-timeout-and-goroutines-accumulation

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!