Why does it not create many threads when many goroutines are blocked in writing file in golang?

后端 未结 3 629
盖世英雄少女心
盖世英雄少女心 2020-12-05 16:01

As we know in go, a thread may be created when the goroutine has to perform a blocking call, such as a system call, or a call to a C library via cgo. Some test code:

相关标签:
3条回答
  • 2020-12-05 16:41

    A goroutine is a lightweight thread, it is not equivalent to an operating system thread. The Language specification specifies it as an "independent concurrent thread of control within the same address space".

    Quoting from the documentation of package runtime:

    The GOMAXPROCS variable limits the number of operating system threads that can execute user-level Go code simultaneously. There is no limit to the number of threads that can be blocked in system calls on behalf of Go code; those do not count against the GOMAXPROCS limit.

    Just because you start 200 goroutines, it doesn't mean 200 threads will be started for them. You set GOMAXPROCS to 2 which means there can be 2 "active" goroutines running at the same time. New threads may be spawed if a goroutine gets blocked (e.g. I/O wait). You didn't mention how big your test file is, goroutines you start might finish writing it too quickly.

    The Effective Go blog article defines them as:

    They're called goroutines because the existing terms—threads, coroutines, processes, and so on—convey inaccurate connotations. A goroutine has a simple model: it is a function executing concurrently with other goroutines in the same address space. It is lightweight, costing little more than the allocation of stack space. And the stacks start small, so they are cheap, and grow by allocating (and freeing) heap storage as required.

    Goroutines are multiplexed onto multiple OS threads so if one should block, such as while waiting for I/O, others continue to run. Their design hides many of the complexities of thread creation and management.

    0 讨论(0)
  • 2020-12-05 16:48

    The issue 4056 discusses how to limit the number of actual threads (not goroutine) created.

    Go 1.2 introduced that thread limit management in commit 665feee.

    You can see a test to check if the number of thread created is actually reached or not in pkg/runtime/crash_test.go#L128-L134:

    func TestThreadExhaustion(t *testing.T) {
        output := executeTest(t, threadExhaustionSource, nil)
        want := "runtime: program exceeds 10-thread limit\nfatal error: thread exhaustion"
        if !strings.HasPrefix(output, want) {
            t.Fatalf("output does not start with %q:\n%s", want, output)
        }
    }
    

    That same file has an example to create an actual thread (for a given goroutine), using runtime.LockOSThread():

    func testInNewThread(name string) {
        c := make(chan bool)
        go func() {
            runtime.LockOSThread()
            test(name)
            c <- true
        }()
        <-c
    }
    
    0 讨论(0)
  • 2020-12-05 16:52

    I altered your program slightly to output a much bigger block

    package main
    
    import (
        "io/ioutil"
        "os"
        "runtime"
        "strconv"
    )
    
    func main() {
        runtime.GOMAXPROCS(2)
        data := make([]byte, 128*1024*1024)
        for i := 0; i < 200; i++ {
            go func(n int) {
                for {
                    err := ioutil.WriteFile("testxxx"+strconv.Itoa(n), []byte(data), os.ModePerm)
                    if err != nil {
                        println(err)
                        break
                    }
                }
            }(i)
        }
        select {}
    }
    

    This then shows >200 threads as you expected

    $ cat /proc/17033/status | grep -i thread
    Threads:    203
    

    So I think the syscalls were exiting too quickly in your original test to show the effect you were expecting.

    0 讨论(0)
提交回复
热议问题