Can Go spawn and communicate with external processes without starting one OS-thread per external process?

Question

Short version:

Is it possible in Golang to spawn a number of external processes (shell commands) in parallel, such that it does not sta

Answer 1

I find that if we not wait processes, the Go runtime will not start 2500 operating system threads. so please use cmd.Start() other than cmd.Output().

But seems it is impossible to read the process's stdout without consuming a OS thread by golang os package. I think it is because os package not use non-block io to read the pipe.

The bottom, following program runs well on my Linux, although it block the process's stdout as @JimB said in comment, maybe it is because we have small output and it fit the system buffers.

func main() {
    concurrentProcessCount := 50
    wtChan := make(chan *result, concurrentProcessCount)
    for i := 0; i < concurrentProcessCount; i++ {
        go func(i int) {
            fmt.Println("Starting process ", i, "...")
            cmd := exec.Command("bash", "-c", "for i in 1 2 3 4 5; do echo to sleep $i seconds;sleep $i;echo done;done;")
            outPipe,_ := cmd.StdoutPipe()
            err := cmd.Start()
            if err != nil {
                panic(err)
            }
            <-time.Tick(time.Second)
            fmt.Println("Finishing process ", i, "...")
            wtChan <- &result{cmd.Process, outPipe}
        }(i)
    }

    fmt.Println("root:",os.Getpid());

    waitDone := 0
    forLoop:
    for{
        select{
        case r:=<-wtChan:
            r.p.Wait()
            waitDone++
            output := &bytes.Buffer{}
            io.Copy(output, r.b)
            fmt.Println(waitDone, output.String())
            if waitDone == concurrentProcessCount{
                break forLoop
            }
        }
    }
}