What are the tradeoffs of the ways to do work in a clojure core.async go-loop?

蹲街弑〆低调 提交于 2019-12-12 08:55:20

问题


As I write more core.async code, a very common pattern that emerges is a go-loop that alts over a sequence of channels and does some work in response to a message, e.g.:

(go-loop [state {}]
  (let [[value task] (alts! tasks)]
    ...work...
    (recur state))

I don't feel like I understand the tradeoffs of the various ways I can actually do the work though, so I thought I'd try to explore them here.

  • Inline or by calling a function: this blocks the loop from continuing until the work is complete. Since it's in a go block, one wouldn't want to do I/O or locking operations.
  • >! a message to a channel monitored by a worker: if the channel is full, this blocks the loop by parking until the channel has capacity. This allows the thread to do other work and allows back pressure.
  • >!! a message: if the channel is full, this blocks by sleeping the thread running the go loop. This is probably undesirable because go threads are a strictly finite resource.
  • >! a message within another go block: this will succeed nearly immediately unless there are no go threads available. Conversely, if the channel is full and is being consumed slowly, this could starve the system of go threads in short order.
  • >!! a message with a thread block: similar to the go block, but consuming system threads instead of go threads, so the upper bound is probably higher
  • puts! a message: it's unclear what the tradeoffs are
  • call the work function in a future: gives the work to a thread from the clojure agent pool to do, allows the go loop to continue. If the input rate exceeds the output rate, this grows the agent pool queue without bound.

Is this summary correct and comprehensive?


回答1:


If the work to be done is entirely CPU-bound, then I would probably do it inline in the go block, unless it's an operation that may take a long time and I want the go block to continue responding to other messages.

In general, any work which doesn't block, sleep, or do I/O can be safely put in a go block without having a major impact on the throughput of the system.

You can use >! to submit work to a worker or pool of workers. I would almost never use >!! in a go block because it can block one of the finite number of threads allocated to running go blocks.

When you need to do I/O or a potentially long-running computation, use a thread instead of a go. This is very similar to future — it creates a real thread — but it returns a channel like go.

put! is a lower-level operation generally used at the "boundaries" of core.async to connect it to conventional callback-based interfaces. There's rarely any reason to use put! inside a go.

core.async can support fine-grained control over how threads are created. I demonstrated a few possibilities in a blog post, Parallel Processing with core.async.



来源:https://stackoverflow.com/questions/24915034/what-are-the-tradeoffs-of-the-ways-to-do-work-in-a-clojure-core-async-go-loop

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!