What's the conceptual difference between Machines and Conduits (or other similar libraries)?

Question

I'd like to learn the concept, so that I'd be able to understand and use libraries such as machines.

I tried to follow Rúnar Bjarnason's talk on machines, but there is too little information, basically just a bunch of data types. I can't even understand what k is in

newtype Machine k o = Step k o (Machine k o)
data Step k o r = Stop
                | Yield o r
                | forall t . Await (t -> r) (k t) r

or what's t is and why it's quantified. Or, what's the conceptual difference between conduit-like libraries and machines?

Answer 1

conduit and pipes are both far more mature than machines, but -- that said -- machines is trying to take a different path than conduit and pipes.

With machines, I'm trying for a relatively simple API in terms of type arguments. Both conduit and pipes have chosen to unify all of their concepts by using 5-6 different type variable arguments.

Machines takes a different approach of parameterizing a machine (or Plan) on its "input language", which puts all the onus on a single extra argument (or two in the case of a Plan). Also by choosing to parameterize the input language in this way it opens up possibilities of using machines that can accept input (non-)deterministically from multiple input sources. The result is basically just a free monad with an extra 'emit' instruction!

In exchange for a slightly more rigorous policy about how you build and use machines, it can eventually offer better safety about asymptotics of the resulting code.

That said, pipes and conduit have had a lot of real world use and machines is more or less a playground for me, Rúnar Bjarnason and Paul Chiusano.

It is currently suitable for working on input that you intend to consume entirely, but less so for working with complicated resources or for parsing than what you get with those other two APIs.

Now, about that quantifier!

t there is actually existentially quantified. By doing so we can make the Monad for machines not care about the functoriality of the k parameter. This is important because of the way Source is implemented. If I didn't need Source to work, then we could use the simpler

data Step k o r = Stop
                | Yield o r
                | Await (k r) r

This would have the unfortunate side-effect that when you went to compose a Machine with a Source, that the compiler wouldn't know what Functor instance to pick and you'd be swimming in unnecessary type annotations.

The existential quantification there is a trick I picked up when working on the kan-extensions package. It is a generalization of one of the Yoneda types from there.