conduit

Haskell streaming download

我是研究僧i 提交于 2019-12-10 18:11:57
问题 The two resources I found that suggested recipes for streaming downloads using popular Haskell libraries were: https://haskell-lang.org/library/http-client#Streaming http://www.alfredodinapoli.com/posts/2013-07-20-slick-http-download-in-haskell.html How would I modify the code in the former to (a) save to file, and (b) print only a (take 5) of the byte response, rather than the whole response to stdout? My attempt at (b) is: #!/usr/bin/env stack {- stack --install-ghc --resolver lts-5.13

Is there any difference between “MonadIO m” and “MonadBaseControl IO m”?

让人想犯罪 __ 提交于 2019-12-10 02:17:17
问题 Function runTCPClient from network-conduit has the following signature: runTCPClient :: (MonadIO m, MonadBaseControl IO m) => ClientSettings m -> Application m -> m () MonadIO m provides liftIO :: IO a -> m a and MonadBaseControl IO m provides liftBase :: IO a -> m a There is no visible difference. Do they provide the same functionality? If yes, why the duplication in the type signature? If not, what's the difference? 回答1: liftBase is part of MonadBase which is a generalization of MonadIO for

Why does this cause a memory leak in the Haskell Conduit library?

一笑奈何 提交于 2019-12-10 01:42:43
问题 I have a conduit pipeline processing a long file. I want to print a progress report for the user every 1000 records, so I've written this: -- | Every n records, perform the IO action. -- Used for progress reports to the user. progress :: (MonadIO m) => Int -> (Int -> i -> IO ()) -> Conduit i m i progress n act = skipN n 1 where skipN c t = do mv <- await case mv of Nothing -> return () Just v -> if c <= 1 then do liftIO $ act t v yield v skipN n (succ t) else do yield v skipN (pred c) (succ t

How do you save a file using Conduit?

落花浮王杯 提交于 2019-12-07 23:01:48
问题 How do you save a file using conduit's library? I looked through conduit's tutorial but can't seem to find anything, here is my use case: main :: IO () main = do xxs <- lines <$> (readFile filePath) sourceList xxs =$ pipe $$ saveFile pipe :: Monad m => Conduit String m String pipe = undefined So there's two question here: Does it make sense to use lines to convert a string into a list of strings and then feeding it to sourceList ? How should I implement the saveFile function so that when the

Http-Conduit frequent connection failures

白昼怎懂夜的黑 提交于 2019-12-06 20:07:45
问题 I am writing application which will download some files by HTTP. Up to some point I was using following code snippet to download page body: import network.HTTP simpleHTTP (getRequest "http://www.haskell.org/") >>= getResponseBody It was working fine but it could not establish connection by HTTPS protocol. So to fix this I have switched to HTTP-Conduit and now I am using following code: simpleHttp' :: Manager -> String -> IO (C.Response LBS.ByteString) simpleHttp' manager url = do request <-

How do you save a file using Conduit?

拥有回忆 提交于 2019-12-06 12:05:50
How do you save a file using conduit's library? I looked through conduit's tutorial but can't seem to find anything, here is my use case: main :: IO () main = do xxs <- lines <$> (readFile filePath) sourceList xxs =$ pipe $$ saveFile pipe :: Monad m => Conduit String m String pipe = undefined So there's two question here: Does it make sense to use lines to convert a string into a list of strings and then feeding it to sourceList ? How should I implement the saveFile function so that when the strings xxs are fully processed, I can write it to disk? A small minimal example of what you are trying

connecting http-conduit to xml-conduit

喜欢而已 提交于 2019-12-05 16:49:19
I'm struggling converting a Response from http-conduit to an XML document via xml-conduit. The doPost function takes an XML Document and posts it to the server. The server responds with an XML Document. doPost queryDoc = do runResourceT $ do manager <- liftIO $ newManager def req <- liftIO $ parseUrl hostname let req2 = req { method = H.methodPost , requestHeaders = [(CI.mk $ fromString "Content-Type", fromString "text/xml" :: Ascii) :: Header] , redirectCount = 0 , checkStatus = \_ _ -> Nothing , requestBody = RequestBodyLBS $ (renderLBS def queryDoc) } res <- http req2 manager return $ res

Is there any difference between “MonadIO m” and “MonadBaseControl IO m”?

给你一囗甜甜゛ 提交于 2019-12-05 01:58:44
Function runTCPClient from network-conduit has the following signature: runTCPClient :: (MonadIO m, MonadBaseControl IO m) => ClientSettings m -> Application m -> m () MonadIO m provides liftIO :: IO a -> m a and MonadBaseControl IO m provides liftBase :: IO a -> m a There is no visible difference. Do they provide the same functionality? If yes, why the duplication in the type signature? If not, what's the difference? liftBase is part of MonadBase which is a generalization of MonadIO for any base monad and, as you said, MonadBase IO provides the same functionality as MonadIO . However,

Http-Conduit frequent connection failures

二次信任 提交于 2019-12-05 01:48:36
I am writing application which will download some files by HTTP. Up to some point I was using following code snippet to download page body: import network.HTTP simpleHTTP (getRequest "http://www.haskell.org/") >>= getResponseBody It was working fine but it could not establish connection by HTTPS protocol. So to fix this I have switched to HTTP-Conduit and now I am using following code: simpleHttp' :: Manager -> String -> IO (C.Response LBS.ByteString) simpleHttp' manager url = do request <- parseUrl url runResourceT $ httpLbs request manager It can connect to HTTPS but new frustrating problem

Conduit: Multiple Stream Consumers

故事扮演 提交于 2019-12-04 06:14:16
I write a program which counts the frequencies of NGrams in a corpus. I already have a function that consumes a stream of tokens and produces NGrams of one single order: ngram :: Monad m => Int -> Conduit t m [t] trigrams = ngram 3 countFreq :: (Ord t, Monad m) => Consumer [t] m (Map [t] Int) At the moment i just can connect one stream consumer to a stream source: tokens --- trigrams --- countFreq How do I connect multiple stream consumers to the same stream source? I would like to have something like this: .--- unigrams --- countFreq |--- bigrams --- countFreq tokens ----|--- trigrams ---