在haskell-pipes中分流流媒体流

时间:2015-11-01 16:42:05

标签: haskell haskell-pipes

我无法通过带有haskell管道的管道引导流量。基本上,我分析了一堆文件然后我必须

  1. 以人性化的方式将结果打印到终端
  2. 将结果编码为JSON
  3. 所选路径取决于命令行选项 在第二种情况下,我必须输出一个开括号,然后输入每个传入的值后跟一个逗号,然后是一个右括号。目前insertCommas永远不会终止,因此不会输出结束括号。

    import Pipes
    import Data.ByteString.Lazy as B
    import Data.Aeson (encode)
    
    insertCommas :: Consumer B.ByteString IO ()
    insertCommas = do
        first <- await
        lift $ B.putStr first
        for cat $ \obj -> lift $ do
            putStr ","
            B.putStr obj
    
    jsonExporter :: Consumer (FilePath, AnalysisResult) IO ()
    jsonExporter = do
        lift $ putStr "["
        P.map encode >-> insertCommas
        lift $ putStr "]"
    
    exportStream :: Config -> Consumer (FilePath, AnalysisResult) IO ()
    exportStream conf =
        case outputMode conf of
          JSON -> jsonExporter
          _    -> P.map (export conf) >-> P.stdoutLn
    
    main :: IO ()
    main = do
        -- The first two lines are Docopt stuff, not relevant
        args <- parseArgsOrExit patterns =<< getArgs
        ins  <- allFiles $ args `getAllArgs` argument "paths"
        let conf = readConfig args
        runEffect $ each ins
                 >-> P.mapM analyze
                 >-> P.map (filterResults conf)
                 >-> P.filter filterNulls
                 >-> exportStream conf
    

2 个答案:

答案 0 :(得分:3)

AFAIK消费者无法检测到流的结束。为此,您需要使用Pipes.Parser并反转控件。

这是一个在String元素之间插入逗号的Parser:

import Pipes
import qualified Pipes.Prelude as P
import Pipes.Parse (draw, evalStateT)

commify = do
  lift $ putStrLn "["
  m1 <- draw
  case m1 of
    Nothing -> lift $ putStrLn "]"
    Just x1 -> do
      lift $ putStrLn x1
      let loop = do mx <- draw
                    case mx of
                      Nothing -> lift $ putStrLn "]"
                      Just x  -> lift (putStr "," >> putStrLn x) >> loop
      loop

test1 = evalStateT commify ( mapM_ yield (words "this is a test") )
test2 = evalStateT commify P.stdinLn

为了处理不同的输出格式,我可能会将这两种格式都变成Parser:

exportParser = do
  mx <- draw
  case mx of
    Nothing -> return ()
    Just x  -> (lift $ putStrLn $ export x) >> exportParser

然后:

let parser = case outputMode of
               JSON -> commify
               _    -> exportParser
evalStateT parser (P.mapM analyze
                      >-> P.map (filterResults conf)
                      >-> P.filter filterNulls)

根据exportParser,可能有一种更为流畅的方式来编写foldAllM。您还可以使用MaybeT转换器更简洁地编写commify解析器。我明确地写了两篇文章,以便更容易理解。

答案 1 :(得分:1)

我认为你应该&#39;通知&#39;与管道组。它有intercalates,但不是散布,但写起来并不是什么大不了的事。对于这类问题,我认为你应该远离消费者端。

{-#LANGUAGE OverloadedStrings #-}
import Pipes
import qualified Pipes.Prelude as P
import qualified Data.ByteString.Lazy.Char8 as B
import Pipes.Group
import Lens.Simple  -- or Control.Lens or Lens.Micro or anything with view/^.
import System.Environment

intersperse_ :: Monad m => a -> Producer a m r -> Producer a m r
intersperse_ a producer = intercalates (yield a) (producer ^. chunksOf 1) 

main = do 
  args <- getArgs
  let op prod = case args of 
        "json":_ -> yield "[" *> intersperse_ "," prod <* yield "]"
        _        -> intersperse_ " " prod
  runEffect $ op producer >-> P.mapM_ B.putStr
  putStrLn ""
  where 
    producer = mapM_ yield (B.words "this is a test")

给我这个

    >>> :main json
    [this,is,a,test]
    >>> :main ---
    this is a test