如何使用wreq只下载部分响应?

时间:2016-01-10 14:27:52

标签: http haskell request streaming

如何使用wreq库仅下载响应的前1 KiB?

理论上我可以使用Range HTTP标头。但是,这需要服务器上的支持,我不能依赖它。

如果客户端上有足够的数据,我怎样才能中止请求?

1 个答案:

答案 0 :(得分:1)

一种选择是使用foldGet函数,在数据到达时累积数据块,并在下载足够数据后抛出异常。累积的数据将在异常中传递出去。

import qualified Control.Exception           as E
import qualified Data.ByteString             as BS
import qualified Data.ByteString.Lazy        as BL
import           Data.ByteString.Builder

data EnoughException = Enough Builder

instance Show EnoughException where
    show _ = "EnoughException"

instance E.Exception EnoughException

getFile :: Int -> String -> IO BL.ByteString
getFile maxLen url = ((extract . fst) <$> foldGet process (mempty, 0) url) `E.catch` enough
  where
    extract :: Builder -> String
    extract = BL.take (fromIntegral maxLen) . toLazyByteString

    process :: (Builder, Int)   -- ^(Accumulated data, current length)
            -> BS.ByteString    -- ^Downloded chunk
            -> IO (Builder, Int)
    process (acc, cur) now = do
        let acc' = acc <> byteString now
        let cur' = cur + BS.length now
        if cur' > maxLen
            then E.throw (Enough acc')
            else return (acc', cur')

    enough (Enough d) = return (extract d)

(注意:此函数不处理 wreq 可能抛出的异常。对于实际使用而言是必要的。)