Question

我无法理解此代码中发生的错误：

import Network.URI.Encode as EN

downloadFile :: (MonadHttp m) => String -> m ()
downloadFile url = do

    traceShowM (T.pack url)

    -- Pull them into memory
    contents <- req GET (https (T.pack $ EN.encode url)) NoReqBody bsResponse mempty

    -- debug read file wrote file etc

    -- Get the filename itself
    let fileName = head $ reverse $ T.splitOn "/" (T.pack url)

这是主要功能（eatChunks）的输出：


*Main UsCensusDataLodesScraper> eatChunks
"https://lehd.ces.census.gov/data/lodes/LODES7/ak/od/ak_od_aux_JT00_2003.csv.gz"
"https://lehd.ces.census.gov/data/lodes/LODES7/ak/od/ak_od_aux_JT00_2002.csv.gz"
"https://lehd.ces.census.gov/data/lodes/LODES7/ak/od/ak_od_aux_JT00_2004.csv.gz"
"https://lehd.ces.census.gov/data/lodes/LODES7/ak/od/ak_od_aux_JT00_2006.csv.gz"
"https://lehd.ces.census.gov/data/lodes/LODES7/ak/od/ak_od_aux_JT00_2008.csv.gz"
"https://lehd.ces.census.gov/data/lodes/LODES7/ak/od/ak_od_aux_JT00_2009.csv.gz"
"https://lehd.ces.census.gov/data/lodes/LODES7/ak/od/ak_od_aux_JT00_2007.csv.gz"
"https://lehd.ces.census.gov/data/lodes/LODES7/ak/od/ak_od_aux_JT00_2010.csv.gz"
"https://lehd.ces.census.gov/data/lodes/LODES7/ak/od/ak_od_aux_JT00_2011.csv.gz"
*** Exception: VanillaHttpException (HttpExceptionRequest Request {
  host                 = "https%253A%252F%252Flehd.ces.census.gov%252Fdata%252Flodes%252FLODES7%252Fak%252Fod%252Fak_od_aux_JT00_2002.csv.gz"
  port                 = 443
  secure               = True
  requestHeaders       = []
  path                 = ""
  queryString          = ""
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}
 (ConnectionFailure Network.Socket.getAddrInfo (called with preferred socket type/protocol: AddrInfo {addrFlags = [AI_ADDRCONFIG], addrFamily = AF_UNSPEC, addrSocketType = Stream, addrProtocol = 6, addrAddress = <assumed to be undefined>, addrCanonName = <assumed to be undefined>}, host name: Just "https%253A%252F%252Flehd.ces.census.gov%252Fdata%252Flodes%252FLODES7%252Fak%252Fod%252Fak_od_aux_JT00_2002.csv.gz", service name: Just "443"): does not exist (nodename nor servname provided, or not known)))

Answer 1

您没有正确构建Url。来自

的文档

http://hackage.haskell.org/package/req-2.0.1/docs/Network-HTTP-Req.html#g:5

我们看到可以通过几种方式构造Url。

亲手：

https "lehd.ces.census.gov"
      /~ "data"
      /~ "lodes"
      /~ "LODES7"
      /~ "ak"
      /~ "od"
      /~ "ak_od_aux_JT00_2003.csv.gz"

通过解析：

-- Don't actually use fromJust, this is just an example
fst . fromJust $ parseUrlHttps "https://lehd.ces.census.gov/data/lodes/LODES7/ak/od/ak_od_aux_JT00_2003.csv.gz"

尤其要注意https的实际作用：给定主机名，它会使用HTTPS方案，该主机名和空路径构造一个Url。 https someUrlString将像构造一个Url

https://<the fully url-encoded version of someUrlString>

这不是您想要的。

Answer 2

您几乎可以肯定不想打EN.encode。可能只是删除它。

Haskell的req Web抓取库出现奇怪的字符编码问题

2 个答案: