Scala中的递归HTTP请求

时间:2018-10-22 18:49:28

标签: scala jackson gitlab gitlab-api

我需要执行递归请求,然后将所有模型收集到一个列表中,但不了解如何执行。请告诉我我在想正确的方法吗?

package kindSir.main

import dispatch.Defaults._
import dispatch._
import kindSir.models._
import org.json4s._
import org.json4s.jackson.JsonMethods._

object ApplicationMain extends App {

  def fetchMergeRequests(startPage: Int = 1): Future[List[MergeRequest]] = {
    val requestsUrl = url(s"https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab-ce/merge_requests?state=opened&per_page=3&page=${startPage}")
    Http(requestsUrl).map { res =>
      (parse(res.getResponseBody), res.getHeader("X-Next-Page").toInt) match {
        case (list@JArray(_), nextPage: Int) =>
          val currentList: List[MergeRequest] = MergeRequest.parseList(list).get
          val nextPageListFuture: Future[List[MergeRequest]] = fetchMergeRequests(nextPage)
          // And how to merge these two lists?
        case (list@JArray(_), _) => MergeRequest.parseList(list).get
        case _ => throw new RuntimeException(s"No merge requests for project found")
      }
    }
  }

}

1 个答案:

答案 0 :(得分:1)

此处要处理的主要问题是,您试图将已经拥有的数据(List[MergeRequest])与将来将要检索的数据(Future[List[MergeRequest]])合并。您需要做一些事情来处理这种情况:

  • 在HTTP请求的结果上使用flatMap代替map。这样,您就可以在递归中提出其他HTTP请求,但将它们映射回单个Future
  • 在递归map的结果上调用fetchMergeRequests(nextPage),以将您已有的数据与来自递归的将来数据进行合并。
  • Future.successful()中包装另一个列表,因为flatMap要求所有模式匹配都返回Future —例外。

我对您所使用的库不熟悉,因此我尚未对其进行测试,但是我认为您的代码应该像这样工作:

def fetchMergeRequests(startPage: Int = 1): Future[List[MergeRequest]] = {
  val requestsUrl = url(s"https://gitlab.com/api/v4/projects/gitlab-org%2Fgitlab-ce/merge_requests?state=opened&per_page=3&page=${startPage}")
  Http(requestsUrl).flatMap { res =>
    (parse(res.getResponseBody), res.getHeader("X-Next-Page").toInt) match {
      case (list@JArray(_), nextPage: Int) =>
        val currentList: List[MergeRequest] = MergeRequest.parseList(list).get
        val nextPageListFuture: Future[List[MergeRequest]] = fetchMergeRequests(nextPage)
        nextPageListFuture.map(nextPageList => currentList ++ nextPageList)
      case (list@JArray(_), _) =>
        Future.successful(MergeRequest.parseList(list).get)
      case _ => throw new RuntimeException(s"No merge requests for project found")
    }
  }
}