使用光滑的非阻塞方式插入postgres

时间:2018-02-16 13:57:24

标签: postgresql scala slick slick-3.0

class Employee(tag: Tag) extends Table[table_types.user](tag,  "EMPLOYEE") {

  def employeeID = column[Int]("EMPLOYEE_ID")
  def empName = column[String]("NAME")
  def startDate = column[String]("START_DATE")

  def * = (employeeID, empName, startDate)
}
object employeeHandle {

    def insert(emp:Employee):Future[Any] = {
      val dao = new SlickPostgresDAO
      val db = dao.db
      val insertdb = DBIO.seq(employee += (emp))
      db.run(insertdb)
  }
}

将数百万员工记录插入数据库

object Hello extends App {
    val employees = List[*1 million employee list*]
    for(employee<-employees) {
       employeeHandle.insert(employee)
       *Code to connect to rest api to confirm entry*
    }
}

然而,当我运行上面的代码时,我很快就会失去与Postgres的连接。我怎样才能并行(以非阻塞的方式),但同时确保我没有连接到postgres。

2 个答案:

答案 0 :(得分:0)

我认为您不需要并行执行;我不明白它是如何解决它的。相反,您可以在开始循环之前创建连接并将其传递给employeeHandle.insert(db, employee)

像(我不知道scala):

object Hello extends App {
    val dao = new SlickPostgresDAO
    val db = dao.db
    val employees = List[*1 million employee list*]
    for(employee<-employees) {
       employeeHandle.insert(db, employee)
       *Code to connect to rest api to confirm entry*
    }
}

答案 1 :(得分:0)

我遇到的光滑插入的几乎所有示例都使用阻塞来完成结果。如果有一个没有的话会很高兴。

我接受它:

object Hello extends App {
    val employees = List[*1 million employee list*]
    val groupedList = employees.grouped(10).toList
    insertTests()
    def insertTests(l: List[List[Employee]] = groupedList): Unit = {
      val ins = l.head
      val futures = ins.map { no => employeeHandle.insert(employee)}
      val seq = Future.sequence(futures)
      Await.result(seq, Duration.Inf)
      if(l.nonEmpty) insertTests(l.tail)
    }
}

插入句柄中的连接参数也应该在

之外
object employeeHandle {
      val dao = new SlickPostgresDAO
      val db = dao.db
    def insert(emp:Employee):Future[Any] = {
      val insertdb = DBIO.seq(employee += (emp))
      db.run(insertdb)
  }
}