Question

我正在学习Akka Streams，作为练习，我想将日志插入Cassandra。问题是我无法使流将日志插入数据库。

我天真地尝试了以下内容：

object Application extends AkkaApp with LogApacheDao {

  // The log file is read line by line
  val source: Source[String, Unit] = Source.fromIterator(() => scala.io.Source.fromFile(filename).getLines())

  // Each line is converted to an ApacheLog object
  val flow: Flow[String, ApacheLog, Unit] = Flow[String]
    .map(rawLine => {
      rawLine.split(",") // implicit conversion Array[String] -> ApacheLog
    })

  // Log objects are inserted to Cassandra
  val sink: Sink[ApacheLog, Future[Unit]] = Sink.foreach[ApacheLog] { log => saveLog(log) }

  source.via(flow).to(sink).run()

}

saveLog（）在LogApacheDao中定义如下（我省略了更清晰代码的列值）：

val session = cluster.connect()

session.execute(s"CREATE KEYSPACE IF NOT EXISTS $keyspace WITH replication = {'class':'SimpleStrategy', 'replication_factor':1};")

session.execute(s"DROP TABLE IF EXISTS $keyspace.$table;")

session.execute(s"CREATE TABLE $keyspace.$table (...)")

val preparedStatement = session.prepare(s"INSERT INTO $keyspace.$table (...) VALUES (...);")

def saveLog(logEntry: ApacheLog) = {
    val stmt = preparedStatement.bind(...)

    session.executeAsync(stmt)
  }

进入接收器时从Array [String]到ApacheLog的转换没有问题（用println验证）。此外，键空间和表都是创建的，但是当执行到saveLog时，似乎有些东西被阻塞而且没有插入。

我没有任何错误，但Cassandra驱动程序核心（3.0.0）一直给我：

Connection[/172.17.0.2:9042-1, inFlight=0, closed=false] was inactive for 30 seconds, sending heartbeat
Connection[/172.17.0.2:9042-2, inFlight=0, closed=false] heartbeat query succeeded

我应该补充一点，我使用了一个dockerized Cassandra。

Answer 1

尝试使用Cassandra Connector in alpakka。

使用Akka Streams插入Cassandra

1 个答案: