Akka-http第一个Websocket客户端仅从kafka主题接收数据一次

时间:2018-07-20 19:26:26

标签: scala akka akka-stream akka-http

我正在使用akka-http websocket将消息从kafka主题推送到websocket客户端。

为此,我创建了一个普通的kafka用户(使用akka-streams-kafka连接器),其偏移设置为“最早”,以便每个新的Websocket客户端连接都从一开始就获取所有数据。

问题在于,第一个连接的websocket客户端获取所有数据,而其他ws客户端(在第一个客户端获取所有数据之后进行连接)没有任何获取。 kafka主题有100万条记录。

我正在使用Akka流中的BroadcastHub。

赞赏任何建议。

lazy private val kafkaPlainSource: Source[String, NotUsed] = {

  val consumerSettings = ConsumerSettings(system, new StringDeserializer, new StringDeserializer)
  .withBootstrapServers(KAFKA_BROKERS)
  .withGroupId(UUID.randomUUID().toString)
  .withProperty(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest")

  val kafkaSource = Consumer.plainSource(consumerSettings, Subscriptions.topics(KAFKA_TOPIC))
  .mapAsync(PARALLELISM) { cr =>
    Future {
      cr.value         
    }
  }
  kafkaSource.toMat(BroadcastHub.sink)(Keep.right).run
}

def logicFlow: Flow[String, String, NotUsed] = 
   Flow.fromSinkAndSourceCoupled(Sink.ignore, kafkaSource)

val websocketFlow: Flow[Message, Message, Any] = {
  Flow[Message]
  .map {
     case TextMessage.Strict(msg) => msg
     case _ => println("ignore streamed message")
}   
.via(logicFlow)
.map { msg: String => TextMessage.Strict(msg) }
}

lazy private val streamRoute =
 path("stream") {
   handleWebSocketMessages {
    websocketFlow          
      .watchTermination() { (_, done) =>
      done.onComplete {
        case Success(_) =>
          log.info("Stream route completed successfully")
        case Failure(ex) =>
          log.error(s"Stream route completed with failure : $ex")
      }
    }
  }
}

def startServer(): Unit = {
 bindingFuture = Http().bindAndHandle(wsRoutes, HOST, PORT)
 log.info(s"Server online at http://localhost:9000/")
}

def stopServer(): Unit = {
 bindingFuture
 .flatMap(_.unbind())
 .onComplete{
  _ => system.terminate()
  log.info("terminated")
 }
}

0 个答案:

没有答案