Question

我有一个简单的案例类：

  case class User(id: String, login: String, key: String)

我要添加字段“名称”

  case class User(id: String, login: String, name: String, key: String)

然后将其添加到Avro模式（user.avsc）

{
  "namespace": "test",
  "type": "record",
  "name": "User",
  "fields": [
    { "name": "id", "type": "string" },
    { "name": "login", "type": "string" },
    { "name": "name", "type": "string" },
    { "name": "key", "type": "string" }
  ]
}

该类用于其他案例类：

case class AuthRequest(user: User, session: String)

chema（auth_request.avsc）

{
  "namespace": "test",
  "type": "record",
  "name": "AuthRequest",
  "fields": [
    { "name": "user", "type": "User" },
    { "name": "session", "type": "string" }
  ]
}

在更改之后，我的消费者开始抛出异常

Consumer.committableSource(consumerSettings, Subscriptions.topics("token_service_auth_request"))
    .map { msg =>
      Try {
        val in: ByteArrayInputStream = new ByteArrayInputStream(msg.record.value())
        val input: AvroBinaryInputStream[AuthRequest] = AvroInputStream.binary[AuthRequest](in)
        val result: AuthRequest = input.iterator.toSeq.head !!!! here is exception
        msg.committableOffset.commitScaladsl()

        (msg.record.value(), result, msg.record.key())
      } match {
        case Success((a: Array[Byte], value: AuthRequest, key: String)) =>
          log.info(s"listener got $msg -> $a -> $value")

          context.parent ! value

        case Failure(e) => e.printStackTrace()
      }
    }
    .runWith(Sink.ignore)

java.util.NoSuchElementException：空流头位于 scala.collection.immutable.Stream $ Empty $ .head（Stream.scala：1104）在 scala.collection.immutable.Stream $ Empty $ .head（Stream.scala：1102）在 test.consumers.AuthRequestListener。$ anonfun $ new $ 2（AuthRequestListener.scala：39）在scala.util.Try $ .apply（Try.scala：209）在 test.consumers.AuthRequestListener。$ anonfun $ new $ 1（AuthRequestListener.scala：36）在 test.consumers.AuthRequestListener。$ anonfun $ new $ 1 $ adapted（AuthRequestListener.scala：35）在akka.stream.impl.fusing.Map $$ anon $ 9.onPush（Ops.scala：51）在 akka.stream.impl.fusing.GraphInterpreter.processPush（GraphInterpreter.scala：519）在 akka.stream.impl.fusing.GraphInterpreter.processEvent（GraphInterpreter.scala：482）在 akka.stream.impl.fusing.GraphInterpreter.execute（GraphInterpreter.scala：378）在 akka.stream.impl.fusing.GraphInterpreterShell.runBatch（ActorGraphInterpreter.scala：588）在 akka.stream.impl.fusing.GraphInterpreterShell $ AsyncInput.execute（ActorGraphInterpreter.scala：472）在 akka.stream.impl.fusing.GraphInterpreterShell.processEvent（ActorGraphInterpreter.scala：563）在 akka.stream.impl.fusing.ActorGraphInterpreter.akka $ stream $ impl $ fusing $ ActorGraphInterpreter $$ processEvent（ActorGraphInterpreter.scala：745）在 akka.stream.impl.fusing.ActorGraphInterpreter $$ anonfun $ receive $ 1.applyOrElse（ActorGraphInterpreter.scala：760）在akka.actor.Actor.aroundReceive（Actor.scala：517）在 akka.actor.Actor.aroundReceive $（Actor.scala：515）在 akka.stream.impl.fusing.ActorGraphInterpreter.aroundReceive（ActorGraphInterpreter.scala：670）在akka.actor.ActorCell.receiveMessage（ActorCell.scala：588）在 akka.actor.ActorCell.invoke（ActorCell.scala：557）在 akka.dispatch.Mailbox.processMailbox（Mailbox.scala：258）位于 akka.dispatch.Mailbox.run（Mailbox.scala：225）在 akka.dispatch.Mailbox.exec（Mailbox.scala：235）在 akka.dispatch.forkjoin.ForkJoinTask.doExec（ForkJoinTask.java:260）在 akka.dispatch.forkjoin.ForkJoinPool $ WorkQueue.runTask（ForkJoinPool.java:1339）在 akka.dispatch.forkjoin.ForkJoinPool.runWorker（ForkJoinPool.java:1979）在 akka.dispatch.forkjoin.ForkJoinWorkerThread.run（ForkJoinWorkerThread.java:107）

我试图清理内部版本并使缓存无效-我似乎在某种程度上缓存了先前版本的架构请帮忙！

Answer 1

您需要使您的更改向后兼容，以使新字段可以为空并为其添加默认值。

{
  "namespace": "test",
  "type": "record",
  "name": "User",
  "fields": [
    { "name": "id", "type": "string" },
    { "name": "login", "type": "string" },
    { "name": "name", "type": ["null", "string"], "default": null },
    { "name": "key", "type": "string" }
  ]
}

带有AVRO模式更新的问题

1 个答案: