flink如何反序列化特例

时间:2019-04-21 13:25:51

标签: scala apache-flink flink-streaming spray-json

我正在使用Flink的(1.7.2)kafka消费者。 如何反序列化具有相同特征的几个案例类? 例如

game.js

function highScoreSubmit(e) {
    e.preventDefault();

    const data = {
        name: "asdf",
        score: 14
    }

    const ref = database.ref('scores');
    ref.on('value', gotData);
  }

  function gotData(dataw) {
    const scores = dataw.val();
  }

我正在将此import spray.json.{DefaultJsonProtocol, RootJsonFormat} trait Foo case class Boo(name: String) extends Foo case class Buzz(name: String, age: Int) extends Foo object Formats extends DefaultJsonProtocol{ implicit val booFormat: RootJsonFormat[Boo] = jsonFormat1(Boo.apply) implicit val buzzFormat: RootJsonFormat[Buzz] = jsonFormat2(Buzz.apply) } 用于kafka使用者:

DeserializationSchema

任何想法都将不胜感激

1 个答案:

答案 0 :(得分:1)

尝试spray-json-shapeless,它可以像这样自动为ADT导出解码器:

sealed trait Foo
case class Boo(name: String) extends Foo
case class Buzz(name: String, age: Int) extends Foo

object MyFormats extends DefaultJsonProtocol with FamilyFormats {
  implicit val formats = shapeless.cachedImplicit[JsonFormat[Foo]]
}

记住将特征设为sealed。请注意,例如,原始JSON需要包含type字段disambiguatorrawJsonString.parseJson.convertTo[Foo]才能正常工作

  object Main extends App {
    import MyFormats._

    val rawJsonBuzz =
      """
        |{
        |  "name": "Picard",
        |  "age": 60,
        |  "type": "Buzz"
        |}
      """.stripMargin

    val buzz = rawJsonBuzz.parseJson.convertTo[Foo]
    println(buzz)

    val rawJsonBoo =
      """
        |{
        |  "name": "Picard",
        |  "type": "Boo"
        |}
      """.stripMargin

    val boo = rawJsonBoo.parseJson.convertTo[Foo]
    println(boo)
  }

应输出

Buzz(Picard,60)
Boo(Picard)