我将Spark结构化流媒体与Kafka集成在一起,其中我正在听2个主题
def main(args: Array[String]): Unit = {
val schema = StructType(
List(
StructField("gatewayId", StringType, true),
StructField("userId", StringType, true)
)
)
val spark = SparkSession
.builder
.master("local[4]")
.appName("DeviceAutomation")
.getOrCreate()
val dfStatus = spark.readStream.
format("kafka").
option("subscribe", "utility-status").
option("kafka.bootstrap.servers", "localhost:9092").
option("startingOffsets", "earliest")
.load()
val dfCritical = spark.readStream.
format("kafka").
option("subscribe", "utility-critical").
option("kafka.bootstrap.servers", "localhost:9092").
option("startingOffsets", "earliest")
.load()
}
由于我将列出更多主题并执行不同的操作,因此我希望将每个主题移到单独的类中,以提高清晰度。是否有可能,以及如何从主班级启动它们?