Question

我设置了一个Kafka主题，并且正在尝试在Hive中创建一个外部表来查询Kafka流。

但是，当查询外部表时，我收到错误消息

Error: java.io.IOException: org.apache.kafka.common.config.ConfigException: Missing required configuration "group.id" which has no default value. (state=,code=0)

尝试在启动Kafka服务器时将group.id放入server.properties。

试图将group.id放在外部表属性中。

CREATE EXTERNAL TABLE kafka_table2
  (`timestamp` timestamp , `page` string,  `newPage` boolean,
  added int, deleted bigint, delta double)
  STORED BY 'org.apache.hadoop.hive.kafka.KafkaStorageHandler'
  TBLPROPERTIES
  ("kafka.topic" = "connect-test", "kafka.bootstrap.servers"="mykafka:9092","kafka.group.id"="1")


INFO  : Completed compiling command(queryId=hive_20190426082255_729f8adb-bb23-4317-8f3f-2f9049b62bd7); Time taken: 0.6 seconds
INFO  : Executing command(queryId=hive_20190426082255_729f8adb-bb23-4317-8f3f-2f9049b62bd7): select * from kafka_table2
INFO  : Completed executing command(queryId=hive_20190426082255_729f8adb-bb23-4317-8f3f-2f9049b62bd7); Time taken: 0.018 seconds
INFO  : OK
Error: java.io.IOException: org.apache.kafka.common.config.ConfigException: Missing required configuration "group.id" which has no default value. (state=,code=0)

Answer 1

您应该在TBLPROPERTIES中放置“ kafka。消费者 .group.id” =“ = 1”，而不是“ kafka.group.id” =” = 1“。

请参阅：https://docs.hortonworks.com/HDPDocuments/HDP3/HDP-3.1.0/integrating-hive/content/hive_set_consumer_producer.html

如何修复org.apache.kafka.common.config.ConfigException：缺少必需的配置“ group.id”，该配置没有默认值

1 个答案: