创建KSQL流:如何从复杂的json中提取值

时间:2018-05-18 10:32:34

标签: json apache-kafka ksql

我正在尝试在Apache / KAFKA KSQL中创建一个流 该主题包含(有点复杂的JSON)

CREATE STREAM tstream (event_type varchar, created_at varchar) WITH (kafka_topic='usage_events', value_format='json');

现在创建一个流很容易就像event_type和created_at这样简单的东西。那就像这样

CREATE STREAM usage (event_type varchar,create_at varchar, used_service_units[0].amount int) WITH (kafka_topic='usage_events', value_format='json');

但现在我需要访问used_service_units .... 我想在上面的JSON中提取“金额”

我该怎么做?

line 1:78: mismatched input '[' expecting {'ADD', 'APPROXIMATE', ...

结果

CREATE STREAM usage (event_type varchar,create_at varchar, used_service_units varchar) WITH (kafka_topic='usage_events', value_format='json');

如果我改为创建一个像这样的流

SELECT EXTRACTJSONFIELD(used_service_units,'$.amount') FROM usage;
SELECT EXTRACTJSONFIELD(used_service_units[0],'$.amount') FROM usage;
SELECT EXTRACTJSONFIELD(used_service_units,'$[0].amount') FROM usage;

然后像这样在流上执行SQL SELECT

SELECT EXTRACTJSONFIELD(used_service_units[0],'$.amount') FROM usage;'

Code generation failed for SelectValueMapper

这些替代品都不起作用......

这个给了我

public static void addeqRestrictions(HashMap<String, Object> r) {
    for (Map.Entry<String, Object> entry : r.entrySet()) {
        criteria.add(Restrictions.eq(entry.getKey(), entry.getValue()));
    }
}

1 个答案:

答案 0 :(得分:5)

似乎这个问题的一个解决方案是使列数据类型成为一个数组 即

CREATE STREAM usage (event_type varchar,created_at varchar, total_charged_amount varchar, used_service_units array<varchar> ) WITH (kafka_topic='usage_events', value_format='json');

现在我可以执行以下操作:

SELECT EXTRACTJSONFIELD(used_service_units[0],'$.amount') FROM usage