Question

我正在尝试使用KPL库写入Amazon Kinesis Firehose。尽管我已尽力，但每次尝试连接和写入库的操作都会导致以下日志行：

2018-09-11 15:23:12.369  INFO 79765 --- [kpl-daemon-0003] c.a.s.k.producer.LogInputStreamReader    : [2018-09-11 15:23:12.368769] [0x0001379a][0x00007fff8bdd9380] [info] [main.cc:382] Starting up main producer
2018-09-11 15:23:12.369  INFO 79765 --- [kpl-daemon-0003] c.a.s.k.producer.LogInputStreamReader    : [2018-09-11 15:23:12.368833] [0x0001379a][0x00007fff8bdd9380] [info] [kinesis_producer.cc:101] Using Region: us-west-2
2018-09-11 15:23:12.369  INFO 79765 --- [kpl-daemon-0003] c.a.s.k.producer.LogInputStreamReader    : [2018-09-11 15:23:12.368850] [0x0001379a][0x00007fff8bdd9380] [info] [kinesis_producer.cc:120] Using per request threading model.
2018-09-11 15:23:12.369  INFO 79765 --- [kpl-daemon-0003] c.a.s.k.producer.LogInputStreamReader    : [2018-09-11 15:23:12.368862] [0x0001379a][0x00007fff8bdd9380] [info] [kinesis_producer.cc:171] Using Kinesis endpoint firehose.us-west-2.amazonaws.com:443
2018-09-11 15:23:12.369  INFO 79765 --- [kpl-daemon-0003] c.a.s.k.producer.LogInputStreamReader    : [2018-09-11 15:23:12.368910] [0x0001379a][0x00007fff8bdd9380] [info] [kinesis_producer.cc:101] Using Region: us-west-2
2018-09-11 15:23:12.369  INFO 79765 --- [kpl-daemon-0003] c.a.s.k.producer.LogInputStreamReader    : [2018-09-11 15:23:12.368926] [0x0001379a][0x00007fff8bdd9380] [info] [kinesis_producer.cc:120] Using per request threading model.
2018-09-11 15:23:12.369  INFO 79765 --- [kpl-daemon-0003] c.a.s.k.producer.LogInputStreamReader    : [2018-09-11 15:23:12.368940] [0x0001379a][0x00007fff8bdd9380] [info] [kinesis_producer.cc:52] Using default CloudWatch endpoint
2018-09-11 15:23:12.369  INFO 79765 --- [kpl-daemon-0003] c.a.s.k.producer.LogInputStreamReader    : [2018-09-11 15:23:12.369002] [0x0001379a][0x00007fff8bdd9380] [info] [main.cc:393] Entering join
2018-09-11 15:23:12.373  INFO 79765 --- [kpl-daemon-0003] c.a.s.k.producer.LogInputStreamReader    : [2018-09-11 15:23:12.372933] [0x0001379a][0x000070000c36a000] [info] [kinesis_producer.cc:197] Created pipeline for stream "redshift-guid-test"
2018-09-11 15:23:12.373  INFO 79765 --- [kpl-daemon-0003] c.a.s.k.producer.LogInputStreamReader    : [2018-09-11 15:23:12.373005] [0x0001379a][0x000070000c36a000] [info] [shard_map.cc:79] Updating shard map for stream "redshift-guid-test"
2018-09-11 15:23:13.102  WARN 79765 --- [kpl-daemon-0003] c.a.s.k.producer.LogInputStreamReader    : [2018-09-11 15:23:13.102494] [0x0001379a][0x000070000c805000] [warning] [AWS Log: WARN](AWSErrorMarshaller)Encountered Unknown AWSError
UnknownOperationException
:

2018-09-11 15:23:13.103 ERROR 79765 --- [kpl-daemon-0003] c.a.s.k.producer.LogInputStreamReader    : [2018-09-11 15:23:13.102572] [0x0001379a][0x000070000c805000] [error] [shard_map.cc:150] Shard map update for stream "redshift-guid-test" failed. Code: UnknownOperationException Message: Unable to parse ExceptionName: UnknownOperationException Message: ; retrying in 1000 ms
2018-09-11 15:23:14.002  WARN 79765 --- [kpl-daemon-0003] c.a.s.k.producer.LogInputStreamReader    : [2018-09-11 15:23:14.002252] [0x0001379a][0x000070000c90b000] [warning] [AWS Log: WARN](AWSErrorMarshaller)Encountered Unknown AWSError
UnknownOperationException

我能够使用标准的AWS Client来连接客户端并发送数据。

这是我创建客户端的方式。

       KinesisProducerConfiguration config = new KinesisProducerConfiguration()
            .setCredentialsProvider(credentials.getTemporaryCredentialsProvider())
            .setKinesisEndpoint("firehose.us-west-2.amazonaws.com")
            .setRegion(awsConfig.getRegion());

    KinesisProducer kinesisProducer = new KinesisProducer(config);

Answer 1

我试图将KPL连接到一个消防水带。我刚刚发现不支持此功能。上帝知道为什么不这样。

使用Amazon KPL获取“流xxx的碎片图更新”失败：（AWSErrorMarshaller）遇到未知的AWSError

1 个答案: