Question

我有以下代码来获取

def getSample(accessKey: String, secretKey: String, streamName: String, size: Int, endpoint: String, region: String): Unit = {
        var client=getKinesisClient(accessKey, secretKey, endpoint, region)
        val decoder = Charset.forName("UTF-8").newDecoder()
        var shards = getShards(accessKey, secretKey, streamName, endpoint, region)
        val numberOfShards = shards.iterator()
        var isStop = false
        var sampleRecords = new StringBuilder
        while (numberOfShards.hasNext() && !isStop) {
            val shard = numberOfShards.next()
            var shardIterator: String = null
            val getShardIteratorRequest = new GetShardIteratorRequest()
            getShardIteratorRequest.setStreamName(streamName)
            getShardIteratorRequest.setShardId(shard.getShardId())
            getShardIteratorRequest.setShardIteratorType("TRIM_HORIZON")
            val getShardIteratorResult = client.getShardIterator(getShardIteratorRequest)
            shardIterator = getShardIteratorResult.getShardIterator
            var getRecordsRequest = new GetRecordsRequest
            getRecordsRequest.setShardIterator(shardIterator)
            var records: List[Record] = client.getRecords(getRecordsRequest).getRecords()
            println(records.size())
            var itr = records.iterator()
            var SampleData = new ArrayList[String]()
            while (itr.hasNext()) {
                val record = itr.next()
                val data = decoder.decode(record.getData).toString
                println(data)
            }
        }
    }

这打印记录计数为0.我和aws cli相同

aws kinesis put-records --records "Data=hello world - hemant,PartitionKey=20150421" --stream-name simpleStream

它返回0个失败的记录然后我运行了以下命令

aws kinesis get-shard-iterator --stream-name simpleStream --shard-id 0 --shard-iterator-type TRIM_HORIZON

和

aws kinesis get-records --shard-iterator xxxxxxxxx

它也返回空数组。

有人可以帮帮我吗？感谢

Answer 1

即使分片中有记录，

GetRecords也不会始终返回记录。这在FAQ中有描述。

https://aws.amazon.com/kinesis/streams/faqs/

问：为什么当我的Amazon Kinesis流中有数据时，GetRecords调用返回空结果？

您要求TRIM_HORIZON位置的记录，这是分片中最旧的位置。如果连续多次调用get-records（10或20或更多）并继续执行迭代器点，则会遇到记录。

以下论坛也很有用:: https://forums.aws.amazon.com/thread.jspa?messageID=509980

AWS kinesis不会返回记录

1 个答案: