BatchWriteItem使用DynamoDB和AWSSDK.net

时间:2013-03-07 05:30:25

标签: c# .net amazon-dynamodb

我在使用DynamoDB调用BatchWriteItem请求时遇到问题。我收到以下错误

“无法将数据保存到数据库。超出了表格配置的预配置吞吐量级别。请考虑使用UpdateTable API提高配置级别”

配置:

ThroughPut:读= 5;写= 5;

这是我的情景: 我需要一个接一个地对同一个表执行2组数据写请求   - 使用聚合数据(932KB:拆分成63KB的块,大约15块)   - 非聚合数据(940 KB:拆分为63KB的块,约为15块)

在执行写操作之前,我将根据哈希键和范围键值删除dynamo-db表中的所有项目(对于聚合数据:删除16个项目)。

以下是代码:

public void PutItems(string tableName, string id, string message, bool aggregated)
    {
         if (aggregated)
             id = "a_" + id;
         else
             id = "r_" + id;
        List<WriteRequest> writeRequests = new List<WriteRequest>();
        List<WriteRequest> DeleteWriteRequest = new List<WriteRequest>();
        var itemsToDelete = GetItemsToDelete(tableName, id);
        if (itemsToDelete.Count > 0)
        {
            foreach (Tuple<string, int> item in itemsToDelete)
            {

                DeleteRequest deleteRequest = new DeleteRequest
                {
                    Key = new Key
                    {
                        HashKeyElement = new AttributeValue { S = item.Item1 },
                        RangeKeyElement = new AttributeValue { N = item.Item2.ToString() }
                    }
                };
                WriteRequest deleteReq = new WriteRequest();
                deleteReq.DeleteRequest = deleteRequest;
                DeleteWriteRequest.Add(deleteReq);
            }
            logger.Info(this.GetType().ToString(), string.Format(".....Deleting Old Data..........."));
            foreach (IEnumerable<WriteRequest> ls in LinqExtensions.Partition(DeleteWriteRequest, SplitLevel))
            {
                BatchWriteItemRequest delWriteRequest = new BatchWriteItemRequest();
                delWriteRequest.WithRequestItems(new KeyValuePair<string, List<WriteRequest>>(tableName, ls.ToList()));

                CallBatchWriteTillCompletion(delWriteRequest);
            }
            logger.Info(this.GetType().ToString(), string.Format(".....Delete Complete!..........")); 
        }

        int MaxLength = 64512; //64KB = 65536 Bytes ; 60KB = 61440 Bytes
        logger.Info(this.GetType().ToString(), string.Format("Message Size : {0}", message.Length)); 
        var str = SplitToChunks(message, MaxLength).ToList();
        for (int i = 0; i < str.Count; i++)
        {



            PutRequest putRequest = new PutRequest
            {
                Item = new Dictionary<string, AttributeValue>()
                    {
                        {"Received", new AttributeValue {S = id}},
                        {"SequenceNum" , new AttributeValue {N = i.ToString()}},
                        {"Message", new AttributeValue {S = str[i]}}
                    }
            };
            WriteRequest request = new WriteRequest();
            request.PutRequest = putRequest;
            writeRequests.Add(request);
        }

        logger.Info(this.GetType().ToString(), string.Format(".....Writing Data..........."));
        foreach (IEnumerable<WriteRequest> ls in LinqExtensions.Partition(writeRequests, SplitLevel))
        {
                System.Threading.Thread.Sleep(1000);
                BatchWriteItemRequest writeRequest = new BatchWriteItemRequest();
                writeRequest.WithRequestItems(new KeyValuePair<string, List<WriteRequest>>(tableName, ls.ToList()));
                CallBatchWriteTillCompletion(writeRequest);


        }
        logger.Info(this.GetType().ToString(), string.Format(".....Write Complete!.........."));
    }


    private void CallBatchWriteTillCompletion(BatchWriteItemRequest request)
    {
        BatchWriteItemResponse response;

        int callCount = 0;
        do
        {
            if (callCount > 0)
            {
                System.Threading.Thread.Sleep(1000);
            }
            logger.Info(this.GetType().ToString(), string.Format("Making Request"));
            response = Instance.Client.BatchWriteItem(request);
            callCount++;

            // Check the response.
            var result = response.BatchWriteItemResult;
            var responses = result.Responses;
            var unprocessed = result.UnprocessedItems;

            logger.Info(this.GetType().ToString(), string.Format("Response"));
            foreach (var resp in responses)
            {

                logger.Info(this.GetType().ToString(), string.Format("{0} - {1}", resp.Key, resp.Value.ConsumedCapacityUnits));
            }

            logger.Info(this.GetType().ToString(), string.Format("Unprocessed"));
            foreach (var unp in unprocessed)
            {

                logger.Info(this.GetType().ToString(), string.Format("{0} - {1}", unp.Key, unp.Value.Count));

            }

            // For the next iteration, the request will have unprocessed items.
            request.RequestItems = unprocessed;

        } while (response.BatchWriteItemResult.UnprocessedItems.Count > 0);

        logger.Info(this.GetType().ToString(), string.Format("Total # of batch write API calls made: {0}", callCount));

    }



    //This method is use to split a string message of smaller chunks of 64KB.
    private static IEnumerable<String> SplitToChunks(String str, int maxLength)
    {
            for (int index = 0; index < str.Length; index += maxLength)
            {
                yield return str.Substring(index, Math.Min(maxLength, str.Length - index));
            }          
    }

静态类LinqExtensions     {

    public static IEnumerable<IEnumerable<T>> Partition<T>(this IEnumerable<T> source, int size)
    {
        T[] array = null;
        int count = 0;
        foreach (T item in source)
        {
            if (array == null)
            {
                array = new T[size];
            }
            array[count] = item;
            count++;
            if (count == size)
            {
                yield return new ReadOnlyCollection<T>(array);
                array = null;
                count = 0;
            }
        }
        if (array != null)
        {
            Array.Resize(ref array, count);
            yield return new ReadOnlyCollection<T>(array);
        }
    }

}

你能告诉我这里我做错了吗?我在多个地方增加了1秒的延迟,因此通过率永远不会超过。我还附上了一个带日志的文本文件 它有ConsumedCapacityUnits。

2013-03-06 21:20:21.6093 INFO     
2013-03-06 21:20:21.7031 INFO     .........Writing Aggregated Messages to DynamoDB ...
2013-03-06 21:20:21.7343 INFO     Heartbeat event fired. Processing heartbeat ...
2013-03-06 21:20:22.2500 INFO     Working mode: MASTER.
2013-03-06 21:20:22.3750 INFO     Heartbeat message '3/6/2013 9:20:22 PM|OAK-VENUGOPALP' written to SQS queue 'HeartbeatDataReceiver_dev'.
2013-03-06 21:20:22.9062 INFO     .....Deleting Old Data...........
2013-03-06 21:20:22.9531 INFO     Making Request
2013-03-06 21:20:23.1250 INFO     Response
2013-03-06 21:20:23.1562 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 320
2013-03-06 21:20:23.2031 INFO     Unprocessed
2013-03-06 21:20:23.2500 INFO     Total # of batch write API calls made: 1
2013-03-06 21:20:23.2812 INFO     Making Request
2013-03-06 21:20:23.4218 INFO     Response
2013-03-06 21:20:23.5468 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 320
2013-03-06 21:20:23.5937 INFO     Unprocessed
2013-03-06 21:20:23.6406 INFO     Total # of batch write API calls made: 1
2013-03-06 21:20:23.6718 INFO     Making Request
2013-03-06 21:20:26.1406 INFO     Response
2013-03-06 21:20:26.2031 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 320
2013-03-06 21:20:26.2500 INFO     Unprocessed
2013-03-06 21:20:26.2968 INFO     Total # of batch write API calls made: 1
2013-03-06 21:20:26.3593 INFO     Making Request
2013-03-06 21:20:28.6875 INFO     Response
2013-03-06 21:20:28.7187 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 54
2013-03-06 21:20:28.7500 INFO     Unprocessed
2013-03-06 21:20:28.7968 INFO     Total # of batch write API calls made: 1
2013-03-06 21:20:28.8750 INFO     .....Delete Complete!..........
2013-03-06 21:20:28.9218 INFO     Message Size : 1022580
2013-03-06 21:20:28.9687 INFO     .....Writing Data...........
2013-03-06 21:20:30.0000 INFO     Making Request
2013-03-06 21:20:33.2812 INFO     Response
2013-03-06 21:20:33.3125 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 128
2013-03-06 21:20:33.3437 INFO     Unprocessed
2013-03-06 21:20:33.3906 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 3
2013-03-06 21:20:34.4218 INFO     Making Request
2013-03-06 21:20:36.2343 INFO     Response
2013-03-06 21:20:36.2656 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 128
2013-03-06 21:20:36.2968 INFO     Unprocessed
2013-03-06 21:20:36.3281 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 1
2013-03-06 21:20:36.6093 INFO     
2013-03-06 21:20:36.6406 INFO     Heartbeat event fired. Processing heartbeat ...
2013-03-06 21:20:37.0156 INFO     Working mode: MASTER.
2013-03-06 21:20:37.0781 INFO     Heartbeat message '3/6/2013 9:20:37 PM|OAK-VENUGOPALP' written to SQS queue 'HeartbeatDataReceiver_dev'.
2013-03-06 21:20:37.3750 INFO     Making Request
2013-03-06 21:20:38.0781 INFO     Response
2013-03-06 21:20:38.1093 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 64
2013-03-06 21:20:38.1406 INFO     Unprocessed
2013-03-06 21:20:38.1718 INFO     Total # of batch write API calls made: 3
2013-03-06 21:20:39.2187 INFO     Making Request
2013-03-06 21:20:42.2812 INFO     Response
2013-03-06 21:20:42.3125 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 128
2013-03-06 21:20:42.3437 INFO     Unprocessed
2013-03-06 21:20:42.3906 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 3
2013-03-06 21:20:43.4218 INFO     Making Request
2013-03-06 21:20:45.2187 INFO     Response
2013-03-06 21:20:45.2500 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 128
2013-03-06 21:20:45.2968 INFO     Unprocessed
2013-03-06 21:20:45.3281 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 1
2013-03-06 21:20:46.3593 INFO     Making Request
2013-03-06 21:20:47.0156 INFO     Response
2013-03-06 21:20:47.0625 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 64
2013-03-06 21:20:47.0937 INFO     Unprocessed
2013-03-06 21:20:47.1250 INFO     Total # of batch write API calls made: 3
2013-03-06 21:20:48.1718 INFO     Making Request
2013-03-06 21:20:51.2968 INFO     Response
2013-03-06 21:20:51.3437 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 128
2013-03-06 21:20:51.3906 INFO     Unprocessed
2013-03-06 21:20:51.4218 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 3
2013-03-06 21:20:51.6093 INFO     
2013-03-06 21:20:51.6406 INFO     Heartbeat event fired. Processing heartbeat ...
2013-03-06 21:20:52.0000 INFO     Working mode: MASTER.
2013-03-06 21:20:52.0781 INFO     Heartbeat message '3/6/2013 9:20:52 PM|OAK-VENUGOPALP' written to SQS queue 'HeartbeatDataReceiver_dev'.
2013-03-06 21:20:52.4531 INFO     Making Request
2013-03-06 21:20:54.2656 INFO     Response
2013-03-06 21:20:54.3125 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 128
2013-03-06 21:20:54.3437 INFO     Unprocessed
2013-03-06 21:20:54.3750 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 1
2013-03-06 21:20:55.6718 INFO     Making Request
2013-03-06 21:20:56.3281 INFO     Response
2013-03-06 21:20:56.3750 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 64
2013-03-06 21:20:56.4375 INFO     Unprocessed
2013-03-06 21:20:56.5312 INFO     Total # of batch write API calls made: 3
2013-03-06 21:20:57.5937 INFO     Making Request
2013-03-06 21:20:58.1875 INFO     Response
2013-03-06 21:20:58.2343 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 54
2013-03-06 21:20:58.3281 INFO     Unprocessed
2013-03-06 21:20:58.5625 INFO     Total # of batch write API calls made: 1
2013-03-06 21:20:58.5937 INFO     .....Write Complete!..........
2013-03-06 21:20:58.6875 INFO     .........Writing Complete! ...
2013-03-06 21:20:58.7187 INFO     .........Writing Raw Messages to DynamoDB ...
2013-03-06 21:21:00.8437 INFO     .....Deleting Old Data...........
2013-03-06 21:21:00.8906 INFO     Making Request
2013-03-06 21:21:06.0000 INFO     New Message Received
2013-03-06 21:21:06.6093 INFO     
2013-03-06 21:21:06.6562 INFO     Heartbeat event fired. Processing heartbeat ...
2013-03-06 21:21:07.0468 INFO     Working mode: MASTER.
2013-03-06 21:21:07.1250 INFO     Heartbeat message '3/6/2013 9:21:07 PM|OAK-VENUGOPALP' written to SQS queue 'HeartbeatDataReceiver_dev'.
2013-03-06 21:21:08.2343 INFO     Response
2013-03-06 21:21:08.2812 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 39
2013-03-06 21:21:08.3281 INFO     Unprocessed
2013-03-06 21:21:08.4062 INFO     Total # of batch write API calls made: 1
2013-03-06 21:21:08.4843 INFO     .....Delete Complete!..........
2013-03-06 21:21:08.5156 INFO     Message Size : 1006556
2013-03-06 21:21:08.5625 INFO     .....Writing Data...........
2013-03-06 21:21:09.6093 INFO     Making Request
2013-03-06 21:21:15.7187 INFO     Response
2013-03-06 21:21:15.7656 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 64
2013-03-06 21:21:15.7968 INFO     Unprocessed
2013-03-06 21:21:15.8437 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 4
2013-03-06 21:21:16.8750 INFO     Making Request
2013-03-06 21:21:21.6093 INFO     
2013-03-06 21:21:21.6562 INFO     Heartbeat event fired. Processing heartbeat ...
2013-03-06 21:21:23.4687 INFO     Working mode: MASTER.
2013-03-06 21:21:24.1406 INFO     Heartbeat message '3/6/2013 9:21:23 PM|OAK-VENUGOPALP' written to SQS queue 'HeartbeatDataReceiver_dev'.
2013-03-06 21:21:27.3593 INFO     Response
2013-03-06 21:21:27.4062 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 64
2013-03-06 21:21:27.4531 INFO     Unprocessed
2013-03-06 21:21:27.5000 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 3
2013-03-06 21:21:28.5312 INFO     Making Request
2013-03-06 21:21:36.6093 INFO     
2013-03-06 21:21:36.9687 INFO     Heartbeat event fired. Processing heartbeat ...
2013-03-06 21:21:38.1718 INFO     Working mode: MASTER.
2013-03-06 21:21:38.2343 INFO     Heartbeat message '3/6/2013 9:21:38 PM|OAK-VENUGOPALP' written to SQS queue 'HeartbeatDataReceiver_dev'.
2013-03-06 21:21:40.2031 INFO     Response
2013-03-06 21:21:40.2500 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 64
2013-03-06 21:21:40.3281 INFO     Unprocessed
2013-03-06 21:21:40.3593 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 2
2013-03-06 21:21:41.3906 INFO     Making Request
2013-03-06 21:21:51.6093 INFO     
2013-03-06 21:21:51.7500 INFO     Heartbeat event fired. Processing heartbeat ...
2013-03-06 21:21:52.1718 INFO     Working mode: MASTER.
2013-03-06 21:21:52.2500 INFO     Heartbeat message '3/6/2013 9:21:52 PM|OAK-VENUGOPALP' written to SQS queue 'HeartbeatDataReceiver_dev'.
2013-03-06 21:21:54.0937 INFO     Response
2013-03-06 21:21:54.1562 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 64
2013-03-06 21:21:54.1875 INFO     Unprocessed
2013-03-06 21:21:54.2343 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 1
2013-03-06 21:21:55.2812 INFO     Making Request
2013-03-06 21:22:06.2031 INFO     New Message Received
2013-03-06 21:22:06.6093 INFO     
2013-03-06 21:22:06.6406 INFO     Heartbeat event fired. Processing heartbeat ...
2013-03-06 21:22:07.0312 INFO     Working mode: MASTER.
2013-03-06 21:22:07.5781 INFO     Response
2013-03-06 21:22:07.6406 INFO     Heartbeat message '3/6/2013 9:22:07 PM|OAK-VENUGOPALP' written to SQS queue 'HeartbeatDataReceiver_dev'.
2013-03-06 21:22:07.7812 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 64
2013-03-06 21:22:07.8437 INFO     Unprocessed
2013-03-06 21:22:07.9375 INFO     Total # of batch write API calls made: 5
2013-03-06 21:22:08.9687 INFO     Making Request
2013-03-06 21:22:18.0156 INFO     Response
2013-03-06 21:22:18.0781 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 64
2013-03-06 21:22:18.1250 INFO     Unprocessed
2013-03-06 21:22:18.1562 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 4
2013-03-06 21:22:19.2031 INFO     Making Request
2013-03-06 21:22:21.6093 INFO     
2013-03-06 21:22:21.6875 INFO     Heartbeat event fired. Processing heartbeat ...
2013-03-06 21:22:21.7031 INFO     Response
2013-03-06 21:22:21.7656 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 64
2013-03-06 21:22:21.8281 INFO     Unprocessed
2013-03-06 21:22:21.8750 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 3
2013-03-06 21:22:22.0937 INFO     Working mode: MASTER.
2013-03-06 21:22:22.1875 INFO     Heartbeat message '3/6/2013 9:22:22 PM|OAK-VENUGOPALP' written to SQS queue 'HeartbeatDataReceiver_dev'.
2013-03-06 21:22:22.9375 INFO     Making Request
2013-03-06 21:22:34.9687 INFO     Response
2013-03-06 21:22:35.0156 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 64
2013-03-06 21:22:35.0468 INFO     Unprocessed
2013-03-06 21:22:35.0781 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 2
2013-03-06 21:22:36.1093 INFO     Making Request
2013-03-06 21:22:36.6093 INFO     
2013-03-06 21:22:36.7656 INFO     Heartbeat event fired. Processing heartbeat ...
2013-03-06 21:22:38.2968 INFO     Working mode: MASTER.
2013-03-06 21:22:38.5468 INFO     Heartbeat message '3/6/2013 9:22:38 PM|OAK-VENUGOPALP' written to SQS queue 'HeartbeatDataReceiver_dev'.
2013-03-06 21:22:48.7968 INFO     Response
2013-03-06 21:22:48.8750 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 64
2013-03-06 21:22:48.9062 INFO     Unprocessed
2013-03-06 21:22:48.9375 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 1
2013-03-06 21:22:50.0000 INFO     Making Request
2013-03-06 21:22:51.6093 INFO     
2013-03-06 21:22:51.8281 INFO     Heartbeat event fired. Processing heartbeat ...
2013-03-06 21:22:52.8125 INFO     Working mode: MASTER.
2013-03-06 21:22:52.8906 INFO     Heartbeat message '3/6/2013 9:22:52 PM|OAK-VENUGOPALP' written to SQS queue 'HeartbeatDataReceiver_dev'.
2013-03-06 21:23:02.2968 INFO     Response
2013-03-06 21:23:02.3281 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 64
2013-03-06 21:23:02.3593 INFO     Unprocessed
2013-03-06 21:23:02.3906 INFO     Total # of batch write API calls made: 5
2013-03-06 21:23:03.4375 INFO     Making Request
2013-03-06 21:23:06.6093 INFO     
2013-03-06 21:23:06.7812 INFO     Heartbeat event fired. Processing heartbeat ...
2013-03-06 21:23:07.5000 INFO     New Message Received
2013-03-06 21:23:08.8593 INFO     Working mode: MASTER.
2013-03-06 21:23:09.3281 INFO     Heartbeat message '3/6/2013 9:23:09 PM|OAK-VENUGOPALP' written to SQS queue 'HeartbeatDataReceiver_dev'.
2013-03-06 21:23:15.3906 INFO     Response
2013-03-06 21:23:15.4375 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 64
2013-03-06 21:23:15.4843 INFO     Unprocessed
2013-03-06 21:23:15.5312 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 4
2013-03-06 21:23:16.5625 INFO     Making Request
2013-03-06 21:23:21.6093 INFO     
2013-03-06 21:23:21.8593 INFO     Heartbeat event fired. Processing heartbeat ...
2013-03-06 21:23:23.7031 INFO     Working mode: MASTER.
2013-03-06 21:23:24.0468 INFO     Heartbeat message '3/6/2013 9:23:23 PM|OAK-VENUGOPALP' written to SQS queue 'HeartbeatDataReceiver_dev'.
2013-03-06 21:23:28.7812 INFO     Response
2013-03-06 21:23:28.8125 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 64
2013-03-06 21:23:28.8750 INFO     Unprocessed
2013-03-06 21:23:29.1718 INFO     Key: TrafficConditionsMessages_dev - Unprocessed Value Count: 3
2013-03-06 21:23:30.4375 INFO     Making Request
2013-03-06 21:23:31.6093 INFO     Run task Called......
2013-03-06 21:23:31.9375 INFO     Processing Read data Method
2013-03-06 21:23:32.7187 INFO     Saving data to DB...
2013-03-06 21:23:33.0156 INFO     .........Writing Aggregated Messages to DynamoDB ...
2013-03-06 21:23:36.8125 INFO     
2013-03-06 21:23:37.0156 INFO     Heartbeat event fired. Processing heartbeat ...
2013-03-06 21:23:37.4531 INFO     .....Deleting Old Data...........
2013-03-06 21:23:37.7500 INFO     Making Request
2013-03-06 21:23:38.5156 INFO     Working mode: MASTER.
2013-03-06 21:23:38.8593 INFO     Heartbeat message '3/6/2013 9:23:38 PM|OAK-VENUGOPALP' written to SQS queue 'HeartbeatDataReceiver_dev'.
2013-03-06 21:23:41.6875 INFO     Response
2013-03-06 21:23:41.9531 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 320
2013-03-06 21:23:42.2500 INFO     Unprocessed
2013-03-06 21:23:42.3593 INFO     Total # of batch write API calls made: 1
2013-03-06 21:23:42.4218 INFO     Making Request
2013-03-06 21:23:50.5468 ERROR    Unable to save data to DB. The level of configured provisioned throughput for the table was exceeded. Consider increasing your provisioning level with the UpdateTable API
2013-03-06 21:23:50.5781 INFO     .....Save to Database Complete!..... 
2013-03-06 21:23:50.7812 INFO     Run task Complete......
2013-03-06 21:23:51.6093 INFO     
2013-03-06 21:23:51.6406 INFO     Heartbeat event fired. Processing heartbeat ...
2013-03-06 21:23:53.1718 INFO     Working mode: MASTER.
2013-03-06 21:23:53.4531 INFO     Heartbeat message '3/6/2013 9:23:53 PM|OAK-VENUGOPALP' written to SQS queue 'HeartbeatDataReceiver_dev'.
2013-03-06 21:23:53.6093 ERROR    Unable to save data to DB. The level of configured provisioned throughput for the table was exceeded. Consider increasing your provisioning level with the UpdateTable API
2013-03-06 21:23:53.6562 INFO     .....Save to Database Complete!..... 
2013-03-06 21:23:53.8281 INFO     Run task Complete......

1 个答案:

答案 0 :(得分:1)

此数字(大致)转换为每秒KB:     ThroughPut:写= 5;

如果您正在尝试写入63KB的项目,则需要编写它,然后等待〜13秒再尝试再次写入。您已经配置了每秒5 KB,然后在一次突发中使用了63 KB。在接下来的13秒内,您写入表中的任何请求都可以被限制。

根据时间戳判断,客户端无形地处理ProvisionedThroughputExceeded错误并重试;例如,您在请求期间看到10秒后通过:

2013-03-06 21:22:08.9687 INFO     Making Request
2013-03-06 21:22:18.0156 INFO     Response

最终,在一堆较小的表之后,该表遇到了很大的要求:

2013-03-06 21:23:41.9531 INFO     Key: TrafficConditionsMessages_dev - Consumed Capacity Units: 320

并且下一个请求花了很长时间,以至于您的客户决定是时候向您展示错误了:

2013-03-06 21:23:42.4218 INFO     Making Request
2013-03-06 21:23:50.5468 ERROR    Unable to save data to DB. The ...

解决方案是提高您的预配置吞吐量,根据返回的ConsumedCapacity休眠更长时间,或者写下更少的数据。