尝试模拟DynamoDB时出现KeyError:“操作”

时间:2020-11-06 16:18:35

标签: python amazon-web-services unit-testing mocking amazon-dynamodb

要测试的代码:

class ElasticSearchReporter:
    def __init__(...):
        ...
        self._dynamodb_client = session.client('dynamodb')

    def upload_record_to_dynamodb(self, es_index, dataclass_instance):
        record_id = str(uuid.uuid4())
        ...
        dataclass_attributes[current_field.name] = {'Value': attribute_value}
        record_key = {'record_id': {'S': record_id}}
        self._dynamodb_client.update_item(TableName=es_index, Key=record_key, AttributeUpdates=dataclass_attributes)

为了测试它,我要做:

import unittest
import boto3

from moto import mock_dynamodb2
from unittest.mock import MagicMock, patch

class TestElasticSearch(unittest.TestCase):
    def setUp(self):
        self.es_record = ...

    @mock_dynamodb2
    @patch('boto3.client')
    def test_upload_record_to_dynamodb(self, mock_boto3_sts_client):
        table_name = 'my_table'
        dynamodb = boto3.resource('dynamodb', region_name='eu-west-1')

        table = dynamodb.create_table(
            TableName=table_name,
            KeySchema=[
                {
                    'AttributeName': 'record_id',
                    'KeyType': 'HASH'
                },
            ],
            AttributeDefinitions=[
                {
                    'AttributeName': 'record_id',
                    'AttributeType': 'S'
                },

            ]
        )
        # Wait until the table exists.
        table.meta.client.get_waiter('table_exists').wait(TableName=table_name)

        rtn_val = {
            'Credentials': {
                'AccessKeyId': 'test_access_key',
                'SecretAccessKey': 'test',
                'SessionToken': 'test_session',
            }
        }
        mock_sts_client = MagicMock()
        mock_sts_client.assume_role.return_value = rtn_val
        mock_boto3_sts_client.return_value = mock_sts_client

        es_reporter = ElasticSearchReporter(...)
        es_reporter.upload_record_to_dynamodb(table_name, self.es_record)

当我运行测试时,它失败并显示:

         es_reporter = ElasticSearchReporter(self.role_arn, None)
         es_reporter.upload_record_to_dynamodb(
             ElasticsearchIndices.ANALYSIS_BACKEND_SESSIONS_ALPHA,
 >           self.es_record
         )
 
 test/test_common/test_helpers/test_elasticsearch.py:167: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
 ../../build/DEV.STD.PTHREAD/build/lib/python3.7/site-packages/atvvqadac_common/helpers/elasticsearch.py:143: in upload_record_to_dynamodb
    self._dynamodb_client.update_item(TableName=es_index, Key=record_key, AttributeUpdates=dataclass_attributes)
<several lines here that I skip>
    item.update_with_attribute_updates(attribute_updates)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _  
 self = Item: {'Attributes': {'record_id': {'S': '6266046e-95c5-4448-aa2d-8d4959b6e306'}}}
 attribute_updates = {'aggregator_results_s3_path': {'Value': {'S': 's3:/my_bucket/2020-11-06T110210251207_81_my_session/inference-results/...:02:10.390290'}}, 'audio_s3_path': {'Value': {'S': 's3:/my_bucket/2020-11-06T110210251207_81_my_session/audio/'}}, ...}
 
     def update_with_attribute_updates(self, attribute_updates):
         for attribute_name, update_action in attribute_updates.items():
 >           action = update_action['Action']
 E           KeyError: 'Action'
 
 /home/gsamaras/pkg-cache/packages/Python-moto/Python-moto-1.x.67811.0/AL2012/DEV.STD.PTHREAD/build/lib/python3.7/site-packages/moto/dynamodb2/models.py:293: KeyError
----------------------------- Captured stderr call -----------------------------
020-11-06 15:55:29,175 INFO [botocore.credentials] Found credentials in environment variables.
2020-11-06 15:55:29 INFO [elastic_search] Record about to be sent to DynamoDB (record ID: 6266046e-95c5-4448-aa2d-8d4959b6e306): {'stage': {<and so on..>

您能帮我解决这个问题吗?

参考:

  1. how-to-mock-aws-dynamodb-service

1 个答案:

答案 0 :(得分:0)

发生崩溃的Moto代码如下:github.com/spulec/moto/blob/master/moto/dynamodb2/models/init.py#L112

def update_with_attribute_updates(self, attribute_updates):
    for attribute_name, update_action in attribute_updates.items():
        action = update_action["Action"]

即使在Boto3 Action中也不必传递DynamoDB.Client.update_item()的位置,因为该参数存在默认值。

这就是为什么要测试的代码仍然运行良好。 Documentation也证实了这一点(强调我的意思):

动作(字符串)-指定如何执行更新。有效值为PUT(默认),...

因此,此错误来自 Boto3和Moto差异

短期修复:

更改此:

dataclass_attributes[current_field.name] = {'Value': attribute_value}

收件人:

dataclass_attributes[current_field.name] = {'Value': attribute_value, 'Action': 'Put'}

但是,我创建了一个GitHub issue,并且我将打开一个带有修复程序的请求请求,以便将来的用户不会有相同的(客户)体验。编辑:漏洞修复刚刚合并!