Question

我在AWS中开发了一个管道，我通过Python 3从我的笔记本中收集CPU温度，使用安全协议发送到AWS IoT Core，规则将数据发送到Cloud Watch并且DynamoDB保存它。创建数据管道是为了将此DynamoDB数据保存到S3，我想使用Quick Sight生成此数据的图。

但是，我无法使Quick Sight正确读取文件。 S3文件如下所示：

{"timestamp":{"s":"1526819850637"},"payload":{"m":{"$Temperature":{"s":"42.000"}}}}
{"timestamp":{"s":"1526819976032"},"payload":{"m":{"$Temperature":{"s":"42.000"}}}}
{"timestamp":{"s":"1526819934216"},"payload":{"m":{"$Temperature":{"s":"42.000"}}}}
{"timestamp":{"s":"1526817845094"},"payload":{"m":{"$Temperature":{"s":"48.000"}}}}

当我使用下面的清单文件时，Quick Sight成功读取数据，但New Calculated字段中的'parseJson'命令消失，无法读取JSON：

{
    "fileLocations": [
        {
            "URIs": [
                "https://s3.amazonaws.com/my-bucket2/2018-05-20-12-32-49/12345-a279-1234-2269-491212345"
            ]
        },
        {
            "URIPrefixes": [
                "https://s3.amazonaws.com/my-bucket2/2018-05-20-12-32-49/12345-a279-1234-2269-491212345"
            ]
        }
    ],"globalUploadSettings": {
        "format": "CSV","delimiter":"\n","textqualifier":"'"
    }

}

Quick Sight将JSON读取为：

{{"timestamp":{"s":"1526819850637"},"payload":{"m":{"$Temperature":{"s":"42.000"}}}}}

...没有'parseJson'命令。

Quick Sight中的数据没有缺失值，AWS管道完美运行。我该怎么办？

Answer 1

我找到了让它发挥作用的方法。简单地：

{
    "fileLocations": [
        {
            "URIs": [
                "https://s3.amazonaws.com/your-bucket2/2018-05-20-12-32-49/123456789"
            ]
        },
        {
            "URIPrefixes": [
            "https://s3.amazonaws.com/your-bucket2/2018-05-20-12-32-49/123456789"
            ]
        }
    ],"globalUploadSettings": {"format":"JSON",
    "delimiter":"\n","textqualifier":"'"
    }
}

无法在AWS Quick Sight中从S3解析JSON文件

1 个答案: