amazonS3client.SelectObjectContentAsync-下载大型jsonline格式文件-不需要的换行符

时间:2018-12-18 11:38:25

标签: c# amazon-s3

我正尝试使用AWSSDK for C#的SelectObjectContentAsync方法从S3存储桶下载文件内容。 但是原始数据中间会有一些不需要的换行符(\ n)。 数据示例:

  

{“ Id”:1,“名称”:“ aaa”},{“ Id”:2,“ N   \ name“:” bbb“}

我的代码:

var amazonS3Client = new AmazonS3Client(awsAccessKeyId, awsSecretAccessKey, region);
                SelectObjectContentRequest selectObjectContentRequest = new SelectObjectContentRequest()
                {
                    Bucket = bucketName,
                    Key = key,
                    ExpressionType = ExpressionType.SQL,
                    Expression = query,
                    InputSerialization = new InputSerialization()
                    {
                        JSON = new JSONInput()
                        {
                            JsonType = JsonType.Lines
                        },
                        CompressionType = CompressionType.Gzip
                    },
                    OutputSerialization = new OutputSerialization()
                    {
                        JSON = new JSONOutput()
                        {
                            RecordDelimiter = ","
                        }
                    }
                };
                using (var content = amazonS3Client.SelectObjectContentAsync(selectObjectContentRequest).Result.Payload)
                {
                    foreach (var item in content)
                    {
                        if (item is RecordsEvent recordsEvent)
                        {
                            using (var reader = new StreamReader(recordsEvent.Payload, Encoding.UTF8))
                            {
                                using (var file = new StreamWriter(path, true))
                                {
                                    file.WriteLine(reader.ReadToEnd());
                                }
                            }
                        }
                    }
                }

0 个答案:

没有答案