使用lambda使用可笑的内存量将文件从URI直接流式传输到S3

时间:2016-01-18 20:27:11

标签: node.js amazon-sns aws-lambda

我能够直接将文件从URI流式传输到S3,而无需使用this guys sample code使用AWS Lambda(node.js)将其下载到磁盘上。我修改了一下,所以我可以从请求对象体创建一个缓冲区。 This is my code

除了一件非常重要的事情之外,它的确很奇怪!这些文件最大为350 MB,但每次运行lambda函数时,最多需要1.5 GB内存甚至更多!函数调用允许的最大内存为1.536 GB。通常使用的数量介于0.8 GB到1.5 GB之间(非常随机)。

我认为某处存在内存泄漏,但我无法弄清楚在哪里!

有什么想法吗?

1 个答案:

答案 0 :(得分:1)

var buffer = new Buffer(body);似乎正在制作内存中整个响应体的副本......将你需要的内存量翻倍。

request似乎提供(或可配置提供)body Buffer,因此您应该能够直接使用body,而不是将其复制到Option Explicit '// values between cell's Sub PasteValues() Dim Rng1 As Range Dim Rng2 As Range Set Rng1 = Range("A1") Set Rng2 = Range("A2") Rng2.Value = Rng1.Value 'or [A2].Value = [A1].Value 'or Range("A2").Value = Range("A1").Value 'or Set Rng1 = Range("A1:A3") Set Rng2 = Range("A1:A3") Rng2("B1:B3").Value = Rng1("A1:A3").Value 'or [B1:B3].Value = [A1:A3].Value '// values between WorkSheets Dim xlWs1 As Worksheet Dim xlWs2 As Worksheet Set xlWs1 = Worksheets("Sheet1") Set Rng1 = xlWs1.Range("A1") Set xlWs2 = Worksheets("Sheet2") Set Rng2 = xlWs2.Range("A1") Rng2.Value = Rng1.Value 'or Set Rng1 = [=Sheet1!A1] Set Rng2 = [=Sheet2!A1] Rng2.Value = Rng1.Value 'or [=Sheet2!A1].Value = [=Sheet1!A1].Value 'or Worksheets("Sheet2").Range("A2").Value = Worksheets("Sheet1").Range("A1").Value '// values between workbooks Dim xlBk1 As Workbook Dim xlBk2 As Workbook Set xlBk1 = Workbooks("Book1.xlsm") Set Rng1 = xlBk1.Worksheets("Sheet1").Range("A1") Set xlBk2 = Workbooks("Book2.xlsm") Set Rng2 = xlBk2.Worksheets("Sheet1").Range("A1") Rng2.Value = Rng1.Value 'or Set Rng1 = Evaluate("[Book1.xlsm]Sheet1!A1") Set Rng2 = Evaluate("[Book2.xlsm]Sheet2!A1") Rng2.Value = Rng1.Value 'or Evaluate("[Book2.xlsm]Sheet2!A1").Value = Evaluate("[Book1.xlsm]Sheet1!A1") 'or Workbooks("Book2.xlsx").Worksheets("Sheet1").Range("A1").Value = _ Workbooks("Book1.xlsx").Worksheets("Sheet1").Range("A1").Value End Sub 通过创建第二个缓冲区来记忆。