Question

我正在使用AWS SDK http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/package-summary.html

编写AWS Lambda中的数据处理

遇到错误

    {
  "errorMessage": "Java heap space",
  "errorType": "java.lang.OutOfMemoryError",
  "stackTrace": [
    "java.util.Arrays.copyOf(Arrays.java:3332)",
    "java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:137)",
    "java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:121)",
    "java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:569)",
    "java.lang.StringBuffer.append(StringBuffer.java:369)",
    "java.io.StringWriter.write(StringWriter.java:94)",
    "org.apache.commons.io.IOUtils.copyLarge(IOUtils.java:1129)",
    "org.apache.commons.io.IOUtils.copy(IOUtils.java:1104)",
    "org.apache.commons.io.IOUtils.copy(IOUtils.java:1050)",
    "org.apache.commons.io.IOUtils.copy(IOUtils.java:1075)",
    "org.apache.commons.io.IOUtils.toString(IOUtils.java:382)",
    "example.Hello.myHandler(Hello.java:47)",
    "sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)",
    "sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)",
    "sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)",
    "java.lang.reflect.Method.invoke(Method.java:497)"
  ]
}

我想通过以编程方式增加堆大小来解决这个问题

导致此问题的代码是

AmazonS3 s3Client = new AmazonS3Client();
S3Object s3Object = s3Client.getObject(new GetObjectRequest(srcBucket, srcKey));
InputStream objectData = s3Object.getObjectContent();

// transfer data into JSON
String theString = IOUtils.toString(objectData, null);

我提取的对象大约是300MB。和Lambda的配置

Memory: 1024 MB

Answer 1

最好使用Gson或Jackson API来解析您的Json对象，而不是使用Lambda限制。

类似的东西可以帮助你开始

AmazonS3 s3Client = new AmazonS3Client();
S3Object s3Object = s3Client.getObject(new GetObjectRequest(srcBucket, srcKey));
InputStream objectData = s3Object.getObjectContent();

// transfer data into JSON
JsonFactory jsonfactory = new JsonFactory(); 
JsonParser parser = jsonfactory.createJsonParser(objectData); 

// starting parsing of JSON String 
while (parser.nextToken() != JsonToken.END_OBJECT) { 
    // do what you need to do
}

如何在aws Lambda中增加Java堆大小？

1 个答案: