在我的应用程序中,我在一天结束时将一些数据同步到app服务器。为此,我将所有数据包装为JSONObject的JSONArray。数据主要包括大约50张图片,每张图片大小约为50kb(以及一些文本数据)。所有这些图片都使用base64编码进行编码。当上传的图片(以及一些文本数据)数量很少时,可以正常工作,但是当我上传大量没有图片时,说50左右然后我看到所有数据都正确形成JSONArray的日志,但是当我尝试使用'array.toString()'方法显示JSONArray时,我遇到内存不足异常。我认为是由于堆已满(但是,当我尝试在清单中使用android:largeHeap =“true”时一切正常,但是我想避免使用这种方法,因为这不是一个好的做法。)我的目的只是将这个JSONArray值写入文件然后将此文件分成小块并将其发送到服务器。 请指导我将JSONAray值写入文件的最佳方法,这不会导致OOM问题。谢谢!
以下是JSONArray的格式:
[{"pid":"000027058451111","popup_time":"2014-01-13 23:36:01","picture":"...base64encoded string......","punching_time":"Absent","status":"Absent"},{"pid":"000027058451111","popup_time":"2014-01-13 23:36:21","picture":"...base64encoded string......","punching_time":"Absent","status":"Absent"}]
以下是我的代码的主要摘要:
JSONObject aux;
JSONArray array = new JSONArray();
.
.
// Looping through each record in the cursor
for (int i = 0; i < count; i++) {
aux = new JSONObject();
try {
aux.put("pid", c.getString(c.getColumnIndex("pid")));
aux.put("status", c.getString(c.getColumnIndex("status")));
aux.put("pop_time", c.getString(c.getColumnIndex("pop_time")));
aux.put("punching_time", c.getString(c.getColumnIndex("punching_time")));
aux.put("picture", c.getString(c.getColumnIndex("image_str"))); // stores base64encoded picture
} catch (Exception e) {
e.printStackTrace();
}
array.put(aux); // Inserting individual objects into the array , works perfectly fine,no error here
c.moveToNext(); // Moving the cursor to the next record
}
Log.d("Log", "length of json array - "+array.length()); // shows me the total no of JSONObjects in the JSONArray,works fine no error
// HAD GOT OOM HERE
//Log.d("Log", "JSONArray is - " + array.toString());
if (array.length() != 0){
try {
String responseCode = writeToFile(array); //Writing the JSONArray value to file,which will then send file to server.
if(responseCode.equals("200"))
Log.d("Log","Data sent successfully from app to app server");
else
Log.d("Log","Data NOT sent successfully from app to app server");
} catch (Exception e) {
e.printStackTrace();
}
}
.
.
private String writeToFile(JSONArray data) {
Log.d("Log", "Inside writeToFile");
File externalStorageDir = new File(Environment.getExternalStorageDirectory().getPath(), "Pictures/File");
if (!externalStorageDir.exists()) {
externalStorageDir.mkdirs();
}
String responseCode = "";
File dataFile = new File(externalStorageDir, "File");
/* FileWriter writer;
String responseCode = "";
try {
writer = new FileWriter(dataFile);
writer.append(data);
writer.flush();
writer.close();
responseCode = sendFileToServer(dataFile.getPath(), AppConstants.url_app_server); // Sends the file to server,worked fine for few pictures
} catch (IOException e) {
e.printStackTrace();
}*/
try {
FileWriter file = new FileWriter("storage/sdcard0/Pictures/File/File");
file.write(data.toString()); // GOT OOM here.
file.flush();
file.close();
Log.d("Log","data written from JSONArray to file");
responseCode = sendFileToServer(dataFile.getPath(), AppConstants.url_app_server); // Sends the file to server,worked fine for few pictures
} catch (IOException e) {
e.printStackTrace();
}
return responseCode;
}
public String sendFileToServer(String filename, String targetUrl) {
.
.
// Sends the file to server,worked fine for few pictures
.
.
return response;
}
答案 0 :(得分:3)
这是问题所在。您正在尝试将整个数据集加载到内存中。你的内存已经不足了。
Android的JSON类(以及一些其他JSON库)旨在获取Java对象(在内存中),将其序列化为对象的解析树(例如JSONObject
,JSONArray
)(在内存中) ),然后将该树转换为String
(在内存中)并将其写出来。
特别是在你的情况下(目前),当它将解析树转换为String
时,它会出现内存不足的情况;那个String
实际上使该点所需的内存量翻了一番。
要解决您的问题,您有几个不同的选择,我会提供3:
根本不要使用JSON。重构只需将文件和信息发送到您的服务器。
重构事物,以便您一次只能将X图像读入内存并拥有多个输出文件。其中X是一些图像。请注意,如果图像尺寸变化很大/无法预测,这仍然存在问题。
切换到使用Jackson作为JSON库。它支持流操作,您可以在创建数组中的每个对象时将JSON流式传输到输出文件。
编辑添加:代码,使用杰克逊看起来像这样:
// Before you get here, have created your `File` object
JsonFactory jsonfactory = new JsonFactory();
JsonGenerator jsonGenerator =
jsonfactory.createJsonGenerator(file, JsonEncoding.UTF8);
jsonGenerator.writeStartArray();
// Note: I don't know what `c` is, but if it's a cursor of some sort it
// should have a "hasNext()" or similar you should be using instead of
// this for loop
for (int i = 0; i < count; i++) {
jsonGenerator.writeStartObject();
jsonGenerator.writeStringField("pid", c.getString(c.getColumnIndex("pid")));
jsonGenerator.writeStringField("status", c.getString(c.getColumnIndex("status")));
jsonGenerator.writeStringField("pop_time", c.getString(c.getColumnIndex("pop_time")));
jsonGenerator.writeStringField("punching_time", c.getString(c.getColumnIndex("punching_time")));
// stores base64encoded picture
jsonGenerator.writeStringField("picture", c.getString(c.getColumnIndex("image_str")));
jsonGenerator.writeEndObject();
c.moveToNext(); // Moving the cursor to the next record
}
jsonGenerator.writeEndArray();
jsonGenerator.close();
以上是未经测试的,但它应该有用(或至少让你朝着正确的方向前进)。
答案 1 :(得分:1)
首先,感谢Brian Roach为我提供帮助。他的投入帮我解决了问题。我正在分享我的答案。
我想解决什么? - 在我的项目中,我有一些用户数据(名称,年龄,picture_time)和每个用户数据的相应图片。在EOD我需要将所有这些数据同步到应用服务器。但是当我尝试同步很多时图片(约50kb约50)我面临一个OOM(内存不足)问题。最初,我试图使用传统的JSONArray方法上传所有数据,但很快我发现我正在打OOM.This,我归因于当我试图访问JSONArray(它有大量的值以及为什么不是?)时,堆已经满了,毕竟我是通过base64encoding对pics进行编码,它相信我有很多字符串数据!)
Brian的输入建议我将所有数据逐个写入文件。所以,在整个过程完成后,我得到一个包含所有数据(name,age,picture_time,base64encoded pictures等)的文件。它,然后我将此文件传输到服务器。
以下是从app数据库中获取用户数据的代码片段,来自sd卡的相应图片,遍历所有记录,使用Jackson Json Library创建JSONArray的JSONArray(您需要将其包含在libs文件夹中,您使用此代码)并将它们存储到一个文件中。然后将此文件流式传输到服务器(此片段不包括在内)。希望这可以帮助某人!
// Sync the values in DB to the server
Log.d("SyncData", "Opening db to read files");
SQLiteDatabase db = context.openOrCreateDatabase("data_monitor", Context.MODE_PRIVATE, null);
db.execSQL("CREATE TABLE IF NOT EXISTS user_data(device_id VARCHAR,name VARCHAR,age VARCHAR,picture_time VARCHAR);");
Cursor c = db.rawQuery("SELECT * FROM user_data", null);
int count = c.getCount();
if (count > 0) {
File file = new File(Environment.getExternalStorageDirectory().getPath(), "Pictures/UserFile/UserFile");
JsonFactory jsonfactory = new JsonFactory();
JsonGenerator jsonGenerator = null;
try {
jsonGenerator = jsonfactory.createJsonGenerator(file, JsonEncoding.UTF8);
jsonGenerator.writeStartObject();
jsonGenerator.writeArrayFieldStart("user_data"); //Name for the JSONArray
} catch (IOException e3) {
e3.printStackTrace();
}
c.moveToFirst();
// Looping through each record in the cursor
for (int i = 0; i < count; i++) {
try {
jsonGenerator.writeStartObject(); //Start of inner object '{'
jsonGenerator.writeStringField("device_id", c.getString(c.getColumnIndex("device_id")));
jsonGenerator.writeStringField("name", c.getString(c.getColumnIndex("name")));
jsonGenerator.writeStringField("age", c.getString(c.getColumnIndex("age")));
jsonGenerator.writeStringField("picture_time", c.getString(c.getColumnIndex("picture_time")));
} catch (Exception e) {
e.printStackTrace();
}
// creating a fourth column for the input of corresponding image from the sd card
Log.d("SyncData", "Name of image - " + c.getString(c.getColumnIndex("picture_time")));
image = c.getString(c.getColumnIndex("picture_time")).replaceAll("[^\\d]", ""); //Removing everything except digits
Log.d("SyncData", "imagename - " + image);
File f = new File(Environment.getExternalStorageDirectory().getPath(), "Pictures/UserPic/" + image + ".jpg");
Log.d("SyncData", "------------size of " + image + ".jpg" + "= " + f.length());
String image_str;
if (!f.exists() || f.length() == 0) {
Log.d("SyncData", "Image has either size of 0 or does not exist");
try {
jsonGenerator.writeStringField("picture", "Error Loading Image");
} catch (Exception e) {
e.printStackTrace();
}
} else {
try {
// Reusing bitmaps to avoid Out Of Memory
Log.d("SyncData", "Image exists,encoding underway...");
if (bitmap_reuse == 0) { //ps : bitmap reuse was initialized to 0 at the start of the code,not included in this snippet
// Create bitmap to be re-used, based on the size of one of the bitmaps
mBitmapOptions = new BitmapFactory.Options();
mBitmapOptions.inJustDecodeBounds = true;
BitmapFactory.decodeFile(f.getPath(), mBitmapOptions);
mCurrentBitmap = Bitmap.createBitmap(mBitmapOptions.outWidth, mBitmapOptions.outHeight, Bitmap.Config.ARGB_8888);
mBitmapOptions.inJustDecodeBounds = false;
mBitmapOptions.inBitmap = mCurrentBitmap;
mBitmapOptions.inSampleSize = 1;
BitmapFactory.decodeFile(f.getPath(), mBitmapOptions);
bitmap_reuse = 1;
}
BitmapFactory.Options bitmapOptions = null;
// Re-use the bitmap by using BitmapOptions.inBitmap
bitmapOptions = mBitmapOptions;
bitmapOptions.inBitmap = mCurrentBitmap;
mCurrentBitmap = BitmapFactory.decodeFile(f.getPath(), mBitmapOptions);
if (mCurrentBitmap != null) {
ByteArrayOutputStream stream = new ByteArrayOutputStream();
try {
mCurrentBitmap.compress(Bitmap.CompressFormat.JPEG, 35, stream);
Log.d("SyncData", "------------size of " + "bitmap_compress" + "= " + mCurrentBitmap.getByteCount());
} catch (Exception e) {
e.printStackTrace();
}
byte[] byte_arr = stream.toByteArray();
Log.d("SyncData", "------------size of " + "image_str" + "= " + byte_arr.length);
stream.close();
stream = null;
image_str = Base64.encodeToString(byte_arr, Base64.DEFAULT);
jsonGenerator.writeStringField("picture", image_str);
}
} catch (Exception e1) {
e1.printStackTrace();
}
}
try {
jsonGenerator.writeEndObject(); //End of inner object '}'
} catch (JsonGenerationException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
c.moveToNext(); // Moving the cursor to the next record
}
try {
jsonGenerator.writeEndArray(); //close the array ']'
//jsonGenerator.writeStringField("file_size", "0"); // If need be, place another object here.
jsonGenerator.writeEndObject();
jsonGenerator.flush();
jsonGenerator.close();
} catch (JsonGenerationException e1) {
e1.printStackTrace();
} catch (IOException e1) {
e1.printStackTrace();
}
c.close();
db.close();
}