Question

伙计们请帮助，这很重要。

我需要在不到5分钟的时间内下载大约2000张图片。所以我决定下载并行图像。

如果有人可以建议我更好地进行并行下载或告诉我哪里出错了，我真的很感激。

错误日志中已完成任务的数量总是多种多样......

请帮助。

我正在使用Artem Zinnatullin的Async任务执行程序

public class AsyncTaskExecutor {

private static final int CORE_POOL_SIZE;
private static final int MAXIMUM_POOL_SIZE;
private static final int KEEP_ALIVE;
private static final TimeUnit TIME_UNIT;

private static final BlockingQueue<Runnable> concurrentPoolWorkQueue;
private static final ThreadFactory concurrentThreadFactory;
    private static final ThreadPoolExecutor concurrentExecutor;

private AsyncTaskExecutor() {}

static {
 CORE_POOL_SIZE = 5;
 MAXIMUM_POOL_SIZE = 128;
 KEEP_ALIVE = 1;
 TIME_UNIT = TimeUnit.SECONDS;

 concurrentPoolWorkQueue = new LinkedBlockingQueue<Runnable>(10);
 concurrentThreadFactory = new AsyncTaskThreadFactory();
 concurrentExecutor = new ThreadPoolExecutor(CORE_POOL_SIZE,MAXIMUM_POOL_SIZE, KEEP_ALIVE, TIME_UNIT, concurrentPoolWorkQueue, concurrentThreadFactory);
} 
/**
* Concurrently executes AsyncTask on any Android version
* @param task to execute
* @param params for task
* @return executing AsyncTask
*/
@SuppressLint("NewApi")
public static <Params, Progress, Result> AsyncTask<Params, Progress, Result>
 executeConcurrently(AsyncTask<Params, Progress, Result> task,
 Params... params) {
 if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.HONEYCOMB) {
 task.executeOnExecutor(concurrentExecutor, params);
 } else {
 task.execute(params);
 }
 return task;
}

    /**
* Thread factory for AsyncTaskExecutor
* @author Artem Zinnatullin
*
*/
private static class AsyncTaskThreadFactory implements ThreadFactory {
 private final AtomicInteger count;

 {
 count = new AtomicInteger(1);
 }

@Override
public Thread newThread(Runnable r) {
return new Thread(r, "AsyncTask #" + count.getAndIncrement());
}
}

}

然后我使用循环下载我的所有图像。

for (PhotoToDBHelper photoToDBHelper : notDownloaded) {
            try {
                photoToDBHelper.download(m_context);
            } catch (Exception e) {
                // TODO: handle exception
            }

        }

其中下载方法如下所示：

public void download(Context context) throws Exception {
    HttpClient httpClient = new DefaultHttpClient();
    HttpGet httpGet = new HttpGet(new URL(getPhotosUrl() + m_fileName).toString());
    HttpResponse response = httpClient.execute(httpGet);
    HttpEntity entity = response.getEntity();

    InputStream inputStream = entity.getContent();
    filePath = Environment.getExternalStorageDirectory().toString() + getFoldeRelativePath() + m_fileName;

    Utils.createFileOnExternalStorage(getFoldeRelativePath(), filePath);
    FileOutputStream fos = new FileOutputStream(filePath, false);

    byte[] buffer = new byte[20000];
    int byteRead = 0;

    while ((byteRead = inputStream.read(buffer)) != -1) {
        fos.write(buffer, 0, byteRead);
    }
    fos.close();
}

当启动我的Android应用程序时，我收到此错误

02-25 01:08:30.133: E/AndroidRuntime(32357): FATAL EXCEPTION: main
02-25 01:08:30.133: E/AndroidRuntime(32357): java.lang.RuntimeException: Unable to start activity ComponentInfo{"package".TabHostActivity}: java.util.concurrent.RejectedExecutionException: Task android.os.AsyncTask$3@2c0809b0 rejected from java.util.concurrent.ThreadPoolExecutor@2be18f48[Running, pool size = 128, active threads = 1, queued tasks = 10, completed tasks = 27]
02-25 01:08:30.133: E/AndroidRuntime(32357):    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1967)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:1992)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at android.app.ActivityThread.access$600(ActivityThread.java:127)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1158)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at android.os.Handler.dispatchMessage(Handler.java:99)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at android.os.Looper.loop(Looper.java:137)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at android.app.ActivityThread.main(ActivityThread.java:4441)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at java.lang.reflect.Method.invokeNative(Native Method)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at java.lang.reflect.Method.invoke(Method.java:511)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:784)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:551)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at dalvik.system.NativeStart.main(Native Method)
02-25 01:08:30.133: E/AndroidRuntime(32357): Caused by: java.util.concurrent.RejectedExecutionException: Task android.os.AsyncTask$3@2c0809b0 rejected from java.util.concurrent.ThreadPoolExecutor@2be18f48[Running, pool size = 128, active threads = 1, queued tasks = 10, completed tasks = 1032]
02-25 01:08:30.133: E/AndroidRuntime(32357):    at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1967)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:782)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1303)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at android.os.AsyncTask.executeOnExecutor(AsyncTask.java:564)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at "package".data.AsyncTaskExecutor.executeConcurrently(AsyncTaskExecutor.java:58)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at "package".TabHostActivity.checkUpdateState(TabHostActivity.java:130)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at "package".TabHostActivity.onCreate(TabHostActivity.java:88)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at android.app.Activity.performCreate(Activity.java:4465)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1049)
02-25 01:08:30.133: E/AndroidRuntime(32357):    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:1931)
02-25 01:08:30.133: E/AndroidRuntime(32357):    ... 11 more

Answer 1

有一些问题需要解决：

5分钟内的2000张图像大约是每秒7张图像。取决于图像大小，网络带宽，处理器和存储速度等......对于任何代码或并发数量，这可能是不可行的。吞吐量的数学计算是：每秒7 * N字节，其中N是平均图像大小。如果有大图像，那可能就是一个死点。
该异常表示线程池已达到排队任务的最大数量（10）。这可以基于线程池管理器实现来配置。总的来说，尝试同时启动2000个线程可能会加剧这种情况。推荐在组中排队任务（使用线程），监控他们的完成情况，然后再开始更多，所有这些都是小组，例如一次五（5）个。
如果有任何服务器控件，请考虑使用精灵模型，其中多个图像存储在一个图像（文件/资源）中，按图像中的相对x / y坐标进行细分。这将最大限度地减少从2000到1（可能）的HTTP或任何网络请求的数量，从而显着提高性能。

Android并行（同步）图像下载

1 个答案: