Question

我有一个程序，其中我使用提交给ExecutorService的Callable对象从S3下载文件。文件很大，需要几分钟才能完全下载。创建另一个从下载器获取未来并观察完成的Callable类是否有意义？我的最终目标是将所有完整下载添加到缓存中位于中心的列表中。

例如：

public void add(final String s3urlToDownload){

    Future<S3Object> futureS3obj = cachedPoolExecutor.submit(new S3Downloader(s3urlToDownload));

    // Instead of calling futureS3obj.get() and waiting, submit the Future to the "Watcher" service.
    // Within FutureWatcher, the S3Object will be added to the List once the download is complete.
    cachedPoolExecutor.submit(new FutureWatcher(downloadedList, futureS3obj))

}

Answer 1

以下是一些虚假物品，用于说明。 “下载”只是随机睡眠：

// for illustration only
class S3Object {
    String id;
}

// for illustration only
class S3Downloader {

    public S3Object download(String url) {
        int min = 2;
        int max = 5;
        Random rand = new Random();
        int random = rand.nextInt((max - min) + 1) + min;

        try { Thread.sleep(1000 * random); } catch (Exception ex) {}
        S3Object result = new S3Object();
        result.id = url;
        return result;
    }
}

我们可以定义一个下载文件，更新（线程安全）列表和减少CountDownLatch的任务：

class MyTask implements Runnable {
    private final List<S3Object> list;
    private final CountDownLatch latch;
    private final String url; 

    public MyTask(List<S3Object> list, CountDownLatch latch, String url) {
        this.list = list;
        this.latch = latch;
        this.url = url;
    }     

    public void run() {
        S3Downloader downloader = new S3Downloader();
        S3Object result = downloader.download(url);
        list.add(result);
        latch.countDown();
    }
}

示例Runner演示了“客户端”。 go方法是驱动程序，并使用add方法（不阻止）：

public class Runner {
    private ExecutorService pool = Executors.newCachedThreadPool();
    private int numUrls = 20;
    private CountDownLatch latch = new CountDownLatch(numUrls);
    private List<S3Object> results = Collections.synchronizedList(new ArrayList<S3Object>());

    public void add(String url) {
        pool.submit(new MyTask(results, latch, url));
    }

    public void go() throws Exception {

        for(int i = 0; i < numUrls; i++) {
            String url = "http://example" + i;
            add(url);
        }

        // wait for all downloads
        latch.await();

        for (S3Object result : results) {
            System.out.println("result id: " + result.id);
        }
    }
}

生产代码必须处理错误，并可能根据需要重新组织客户端。

Answer 2

而不是制作观察者＆＃39;消耗资源，使当前下载在完成时通知主服务器。

使用ExecutorService

2 个答案: