Question

在我的应用程序中有两个阶段，一个下载一些大数据，另一个操作它。所以我创建了两个实现runnable的类：ImageDownloader和ImageManipulator，它们共享一个downloadedBlockingQueue：

        public class ImageDownloader implements Runnable {

        private ArrayBlockingQueue<ImageBean> downloadedImagesBlockingQueue;
        private ArrayBlockingQueue<String> imgUrlsBlockingQueue;

        public ImageDownloader(ArrayBlockingQueue<String> imgUrlsBlockingQueue, ArrayBlockingQueue<ImageBean> downloadedImagesBlockingQueue) {

            this.downloadedImagesBlockingQueue = downloadedImagesBlockingQueue;
            this.imgUrlsBlockingQueue = imgUrlsBlockingQueue;

        }

        @Override
        public void run() {
            while (!this.imgUrlsBlockingQueue.isEmpty()) {
                try {
                    String imgUrl = this.imgUrlsBlockingQueue.take();
                    ImageBean imageBean = doYourThing(imgUrl);
                    this.downloadedImagesBlockingQueue.add(imageBean);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }
    }

        public class ImageManipulator implements Runnable {

        private ArrayBlockingQueue<ImageBean> downloadedImagesBlockingQueue;
        private AtomicInteger capacity;

        public ImageManipulator(ArrayBlockingQueue<ImageBean> downloadedImagesBlockingQueue,
                                AtomicInteger capacity) {
            this.downloadedImagesBlockingQueue = downloadedImagesBlockingQueue;
            this.capacity = capacity;
        }

        @Override
        public void run() {
            while (capacity.get() > 0) {
                try {
                    ImageBean imageBean = downloadedImagesBlockingQueue.take(); // <- HERE I GET THE DEADLOCK
                    capacity.decrementAndGet();

                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
                // ....
            }
        }
    }




    public class Main {
        public static void main(String[] args) {
            String[] imageUrls = new String[]{"url1", "url2"};
            int capacity = imageUrls.length;

            ArrayBlockingQueue<String> imgUrlsBlockingQueue = initImgUrlsBlockingQueue(imageUrls, capacity);
            ArrayBlockingQueue<ImageBean> downloadedImagesBlockingQueue = new ArrayBlockingQueue<>(capacity);

            ExecutorService downloaderExecutor = Executors.newFixedThreadPool(3);
            for (int i = 0; i < 3; i++) {
                Runnable worker = new ImageDownloader(imgUrlsBlockingQueue, downloadedImagesBlockingQueue);
                downloaderExecutor.execute(worker);
            }
            downloaderExecutor.shutdown();

            ExecutorService manipulatorExecutor = Executors.newFixedThreadPool(3);
            AtomicInteger manipulatorCapacity = new AtomicInteger(capacity);

            for (int i = 0; i < 3; i++) {
                Runnable worker = new ImageManipulator(downloadedImagesBlockingQueue, manipulatorCapacity);
                manipulatorExecutor.execute(worker);
            }
            manipulatorExecutor.shutdown();
            while (!downloaderExecutor.isTerminated() && !manipulatorExecutor.isTerminated()) {
            }
        }
    }

发生死锁是因为这种情况： t1检查容量为1。

t2检查其1。

t3检查其1。

t2，将容量设置为0，继续流并最终退出。 t1和t3现在处于死锁状态，导致下载的ImagesBlockingQueue没有添加。

最终我想要这样的东西：达到容量时＆amp;＆amp;队列是空的=打破＆＃34;而＃34;循环，并优雅地终止。

设置＆＃34;队列为空＆＃34;因为只有条件不起作用，所以在开始时它是空的，直到一些ImageDownloader将一个imageBean放入队列。

Answer 1

您可以采取一些措施来防止死锁：

使用容量<{li>的LinkedBlockingQueue
使用offer添加到不阻止的队列
使用drainTo或poll从队列中取出未阻止的项目

您可能还需要考虑一些提示：

使用ThreadPool：
final ExecutorService executorService = Executors.newFixedThreadPool(4);
如果使用固定大小ThreadPool，则可以在完成将数据添加到与ThreadPool的大小相对应的队列后添加"poison pill"，并在{{1}时进行检查}}

使用poll就像这样简单：

ThreadPool

还有鲜为人知的final ExecutorService executorService = Executors.newFixedThreadPool(4); final Future<?> result = executorService.submit(new Runnable() { @Override public void run() { } });抽象整个过程。更多信息here。

Answer 2

您的消费者不需要capacity。它现在在多个线程中读取和更新，这会导致同步问题。

initImgUrlsBlockingQueue使用capacity个网址项创建网址阻止队列。（右？）
ImageDownloader使用imgUrlsBlockingQueue并生成图片，它会在下载所有网址时终止，或者，如果capacity表示应该下载的图片数量，因为可能会出现故障，它在添加capacity个图像时终止。
在ImageDownloader终止之前，它会在downloadedImagesBlockingQueue中添加一个标记，例如~~一个空元素~~，一个静态的最终ImageBean static final ImageBean marker = new ImageBean()。

所有ImageManipulator排除队列使用以下构造，当它看到null元素时，它再次将其添加到队列并终止。

// use identity comparison
while ((imageBean = downloadedImagesBlockingQueue.take()) != marker) {
   // process image
}
downloadedImagesBlockingQueue.add(marker);

请注意，BlockingQueue承诺其方法将其称为原子，但是，如果先检查容量，并根据容量使用元素，则操作组将不是原子的。

Answer 3

我使用了一些建议的功能，但这对我来说是一个完整的解决方案，一个不忙等待并等到Downloader通知它的那个。

public ImageManipulator(LinkedBlockingQueue<ImageBean> downloadedImagesBlockingQueue,
                        LinkedBlockingQueue<ImageBean> manipulatedImagesBlockingQueue,
                        AtomicInteger capacity,
                        ManipulatedData manipulatedData,
                        ReentrantLock downloaderReentrantLock,
                        ReentrantLock manipulatorReentrantLock,
                        Condition downloaderNotFull,
                        Condition manipulatorNotFull) {

    this.downloadedImagesBlockingQueue = downloadedImagesBlockingQueue;
    this.manipulatedImagesBlockingQueue = manipulatedImagesBlockingQueue;
    this.capacity = capacity;
    this.downloaderReentrantLock = downloaderReentrantLock;
    this.manipulatorReentrantLock = manipulatorReentrantLock;
    this.downloaderNotFull = downloaderNotFull;
    this.manipulatorNotFull = manipulatorNotFull;
    this.manipulatedData = manipulatedData;
}

@Override
public void run() {
    while (capacity.get() > 0) {
        downloaderReentrantLock.lock();
        if (capacity.get() > 0) { //checks if the value is updated.

            ImageBean imageBean = downloadedImagesBlockingQueue.poll();

            if (imageBean != null) { // will be null if no downloader finished is work (successfully downloaded or not)

                capacity.decrementAndGet();
                if (capacity.get() == 0) { //signal all the manipulators to wake up and stop waiting for downloaded images.
                    downloaderNotFull.signalAll();
                }
                downloaderReentrantLock.unlock();

                if (imageBean.getOriginalImage() != null) { // the downloader will set it null iff it failes to download it.

                     // business logic
                }

                manipulatedImagesBlockingQueue.add(imageBean);

                signalAllPersisters(); // signal the persisters (which has the same lock/unlock as this manipulator.

            } else {
                try {
                    downloaderNotFull.await(); //manipulator will wait for downloaded image - downloader will signalAllManipulators (same as signalAllPersisters() here) when an imageBean will be inserted to queue.
                    downloaderReentrantLock.unlock();
                } catch (InterruptedException e) {
                    logger.log(Level.ERROR, e.getMessage(), e);
                }
            }
        }
    }

    logger.log(Level.INFO, "Manipulator: " + Thread.currentThread().getId() + "  Ended Gracefully");
}

private void signalAllPersisters() {
    manipulatorReentrantLock.lock();
    manipulatorNotFull.signalAll();
    manipulatorReentrantLock.unlock();
}

要获得完整流程，您可以在我的github上查看此项目：https://github.com/roy-key/image-service/

Answer 4

您的问题是您正在尝试使用计数器来跟踪队列元素，并且不会编写需要原子的操作。你在做检查，接受，减少。这允许队列大小和计数器去同步，并且您的线程永远阻塞。写一个可以关闭的同步原语会更好。这样你就不必保留一个相关的柜台。但是，快速解决方法是更改它，以便原子地获取和减少计数器：

while (capacity.getAndDecrement() > 0) {
    try {
        ImageBean imageBean = downloadedImagesBlockingQueue.take();

    } catch (InterruptedException e) {
        e.printStackTrace();
    }
}

在这种情况下，如果队列中只有3个线程且只剩下一个元素，那么只有一个线程将原子地递减计数器并看到它可以不带阻塞。其他两个线程都会看到0或＆lt; 0并且跳出循环。

您还需要将所有类实例变量设置为final，以便它们具有正确的内存可见性。您还应该确定如何处理中断而不是依赖于默认的打印跟踪模板。

Java生产者消费者ArrayBlockingQueue对take（）的死锁

4 个答案: