使用BlockingQueue时,Runnable不会停止

时间:2015-11-08 13:13:49

标签: java multithreading queue runnable

我有一个Runnable,即使条件设置为停止也不会停止。不知道为什么会这样。有时一些线程还活着,其他的则没有。我已经让它运行了一个多小时,看它是否会停止,但事实并非如此。它是否与我传递参数的方式有关?

public class ParserWorker implements Runnable {

    private BlockingQueue<String> queue = null;
    private ZipReader zip = null;

    public ParserWorker(BlockingQueue<String> queue, ZipReader zip) {
        this.queue = queue;
        this.zip = zip;
    }

    @Override
    public void run() {
        try {
            while (!queue.isEmpty() || !zip.isClosed()) {
                String line = queue.take();
            }
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }
}

这是开始一切的课程。

public Start() throws InterruptedException   {
    File zipFile = new File("C:\\development\\data2.zip");
    BlockingQueue<String> queue = new ArrayBlockingQueue<String>(1024);

    ZipReader zip = new ZipReader(queue, zipFile);

    ParserWorker w1 = new ParserWorker(queue, zip);
    ParserWorker w2 = new ParserWorker(queue, zip);
    ParserWorker w3 = new ParserWorker(queue, zip);
    ParserWorker w4 = new ParserWorker(queue, zip);

    //Start reading zip file
    Thread zipThread = new Thread(zip);
    zipThread.start();

    //Give a little pause to allow the queue to fill
    Thread.sleep(1000);

    //Starts the Consumable Threads
    Thread t1 = new Thread(w1);
    Thread t2 = new Thread(w2);
    Thread t3 = new Thread(w3);
    Thread t4 = new Thread(w4);

    t1.start();
    t2.start();
    t3.start();
    t4.start();

    //waits until the zip file is closed.
    while (!zip.isClosed()) {
        Thread.sleep(5000);
    }

    //By this point the Zip file is closed, queue may still contain items
    System.out.println("Queue isEmpty:" + queue.isEmpty() + ", Zip isClosed:" + zip.isClosed());

    //Waits until the Queue is empty
    while (!queue.isEmpty()) {
        Thread.sleep(5000);
    }

    //By this point the Zip file is closed and the queue is empty.
    System.out.println("Queue isEmpty:" + queue.isEmpty() + ", Zip isClosed:" + zip.isClosed());

    while (t1.isAlive() || t2.isAlive() || t3.isAlive() || t4.isAlive()) {
        System.out.println("T1 alive:" + t1.isAlive() + ", T2 alive:" + t2.isAlive() + ",T3 alive:" + t3.isAlive() + ",T4 alive:" + t4.isAlive());
        Thread.sleep(5000);
    }


    System.out.println("Done");

我已将ZipReader包含在内以供澄清。

public class ZipReader implements Runnable{

    private boolean closed = true;
    private File zipFileName = null;

    protected BlockingQueue<String> queue = null;

    public ZipReader(BlockingQueue<String> queue, File zipFileName) {
        this.queue = queue; 
        this.zipFileName = zipFileName;
    }

    public boolean isClosed() {
        return closed;
    }

    @Override
    public void run() {
      String line = null;
      long index = 1L;
      ZipFile zipFile = null;

      try {
            System.out.println("Opening Zip file");
            zipFile = new ZipFile(zipFileName);
            closed = false;
            System.out.println("Getting entries");
            Enumeration<? extends ZipEntry> entries = zipFile.entries();
            while (entries.hasMoreElements()) {
                ZipEntry zipEntryObj = entries.nextElement();
                System.out.println("Processing file: " + zipEntryObj.getName());
                InputStream input = zipFile.getInputStream(zipEntryObj);
                InputStreamReader isr = new InputStreamReader(input);

                BufferedReader br = new BufferedReader(isr);
                while ((line = br.readLine()) != null) {
//                    if (index++ % 10000 == 0) {
//                        System.out.println("Index:" + index);
//                    }
//                    System.out.println(line);
                    queue.put(line);
                }
                br.close();
                System.out.println();
            }
        } catch (IOException e) {
            e.printStackTrace();
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            System.out.println("Closing zip file");
            try {
                if (zipFile != null) {
                    zipFile.close();
                }
                closed = true;
            } catch (IOException e) {
                e.printStackTrace();
            }

        }
    }

}

这是我得到的输出。

Opening Zip file
Getting entries
Processing file: data01.txt
Processing file: data02.txt
Processing file: data03.txt
Processing file: data04.txt
Processing file: data05.txt
Processing file: data06.txt
Processing file: data07.txt
Processing file: data08.txt
Processing file: data09.txt
Processing file: data10.txt
Closing zip file
Queue isEmpty:true, Zip isClosed:true
Queue isEmpty:true, Zip isClosed:true
T1 alive:true, T2 alive:true,T3 alive:true,T4 alive:true
T1 alive:true, T2 alive:true,T3 alive:true,T4 alive:true
T1 alive:true, T2 alive:true,T3 alive:true,T4 alive:true
T1 alive:true, T2 alive:true,T3 alive:true,T4 alive:true
T1 alive:true, T2 alive:true,T3 alive:true,T4 alive:true
T1 alive:true, T2 alive:true,T3 alive:true,T4 alive:true
T1 alive:true, T2 alive:true,T3 alive:true,T4 alive:true
T1 alive:true, T2 alive:true,T3 alive:true,T4 alive:true
T1 alive:true, T2 alive:true,T3 alive:true,T4 alive:true
T1 alive:true, T2 alive:true,T3 alive:true,T4 alive:true

1 个答案:

答案 0 :(得分:3)

问题在于while类中的ParserWorker循环。我所指的是:

while (!queue.isEmpty() || !zip.isClosed()) {
  String line = queue.take();
}

使用此代码可以解决并发问题。当调用take BlockingQueue方法时,可能会发生两件事:

  1. 如果队列不为空,则线程将以同步方式获取下一个元素。这意味着没有两个线程可以获得相同的元素。
  2. 如果队列为空,则线程将等待直到放置一个元素。
  3. 您的代码的问题在于您假设String line = queue.take();!queue.isEmpty()同时执行而没有并发问题。但是,以下可能会(并且在您的情况下)发生:

    1. 队列中只有一个元素。
    2. 两个线程同时检查!queue.isEmpty()是否为true。对于两个线程,返回值为queue.take(),因为队列不为空。
    3. 第一个(更快的)线程调用while并获取此单个元素,然后再次检查queue.take()条件并终止,因为队列尚未清空。
    4. 第二个(较慢的)线程在空队列上调用while (t1.isAlive() || t2.isAlive() || t3.isAlive() || t4.isAlive()) { System.out.println("T1 alive:" + t1.isAlive() + ", T2 alive:" + t2.isAlive() + ",T3 alive:" + t3.isAlive() + ",T4 alive:" + t4.isAlive()); Thread.sleep(5000); } ,因为第一个线程已经从队列中获取了该元素。因此,线程会阻塞,直到将新元素放入队列。但是,在某些情况下,这种情况在您的方案中永远不会发生,因为所有行都被读取。这意味着线程永远被阻塞,因为没有其他线程可以为它添加元素。
    5. 有很多解决方案可以解决这个问题。一个简单的就是改变

      while (t1.isAlive() || t2.isAlive() || t3.isAlive() || t4.isAlive()) {
        System.out.println("T1 alive:" + t1.isAlive() + ", T2 alive:" + t2.isAlive() + ",T3 alive:" + t3.isAlive() + ",T4 alive:" + t4.isAlive());
        queue.put("");
        Thread.sleep(5000);
      }
      

      queue.put("");

      使用行while (!queue.isEmpty() || !zip.isClosed()) { synchronized (queue) { if(!queue.isEmpty()) { String line = queue.take(); } } } 确保在所有线程终止之前放置元素。在您的情况下,这将最多执行3次。它不是最合适的解决方案,并使您的线程读取空字符串,但它是最简单的。如果不起作用,您可以花一些时间来提出适合您需求的其他同步。

      另一种解决方案可能使用队列作为锁来使用其他同步:

      os := [arbitrary long path to an artifact]
      platform := [arbitrary long path to a differ artifact]
      packer := [common parts of my packer build command]
      
      .PHONY: all
      all: $(platform)
      
      .PHONY: platform
      platform: $(platform)
      
      $(platform): platform.json  $(os)
          @$(packer) $<
      
      .PHONY: os
      os: $(os)
      
      $(os): os.json
          @$(packer) $<
      
      .PHONY: clean
      clean:
          rm -fr build/
      

      以这种方式确保检查队列是否为空并且在不对每个线程执行的情况下获取元素,以便没有线程将尝试弹出空队列。你必须决定你喜欢哪种方法。第二个需要额外的同步,但不会强制你的线程弹出空字符串。