Question

我想实现这个目标：

在我的应用程序中，我将有一个单独的线程（生产者），它将从用户输入（用户想要抓取的目录）进行一些处理，并将该目录中的所有文件夹名称添加到名为A的linkedBlockingQueue中，并且同时当生产者加入A时，多个消费者线程就说2，将取出A中的文件夹名称。每个消费者线程一次取出一个对象，处理它并放入另一个priorityBlockingQueue命名乙

我现在遇到线程问题，我现在做了什么：

主：

Thread[] workerThreads = new Thread[numOfThreads];

for (int i = 0; i < numOfThreads; i++) {
    /* for each thread to concurrently retrieving directories from the workQueue 
     *  and placing data into another structure(matchQueue)*/
    workerThreads[i] = new Thread(new WorkerThread(workQueue, matchQueue, pattern));
    workerThreads[i].start();
    // System.out.println("Thread "+ i +" Started");
}   

Thread mainThread = new Thread(new ProducerThread(workQueue, args));
mainThread.start();
// System.out.println("Main Thread Started");



// when threads done, join them together
for (int i = 0; i < numOfThreads; i++) {
    try {
        workerThreads[i].join();
        System.out.println("worker thread " + i + "joined");
    }
    catch (InterruptedException e) {
    }
}

我的制作人主题：

public class ProducerThread implements Runnable {
    private LinkedBlockingQueue<String> workQueue;
    private String[] args;

    public ProducerThread(LinkedBlockingQueue<String> workQueue, String[] args) {
        this.workQueue = workQueue;
        this.args = args;
    }

    public void run() {
        // process every argument sequentially
        // for(String arg : args)
        for (int i = 1; i < args.length; i++) {
            processDirectory(args[i]);
             if(i==args.length-1){
          fileCrawler.producerFinished = true;
      }         
}
    }
}

我的工作线程：

public class WorkerThread implements Runnable {

    private LinkedBlockingQueue<String> workQueue;
    private PriorityBlockingQueue<String> matchQueue;
    private String pattern;

    public WorkerThread(LinkedBlockingQueue<String> workQueue, PriorityBlockingQueue<String> matchQueue, String pattern) {
        this.workQueue = workQueue;
        this.matchQueue = matchQueue;
        this.pattern = pattern;
    }

    public void run() {
        do{
    String currentWorkToProcess = "";
    try{
    currentWorkToProcess = workQueue.take();
    System.out.println("Workerthread took from WorkQueue: " + currentWorkToProcess);

    }catch(InterruptedException e1){
        System.out.println(e1.toString());
    }


    File dir = new File(currentWorkToProcess);
    String[] files = dir.list();
    //System.out.println(pattern);
    Pattern p = Pattern.compile(pattern);

    for (String file : files)
    {

        File currentFile = new File(currentWorkToProcess + "/" + file);

        if(currentFile.isFile())
        {   //if current file is a file
            //create a matcher against that line of input
            //System.out.println(file);
            Matcher m = p.matcher(file);

            if (m.matches()){
                System.out.println("File to be added: "+currentFile.getName());
                matchQueue.add(file);
            }
        }
    }
}while(!fileCrawler.producerFinished);

}

}

目前有3个问题，我不知道如何纠正，首先我的序列是错误的，现在我的主线程已经完成所有已完成然后我的工作线程将开始取出。他们不同时工作。其次，我的工作线程完成处理文件夹中的所有文件，它们被连接并结束。右边应该有8个文件。第三，我如何确定我的制作人是否已经结束了他们想要做的事以及如何确定我的工作线程都已完成？

对不起这篇篇幅很长的帖子...对于线程而言太过gd.Very很困惑。

Answer 1

我认为您的排序只是竞争条件 - 您的生产者线程在您的工作线程开始之前已将所有目录名称添加到队列中。如果你想证明这只是在主线程的循环中添加一些睡眠。

您的工作线程没有处理所有文件，因为它们中没有循环。他们只需要一件物品然后完成。

为了监控你的主线程你也可以简单地加入它，或者像这样添加一个等待：

while(mainThread.isAlive()) {
    Thread.sleep(500);
}

我还会考虑使用java.util.concurrent ExecutorService功能，而不是管理自己的工作线程。然后，您可以让主线程为每个目录任务创建一个Callable，并将其传递给执行器服务，这将更加简单。这是一个最小的例子：

public class FileProcessing {

    public static void main(String[] args) throws Exception {
        ExecutorService executorService = Executors.newFixedThreadPool(2);

        Thread mainThread = new Thread(new ProducerThread(executorService));
        mainThread.start();
    }

    public static class ProducerThread implements Runnable {
        private ExecutorService executorService;

        public ProducerThread(ExecutorService executorService) throws InterruptedException {
            this.executorService = executorService;
            Thread.sleep(2000);
        }

        @Override
        public void run() {
            for(File f: new File("c:/").listFiles()) {
                if(f.isDirectory()) {
                    try {
                        final String fileName = f.getCanonicalPath();
                        executorService.submit(new Callable<Object>() {
                            @Override
                            public Object call() throws Exception {
                                System.out.println("Processing: " + fileName);
                                return null;
                            }
                        });
                        System.out.println("Added: " + f.getCanonicalPath());
                    } catch (Exception e) {
                        e.printStackTrace();
                    }
                }
            }
        }
    }
}

如何实现文件扫描程序的这种多线程方案？

1 个答案: