Java Executor服务并发问题

时间:2014-12-26 01:56:21

标签: java multithreading text-mining

我正在学习Java中的多线程。我正在使用执行程序服务(可调用)因为我需要最终收集我的结果并在进一步之前将它们组合起来。我实现了多线程并抛出错误,这是一个type.cast错误。只是为了让你知道,一旦多线程工作,并且它不起作用。对于每个线程返回类型是TreeMap。而且他们之间没有依赖关系。这是我的实施:

    class AbnerCallable implements Callable<TreeMap<String, ArrayList>>{     
  TreeMap<String,ArrayList> maps;

  AbnerCallable(TreeMap abs){
       maps = abs;
  }

  @Override
  public TreeMap call() throws Exception {
      TreeMap<String,ArrayList> nerRes = new TreeMap<>();
      AbstractTagger nerTagger = new AbstractTagger(maps);
      nerRes = nerTagger.tagAbstracts();
      System.out.println("######################One NER is finished.");
      return nerRes;
    }

  }


public TreeMap NERmultithreading(TreeMap abstracts, int thread) throws InterruptedException, ExecutionException {

    ListOperations breakList = new ListOperations();
    //variable to store sum
    TreeMap<String, ArrayList> abnerResultA = new TreeMap();
    ArrayList<TreeMap<String, ArrayList>> abnerResultM = new ArrayList<>();
    TreeMap<String, ArrayList> abnerResult = new TreeMap<>();
    ExecutorService executor = Executors.newFixedThreadPool(thread);
    List<Callable<TreeMap<String, ArrayList>>> callableList = new ArrayList<>();

    //Send abstract TreeMap for division in TreeMaps
    List<TreeMap> divTreemap = breakList.divideTreemap(abstracts);

    //AbstractTagger nerTagger = new AbstractTagger();
    System.out.println("Size of divTreeMap:"+divTreemap.size());

    for (final TreeMap maps : divTreemap) {
        System.out.println("TreeMap: "+maps.size());
        Object[] mapKey = maps.keySet().toArray();
        Callable<TreeMap<String, ArrayList>> nerCallable = new AbnerCallable(maps);
        callableList.add(nerCallable);
    }

    System.out.println("Size of callable: "+callableList.size());

    //Returns after all tasks complete
    List<Future<TreeMap<String, ArrayList>>> resFuture = executor.invokeAll(callableList);
    System.out.println("Size of future object: "+ resFuture.size());

    //Print results as future objects
    for (Future<TreeMap<String, ArrayList>> future : resFuture) {
        System.out.println("Status of future : " + future.isDone());
        abnerResultA = future.get();
        abnerResultM.add(abnerResultA);
    }
    executor.shutdown();
    abnerResult = breakList.joinMaps(abnerResultM);
    System.out.println("Size of abner result: "+abnerResultM.size());
    System.out.println("This is the abner map: "+abnerResultM);
    return abnerResult;
}

public List divideTreemap(TreeMap abstracts) {        
    Object[] keyList = abstracts.keySet().toArray();
    List<TreeMap> dividedAbsList = new ArrayList<>(); // To hold multiple TreeMaps

    int size = abstracts.size();
    System.out.println("Size:"+size);
    //System.out.println(abstracts);

    int divList = size / 4;
    int lastElement = divList-1;
    int firstElement = 0;
    System.out.println("Check Point:1");

    for (int i = 1; i <= 4; i++) {
        System.out.println("i:"+i);
        if (i < 4) {
            TreeMap<String, Integer> las = new TreeMap<>(abstracts.subMap(keyList[firstElement], true, keyList[lastElement], true));
            dividedAbsList.add((las));
            firstElement = lastElement+1;
            lastElement = lastElement + divList;
        } else {
            TreeMap<String, Integer> las = new TreeMap<>(abstracts.subMap(keyList[firstElement], true, keyList[size-1], true));
            dividedAbsList.add(las);
        }
    }

    return dividedAbsList;
}    

这是parital output的错误示例....

Exception on pipe 27. java.lang.ArrayIndexOutOfBoundsException: 33860864
java.lang.ArrayIndexOutOfBoundsException: 33860864
at gnu.trove.TObjectIntHashMap.rehash(TObjectIntHashMap.java:171)
at gnu.trove.THash.postInsertHook(THash.java:243)
at gnu.trove.TObjectIntHashMap.put(TObjectIntHashMap.java:148)
at edu.umass.cs.mallet.base.types.Alphabet.lookupIndex(Alphabet.java:107)
at edu.umass.cs.mallet.base.types.AugmentableFeatureVector.<init>(AugmentableFeatureVector.java:88)
at edu.umass.cs.mallet.base.types.FeatureVectorSequence.<init>(FeatureVectorSequence.java:39)
at edu.umass.cs.mallet.base.pipe.TokenSequence2FeatureVectorSequence.pipe(TokenSequence2FeatureVectorSequence.java:60)
at edu.umass.cs.mallet.base.pipe.SerialPipes.pipe(SerialPipes.java:141)
at edu.umass.cs.mallet.base.pipe.SerialPipes.pipe(SerialPipes.java:217)
at edu.umass.cs.mallet.base.types.Instance.<init>(Instance.java:105)
at edu.umass.cs.mallet.base.types.InstanceList.add(InstanceList.java:741)
at de.bonn.limes.gcam.abner.MyTagger.doTagging(MyTagger.java:445)
at de.bonn.limes.gcam.abner.MyTagger.getEntities(MyTagger.java:297)
at de.bonn.limes.core.AbnerAnalysis.NERanalysis(AbnerAnalysis.java:91)
at de.bonn.limes.core.AbstractTagger.tagAbstracts(AbstractTagger.java:61)
at de.bonn.limes.core.ListOperations$AbnerCallable.call(ListOperations.java:48)
at de.bonn.limes.core.ListOperations$AbnerCallable.call(ListOperations.java:37)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Loading external tagging module from '/home/peeyush/Documents/netbeans_projects/GCAM-1.0/dependencies/nlpba.crf'...
Loading external tagging module from '/home/peeyush/Documents/netbeans_projects/GCAM-1.0/dependencies/nlpba.crf'...
######################One NER is finished.
######################One NER is finished.
Exception in thread "main" java.util.concurrent.ExecutionException:  java.lang.ClassCastException: edu.umass.cs.mallet.base.types.TokenSequence cannot be cast to edu.umass.cs.mallet.base.types.FeatureVectorSequence
Size of future object: 4
Status of future : true
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:192)
at de.bonn.limes.core.ListOperations.NERmultithreading(ListOperations.java:88)
at de.bonn.limes.core.Main.main(Main.java:208)
Caused by: java.lang.ClassCastException: edu.umass.cs.mallet.base.types.TokenSequence cannot be cast to edu.umass.cs.mallet.base.types.FeatureVectorSequence
at edu.umass.cs.mallet.base.fst.CRF4$State.transitionIterator(CRF4.java:1981)
at edu.umass.cs.mallet.base.fst.Transducer$ViterbiPath.<init>(Transducer.java:1080)
at edu.umass.cs.mallet.base.fst.Transducer.viterbiPath(Transducer.java:994)
at edu.umass.cs.mallet.base.fst.Transducer.viterbiPath(Transducer.java:986)
at de.bonn.limes.gcam.abner.MyTagger.doTagging(MyTagger.java:458)
at de.bonn.limes.gcam.abner.MyTagger.getEntities(MyTagger.java:297)
at de.bonn.limes.core.AbnerAnalysis.NERanalysis(AbnerAnalysis.java:91)
at de.bonn.limes.core.AbstractTagger.tagAbstracts(AbstractTagger.java:61)
at de.bonn.limes.core.ListOperations$AbnerCallable.call(ListOperations.java:48)
at de.bonn.limes.core.ListOperations$AbnerCallable.call(ListOperations.java:37)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)    

当我在单线程上运行时,我想再添加一些这些错误不在那里的东西。如果您需要更多信息,请与我们联系。 谢谢你提前帮忙。

0 个答案:

没有答案