Question

我正在处理一段代码，在这段代码中我需要为结果集中提取的一些记录调用其余的Web服务，然后在表中更新它们。

我正在设置执行器框架，该框架执行NewWorkerThread（Runnable实例），后者又包含一个固定线程大小为30的线程池实现，并实现ThreadHelper（一个runnable接口的实例），它试图调用Web服务并行并在达到批量大小500时更新数据库中的响应。现在我遇到的问题是没有处理从结果集中提取的所有记录。很少有记录保持未处理状态。我附加了NewWorkerThread和ThreadHelper的实现。我也在使用GlobalAccessor类，它包含可以在多个线程之间共享的volatile字段，而不会出现任何同步问题。请说明为什么所有记录都没有得到处理。

public NewWorkerThread(Connection conn, String query, Map map, String state) {

    this.conn = conn;
    // this.dist=dist;
    // this.seqNo=seqNo;
    this.query = query;

    // this.restTemplate=restTemplate;
    this.state = state;
    this.map = map;
    request = new GetNearestStopsRequest();
    // this.log=log;
    executorPool = (ThreadPoolExecutor) Executors.newFixedThreadPool(30);

}

@Override
public void run() {
    // TODO Auto-generated method stub

    // TODO Auto-generated method stub

    int count = 0;
    int totalCount = 0;
    /* PreparedStatement st1=null; */
    Statement stmt = null;
    ResultSet rs = null;
    ResponseEntity<GetNearestStopsResponse> response = null;
    String transit_id = null;
    int start = 0;

    try {
        /* st1= conn.prepareStatement(Queries.updateRepoTable()); */
        stmt = conn.createStatement();
        rs = stmt.executeQuery(query);
    } catch (SQLException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
    try {
        int mainCount = rs.getFetchSize();
        int counter = 0;

        System.out.println("fetch size is : " + mainCount);
        try {
            Thread.sleep(1000);
        } catch (InterruptedException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }

        while (rs.next()) {
            counter++;
            for (start = 0; start < Constants.INTERNAL_THREAD_COUNT; start++) {
                // list = new ArrayList<Future<String>>();
                String add1 = rs.getString("SRC_ADDRESS_LINE_1");
                String add2 = rs.getString("SRC_ADDRESS_LINE_2");
                String lat = rs.getString("SRC_LAT");
                String longitude = rs.getString("SRC_LONG");
                String city = rs.getString("SRC_PRAD_CITY");
                String state = rs.getString("SRC_PRAD_STATE");
                String zip = rs.getString("SRC_PRAD_ZIP");
                transit_id = rs.getString("TRANSIT_ID");
                String provSource = rs.getString("PROV_SOURCE");
                executorPool.execute(new ThreadHelper(add1, add2, lat, longitude, city, state, zip, transit_id,
                        provSource, map, this.conn, start));

            }
            System.out.println("GlobalAccessor.count[0] : " + GlobalAccessor.count[0]);
            start = 0;

            /*
             * for(Future<String> fut : list){ try {
             * System.out.println(fut.get()); st1.addBatch(fut.get());
             * //System.out.println(fut.get()); System.out.println(
             * "Added in batch"); } catch (InterruptedException |
             * ExecutionException e) { e.printStackTrace(); } }
             */

            for (start = 0; start < Constants.INTERNAL_THREAD_COUNT; start++) {
                if (counter % (Integer.parseInt((String) map.get("BATCHSIZE"))) == 0) {
                    System.out.println("Waiting for tasks to end");
                    while (executorPool.getActiveCount() > 0) {

                    }

                    log.info("Count for " + start + " : " + GlobalAccessor.count[start]);
                    System.out.println("Going to execute !! ");
                    GlobalAccessor.recordCount = GlobalAccessor.recordCount + GlobalAccessor.count[start];
                    GlobalAccessor.count[start] = 0;
                    GlobalAccessor.globalSt.get(start).executeBatch();
                    GlobalAccessor.globalSt.get(start).clearBatch();
                    log.info("One batch completed for " + state);

                }
            }
            start = 0;

        }

        executorPool.shutdown();

        while (!executorPool.isTerminated()) {
        }
        System.out.println("going for final execute");
        for (start = 0; start < Constants.INTERNAL_THREAD_COUNT; start++) {
            GlobalAccessor.globalSt.get(start).executeBatch();
            GlobalAccessor.recordCount = GlobalAccessor.recordCount + GlobalAccessor.count[start];
            GlobalAccessor.count[start] = 0;
        }

        // conn.commit();
        log.info("Successfully executed for the state" + state + " in repo table");

        rs.close();
        stmt.close();

    } catch (SQLException e) {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }

}

Answer 1

我认为你使用的是错误的模式。一定明确要避免使用<CommandLine>powershell -Noninteractive -Command "& {(Get-NetConnectionProfile).Name | Set-NetConnectionProfile -NetCategory Private}" *> c:\windows\temp\4.log</CommandLine>。

问题是关于线程的执行，其结尾不是以正确的方式控制的。

如果要运行多个并行实例，则需要运行它们，然后使用提供的while (executorPool.getActiveCount() > 0) { }对象和方法等待它们结束：您不必实现线程等待，执行程序在代表你。当最后一个线程结束时，处理可以继续。

所以模式应该是这样的：

java.util.concurrent

通过这种方式，＆＃34; invokeAll＆＃34;并行调用所有可调用对象，然后等待它们全部结束。最终，不超过180秒。避免使用大量执行程序，或管理内部fork / join风险的开销非常高。绝对，避免荒谬的public void worker() { ExecutorService exec = Executors.newFixedThreadPool(30); ResultSet rs = findResultsetWithQuery(...) int idx = 0; while (rs.hasNext()) { List<Callable<...>> callables = new ArrayList<>(); while (++idx % 30 > 0 && rs.hasNext()) { callables.add(new ThreadHelper(rs)); } List<Future<...>> result = null; try { result = exec.invokeAll(callables, 180, TimeUnit.SECONDS); catch (Exception e) { ... blah .... } // here you can manage callables' results and exceptions } }

我希望这可以提供帮助。

所有记录都没有在多线程代码中正确更新

1 个答案: