Question

我有一个供客户使用的库，他们正在传递DataRequest对象，其中包含userid，timeout和其他一些字段。现在我使用这个DataRequest对象创建一个URL，然后使用RestTemplate进行HTTP调用，我的服务返回一个JSON响应，我用它来创建一个DataResponse对象并返回这个DataResponse对象回复给他们。

以下是客户通过将DataClient对象传递给它时使用的DataRequest类。我使用DataRequest中的客户传递的超时值，如果在getSyncData方法中花费太多时间，则超时请求。

public class DataClient implements Client {

    private RestTemplate restTemplate = new RestTemplate();
    // first executor
    private ExecutorService service = Executors.newFixedThreadPool(15);

    @Override
    public DataResponse getSyncData(DataRequest key) {
        DataResponse response = null;
        Future<DataResponse> responseFuture = null;

        try {
            responseFuture = getAsyncData(key);
            response = responseFuture.get(key.getTimeout(), key.getTimeoutUnit());
        } catch (TimeoutException ex) {
            response = new DataResponse(DataErrorEnum.CLIENT_TIMEOUT, DataStatusEnum.ERROR);
            responseFuture.cancel(true);
            // logging exception here               
        }

        return response;
    }   

    @Override
    public Future<DataResponse> getAsyncData(DataRequest key) {
        DataFetcherTask task = new DataFetcherTask(key, restTemplate);
        Future<DataResponse> future = service.submit(task);

        return future;
    }
}

DataFetcherTask上课：

public class DataFetcherTask implements Callable<DataResponse> {

    private DataRequest key;
    private RestTemplate restTemplate;

    public DataFetcherTask(DataRequest key, RestTemplate restTemplate) {
        this.key = key;
        this.restTemplate = restTemplate;
    }

    @Override
    public DataResponse call() throws Exception {
        // In a nutshell below is what I am doing here. 
        // 1. Make an url using DataRequest key.
        // 2. And then execute the url RestTemplate.
        // 3. Make a DataResponse object and return it.

        // I am calling this whole logic in call method as LogicA
    }
}

截至目前，我的DataFetcherTask班级负责一个DataRequest密钥，如上所示..

问题陈述： -

现在我有一个小的设计变化。客户将DataRequest（例如keyA）对象传递给我的库，然后我将使用{{1}中的用户ID对另一个服务（我目前的设计中没有这样做）进行新的http调用}（keyA）对象将返回用户ID的列表，因此我将使用这些用户ID并为每个用户创建一个其他DataRequest（keyB，keyC，keyD）对象id在响应中返回。然后我将拥有DataRequest对象，该对象将具有keyB，keyC和keyD List<DataRequest>对象。 DataRequest中的最大元素为3，即全部。

现在，对于List<DataRequest>中的每个DataRequest对象，我希望并行执行上述List<DataRequest>方法，然后通过添加DataFetcherTask.call来List<DataResponse>每把钥匙。所以我将对DataResponse进行三次并行调用。此并行调用背后的想法是在相同的全局超时值中获取所有这三个键的数据。

所以我的提案是 - DataFetcherTask.call课程将返回DataFetcherTask对象而不是List<DataResponse>，然后DataResponse和getSyncData方法的签名也会改变。所以这是算法：

使用客户传递的DataRequest对象通过调用另一个HTTP服务来生成getAsyncData。
对List<DataRequest>到DataRequest方法中的每个List<DataRequest>进行并行调用，并将DataFetcherTask.call对象返回给客户，而不是List<DataResponse>。

通过这种方式，我可以在步骤1和第2步应用相同的全局超时。如果上述任一步骤花费时间，我们将在DataResponse方法中暂停。

设计变更后

getSyncData课程：

DataFetcherTask

现在我的问题是 -

它必须是这样的吗？解决这个问题的正确设计是什么？我的意思是在另一个public class DataFetcherTask implements Callable<List<DataResponse>> { private DataRequest key; private RestTemplate restTemplate; // second executor here private ExecutorService executorService = Executors.newFixedThreadPool(10); public DataFetcherTask(DataRequest key, RestTemplate restTemplate) { this.key = key; this.restTemplate = restTemplate; } @Override public List<DataResponse> call() throws Exception { List<DataRequest> keys = generateKeys(); CompletionService<DataResponse> comp = new ExecutorCompletionService<>(executorService); int count = 0; for (final DataRequest key : keys) { comp.submit(new Callable<DataResponse>() { @Override public DataResponse call() throws Exception { return performDataRequest(key); } }); } List<DataResponse> responseList = new ArrayList<DataResponse>(); while (count-- > 0) { Future<DataResponse> future = comp.take(); responseList.add(future.get()); } return responseList; } // In this method I am making a HTTP call to another service // and then I will make List<DataRequest> accordingly. private List<DataRequest> generateKeys() { List<DataRequest> keys = new ArrayList<>(); // use key object which is passed in contructor to make HTTP call to another service // and then make List of DataRequest object and return keys. return keys; } private DataResponse performDataRequest(DataRequest key) { // This will have all LogicA code here which is shown in my original design. // everything as it is same.. } }方法中使用call方法看起来很奇怪？
我们需要在代码中拥有两个执行器吗？有没有更好的方法来解决这个问题或我们可以在这里做的任何简化/设计变更？

我已经简化了代码，以便让想法明确我想要做的事情。

Answer 1

据我所知，RestTemplate正在阻塞，在ForkJoinTask中的ForkJoinPool JavaDoc中说：

计算应该避免同步的方法或块，并且应该最小化其他阻塞同步，除了加入其他任务或使用同步器，如Phasers，通告与fork / join调度协作。 ...
任务也不应该执行阻塞IO，...

通话中的电话是多余的。
你不需要两个遗嘱执行人。您也可以在getSyncData(DataRequest key)中返回部分结果。这可以这样做

的 DataClient.java

public class DataClient implements Client { private RestTemplate restTemplate = new RestTemplate(); // first executor private ExecutorService service = Executors.newFixedThreadPool(15); @Override public List<DataResponse> getSyncData(DataRequest key) { List<DataResponse> responseList = null; DataFetcherResult response = null; try { response = getAsyncData(key); responseList = response.get(key.getTimeout(), key.getTimeoutUnit()); } catch (TimeoutException ex) { response.cancel(true); responseList = response.getPartialResult(); } return responseList; } @Override public DataFetcherResult getAsyncData(DataRequest key) { List<DataRequest> keys = generateKeys(key); final List<Future<DataResponse>> responseList = new ArrayList<>(); final CountDownLatch latch = new CountDownLatch(keys.size());//assume keys is not null for (final DataRequest _key : keys) { responseList.add(service.submit(new Callable<DataResponse>() { @Override public DataResponse call() throws Exception { DataResponse response = null; try { response = performDataRequest(_key); } finally { latch.countDown(); return response; } } })); } return new DataFetcherResult(responseList, latch); } // In this method I am making a HTTP call to another service // and then I will make List<DataRequest> accordingly. private List<DataRequest> generateKeys(DataRequest key) { List<DataRequest> keys = new ArrayList<>(); // use key object which is passed in contructor to make HTTP call to another service // and then make List of DataRequest object and return keys. return keys; } private DataResponse performDataRequest(DataRequest key) { // This will have all LogicA code here which is shown in my original design. // everything as it is same.. return null; } }

<强> DataFetcherResult.java

public class DataFetcherResult implements Future<List<DataResponse>> { final List<Future<DataResponse>> futures; final CountDownLatch latch; public DataFetcherResult(List<Future<DataResponse>> futures, CountDownLatch latch) { this.futures = futures; this.latch = latch; } //non-blocking public List<DataResponse> getPartialResult() { List<DataResponse> result = new ArrayList<>(futures.size()); for (Future<DataResponse> future : futures) { try { result.add(future.isDone() ? future.get() : null); //instead of null you can return new DataResponse(DataErrorEnum.NOT_READY, DataStatusEnum.ERROR); } catch (InterruptedException | ExecutionException e) { e.printStackTrace(); //ExecutionException or CancellationException could be thrown, especially if DataFetcherResult was cancelled //you can handle them here and return DataResponse with corresponding DataErrorEnum and DataStatusEnum } } return result; } @Override public List<DataResponse> get() throws ExecutionException, InterruptedException { List<DataResponse> result = new ArrayList<>(futures.size()); for (Future<DataResponse> future : futures) { result.add(future.get()); } return result; } @Override public List<DataResponse> get(long timeout, TimeUnit timeUnit) throws ExecutionException, InterruptedException, TimeoutException { if (latch.await(timeout, timeUnit)) { return get(); } throw new TimeoutException();//or getPartialResult() } @Override public boolean cancel(boolean mayInterruptIfRunning) { boolean cancelled = true; for (Future<DataResponse> future : futures) { cancelled &= future.cancel(mayInterruptIfRunning); } return cancelled; } @Override public boolean isCancelled() { boolean cancelled = true; for (Future<DataResponse> future : futures) { cancelled &= future.isCancelled(); } return cancelled; } @Override public boolean isDone() { boolean done = true; for (Future<DataResponse> future : futures) { done &= future.isDone(); } return done; } //and etc. }

我用CountDownLatch写了它，它看起来很棒，但请注意有一个细微差别。您可能会在DataFetcherResult.get(long timeout, TimeUnit timeUnit)中陷入困境一段时间，因为CountDownLatch与未来的状态不同步。可能会发生latch.getCount() == 0但并非所有期货都会同时返回future.isDone() == true。因为他们已经在latch.countDown(); Callable的块内传递了finally {}，但没有更改仍然等于state的内部NEW。
因此在get()内拨打get(long timeout, TimeUnit timeUnit)可能会造成一点延迟。
类似的案例是described here。

获取超时DataFetcherResult.get(...)可以使用期货future.get(long timeout, TimeUnit timeUnit)重写，您可以从班级中删除CountDownLatch。

public List<DataResponse> get(long timeout, TimeUnit timeUnit) throws ExecutionException, InterruptedException{ List<DataResponse> result = new ArrayList<>(futures.size()); long timeoutMs = timeUnit.toMillis(timeout); boolean timeout = false; for (Future<DataResponse> future : futures) { long beforeGet = System.currentTimeMillis(); try { if (!timeout && timeoutMs > 0) { result.add(future.get(timeoutMs, TimeUnit.MILLISECONDS)); timeoutMs -= System.currentTimeMillis() - beforeGet; } else { if (future.isDone()) { result.add(future.get()); } else { //result.add(new DataResponse(DataErrorEnum.NOT_READY, DataStatusEnum.ERROR)); ? } } } catch (TimeoutException e) { result.add(new DataResponse(DataErrorEnum.TIMEOUT, DataStatusEnum.ERROR)); timeout = true; } //you can also handle ExecutionException or CancellationException here } return result; }

这段代码是作为一个例子给出的，它应该在生产中使用之前进行测试，但似乎合法：）

从调用方法

1 个答案: