Question

我一直在尝试将一些代码放在一起 - 除其他外 - 将文件上传到使用NTLM身份验证的Sharepoint站点。早期版本的代码是单线程的，并且工作得很好。他们完全按预期上传文件，没有丝毫问题。但是，我最终尝试多线程化这个应用程序，这样它就可以同时上传许多文件，同时还可以完成其余的业务。

但是当我尝试多线程代码时，它每次都会失败，抛出一个IndexOutOfBoundsException。在诊断问题的实际原因时，这对我来说是无益的。

如果您想知道，如果我更改CachedThreadExecutor SingleThreadExecutor - 强制代码进入单线程状态 - 它再次正常工作。

创建执行程序和连接管理器，并构造线程：

class OrderProcessor implements Runnable {

    //Other variables for object

    private final ExecutorService executorService = Executors
            .newCachedThreadPool();
    //      .newSingleThreadExecutor();

    private HttpClientConnectionManager conManager;

    private void setup() {
        //always called before execution of anything else in object
        conManager = new PoolingHttpClientConnectionManager();
    }
    //lots of other code
}

提交线程的实际代码很复杂，所以这个版本有点简化，但是可以解决这个问题。

for(Request request : requests){
    //Do other stuff
    simpleSubmitFile(request);
    //Do other stuff
}

以下是简化文件提交方法

public Future<Boolean> simpleSubmitFile(Request request){

        transferer = new SharePointTransferer(extractionRequest, conManager);

    Future<Boolean> future = executorService.submit(transferer);
    return future;
}

SharePointTransferer代码

//actual values scrubbed
private final String USERNAME = "";
private final String PASSWORD = "";
private final String DOMAIN = "";

private final File sourceFile;
private final String destinationAddress;
private final CloseableHttpClient client;


public SharePointTransferer(final Request extractionRequest, HttpClientConnectionManager conManager) {
    super(extractionRequest);
    this.sourceFile = this.extractionRequest.getFile();
    this.destinationAddress = this.extractionRequest.getDestinationAddress();
    this.client = HttpClients.custom()
            .setConnectionManager(conManager).build();
}

public Boolean call() throws Exception {

    String httpAddress = correctSharePointAddress(destinationAddress);
    HttpPut put = new HttpPut(httpAddress + sourceFile.getName());

    // construct basic request
    put.setEntity(new FileEntity(sourceFile));
    HttpClientContext context = HttpClientContext.create();

    // set credentials for the SharePoint login
    CredentialsProvider credProvider = new BasicCredentialsProvider();
    credProvider.setCredentials(AuthScope.ANY, new NTCredentials(USERNAME,
            PASSWORD, "", DOMAIN));
    context.setCredentialsProvider(credProvider); 
    // execute request
    try {
        HttpResponse response = client.execute(put, context);
        logger.info("response code was: "
                + response.getStatusLine().getStatusCode());
        if (response.getStatusLine().getStatusCode() != 201) {
            throw new FileTransferException(
                    "Could not upload file. Http response code 201 expected."
                            + "\nActual status code: "
                            + response.getStatusLine().getStatusCode());
        }
    } catch (ClientProtocolException e) {
        throw new FileTransferException(
                "Exception Occurred while Transferring file "
                        + sourceFile.getName(), e);
    } catch (IOException e) {
        throw new FileTransferException(
                "Exception Occurred while Transferring file "
                        + sourceFile.getName(), e);
    }finally{
        logger.info("deleting source file: " + sourceFile.getName());
        sourceFile.delete();
        client.close();
    }

    logger.info("successfully transfered file: "+sourceFile.getName());
    return true;
}

如果我提交多个文件，它会为所有文件抛出基本相同的异常。跟踪如下异常堆栈跟踪

2015-04-16 11:49:26 ERROR OrderProcessor:224 - error processing file: FILE_NAME_SCRUBBED
PACKAGE_SCRUBBED.FileProcessingException: Could not process file: FILE_NAME_SCRUBBED
        at PACKAGE_SCRUBBED.OrderProcessor.finishProcessingOrder(OrderProcessor.java:223)
        at PACKAGE_SCRUBBED.OrderProcessor.run(OrderProcessor.java:124)
        at PACKAGE_SCRUBBED.FileTransferDaemon.process(FileTransferDaemon.java:48)
        at PACKAGE_SCRUBBED.FileTransferDaemon.start(FileTransferDaemon.java:83)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:243)
Caused by: java.util.concurrent.ExecutionException: java.lang.ArrayIndexOutOfBoundsException: 41
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
        at PACKAGE_SCRUBBED.OrderProcessor.finishProcessingOrder(OrderProcessor.java:208)
        ... 8 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 41
        at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addByte(NTLMEngineImpl.java:924)
        at org.apache.http.impl.auth.NTLMEngineImpl$NTLMMessage.addUShort(NTLMEngineImpl.java:946)
        at org.apache.http.impl.auth.NTLMEngineImpl$Type1Message.getResponse(NTLMEngineImpl.java:1052)
        at org.apache.http.impl.auth.NTLMEngineImpl.getType1Message(NTLMEngineImpl.java:148)
        at org.apache.http.impl.auth.NTLMEngineImpl.generateType1Msg(NTLMEngineImpl.java:1641)
        at org.apache.http.impl.auth.NTLMScheme.authenticate(NTLMScheme.java:139)
        at org.apache.http.impl.auth.AuthSchemeBase.authenticate(AuthSchemeBase.java:138)
        at org.apache.http.impl.auth.HttpAuthenticator.doAuth(HttpAuthenticator.java:239)
        at org.apache.http.impl.auth.HttpAuthenticator.generateAuthResponse(HttpAuthenticator.java:202)
        at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:262)
        at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
        at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:88)
        at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
        at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
        at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
        at PACKAGE_SCRUBBED.SharePointTransferer.call(SharePointTransferer.java:74)
        at PACKAGE_SCRUBBED.SharePointTransferer.call(SharePointTransferer.java:1)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        at java.lang.Thread.run(Thread.java:662)

如果有人能弄清楚造成这个问题的原因，我将不胜感激。

编辑：我找到了一个解决方法来解决这个问题，但我仍然会理解究竟发生了什么。

Answer 1

这是一个错误，在httpclient 4.5.2版本中解决了

http://www.apache.org/dist/httpcomponents/httpclient/RELEASE_NOTES-4.5.x.txt

版本4.5.2

更新日志：

[HTTPCLIENT-1715] NTLMEngineImpl＃Type1Message不是线程安全的，但声明为常量。供稿人：Olivier Lafontaine，Gary Gregory

Answer 2

你不能在并发环境中重用和HttpClientContext都不是NTLMScheme，因为它们都被标记为@NotThreadSafe（参见javadoc）。在我的环境中，我得到了同样的错误，解决了类似的事情：

synchronized(context) {
    HttpResponse response = client.execute(put, context);
}

经过身份验证的上下文被重用，但一次是一个线程。

Answer 3

我最终设法通过将每个路由的连接数设置为1来解决此问题，如下所示。

conManager.setDefaultMaxPerRoute(1);

我仍然不确定问题发生的原因，或解决问题的正确方法是什么，但这个解决方案对我有用。

MultiThreaded应用程序中的NTLM身份验证失败

3 个答案:

版本4.5.2

更新日志：