在纱线下的火花作业中连接Kerberos + SSL启用的solr

时间:2018-05-09 19:45:45

标签: apache-spark hadoop ssl yarn kerberos

我有SOLR 6群集,它启用了Kerberos和SSL。当我使用带有CloudSolrClient的测试客户端连接到它时,它可以正常工作。但是在火花作业驱动程序中运行它时相同的代码我得到以下检查和失败错误。

我检查了所有提到的相关校验和问题,如反向dns查找和添加java无限罐,所有纱线节点中的一切看起来都正确。我也可以验证它们是正确的,因为我的普通java客户端能够从所有服务器进行查询。

Caused by: org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error from server at https://10-0-0-64.securonix.com:18987/solr/test3: Expected mime type application/octet-stream but got text/html. <html>
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>
<title>Error 403 GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed)</title>
</head>
<body><h2>HTTP ERROR 403</h2>
<p>Problem accessing /solr/test3/select. Reason:
<pre>    GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed)</pre></p>
</body>
</html>

        at org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:578)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:279)
        at org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:268)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.doRequest(LBHttpSolrClient.java:447)
        at org.apache.solr.client.solrj.impl.LBHttpSolrClient.request(LBHttpSolrClient.java:388)
        ... 18 more

LogType:stdout
Log Upload Time:Wed May 09 13:29:07 -0400 20

我正在添加我的工作协商堆栈跟踪和非工作协议,请让我知道从哪个地方开始调试。

 277313:29:01,363 DEBUG DefaultClientConnectionOperator:176 - Connecting to 10-0-0-64.company.com:18987
 278013:29:01,457 DEBUG RequestAddCookies:122 - CookieSpec selected: solr-portaware
 287413:29:01,458 DEBUG RequestAuthCache:76 - Auth cache not set in the context
 287513:29:01,459 DEBUG RequestTargetAuthentication:79 - Target auth state: UNCHALLENGED
 287613:29:01,459 DEBUG RequestProxyAuthentication:88 - Proxy auth state: UNCHALLENGED
 287613:29:01,459 DEBUG SystemDefaultHttpClient:684 - Attempt 1 to execute request
 287613:29:01,459 DEBUG DefaultClientConnection:276 - Sending request: GET /solr/test3/select?q=*%3A*&_stateVer_=test3%3A46&wt=javabin&version=2 HTTP/1.1
 287613:29:01,460 DEBUG wire:72 -  >> "GET /solr/test3/select?q=*%3A*&_stateVer_=test3%3A46&wt=javabin&version=2 HTTP/1.1[\r][\n]"
 287713:29:01,460 DEBUG wire:72 -  >> "User-Agent: Solr[org.apache.solr.client.solrj.impl.HttpSolrClient] 1.0[\r][\n]"
 287713:29:01,460 DEBUG wire:72 -  >> "Host: 10-0-0-64.company.com:18987[\r][\n]"
 287713:29:01,461 DEBUG wire:72 -  >> "Connection: Keep-Alive[\r][\n]"
 287813:29:01,461 DEBUG wire:72 -  >> "[\r][\n]"
 287813:29:01,461 DEBUG headers:280 - >> GET /solr/test3/select?q=*%3A*&_stateVer_=test3%3A46&wt=javabin&version=2 HTTP/1.1
 287813:29:01,461 DEBUG headers:283 - >> User-Agent: Solr[org.apache.solr.client.solrj.impl.HttpSolrClient] 1.0
 287813:29:01,461 DEBUG headers:283 - >> Host: 10-0-0-64.company.com:18987
 287813:29:01,461 DEBUG headers:283 - >> Connection: Keep-Alive
 287813:29:01,463 DEBUG wire:72 -  << "HTTP/1.1 401 Authentication required[\r][\n]"
 288013:29:01,464 DEBUG wire:72 -  << "WWW-Authenticate: Negotiate[\r][\n]"
 288113:29:01,464 DEBUG wire:72 -  << "Set-Cookie: hadoop.auth=; Path=/; Domain=10-0-0-64.company.com:18987; Expires=Thu, 01-Jan-1970 00:00:00 GMT; Secure; HttpOnly[\r][\n]"
 288113:29:01,464 DEBUG wire:72 -  << "Cache-Control: must-revalidate,no-cache,no-store[\r][\n]"
 288113:29:01,464 DEBUG wire:72 -  << "Content-Type: text/html;charset=iso-8859-1[\r][\n]"
 288113:29:01,465 DEBUG wire:72 -  << "Content-Length: 277[\r][\n]"
 288213:29:01,465 DEBUG wire:72 -  << "[\r][\n]"
 288213:29:01,465 DEBUG DefaultClientConnection:261 - Receiving response: HTTP/1.1 401 Authentication required
 288213:29:01,465 DEBUG headers:264 - << HTTP/1.1 401 Authentication required
 288213:29:01,465 DEBUG headers:267 - << WWW-Authenticate: Negotiate
 288213:29:01,466 DEBUG headers:267 - << Set-Cookie: hadoop.auth=; Path=/; Domain=10-0-0-64.company.com:18987; Expires=Thu, 01-Jan-1970 00:00:00 GMT; Secure; HttpOnly
 288313:29:01,466 DEBUG headers:267 - << Cache-Control: must-revalidate,no-cache,no-store
 288313:29:01,466 DEBUG headers:267 - << Content-Type: text/html;charset=iso-8859-1
 288313:29:01,466 DEBUG headers:267 - << Content-Length: 277
 288313:29:01,472 DEBUG ResponseProcessCookies:118 - Cookie accepted [hadoop.auth="", version:0, domain:10-0-0-64.company.com:18987, path:/, expiry:Wed Dec 31 19:00:00 EST 1969]
 288913:29:01,473 DEBUG SystemDefaultHttpClient:511 - Connection can be kept alive indefinitely
 289013:29:01,473 DEBUG SystemDefaultHttpClient:77 - Authentication required
 289013:29:01,473 DEBUG SystemDefaultHttpClient:107 - 10-0-0-64.company.com:18987 requested authentication
 289013:29:01,474 DEBUG TargetAuthenticationStrategy:174 - Authentication schemes in the order of preference: [Negotiate, Kerberos, NTLM, Digest, Basic]
 289113:29:01,476 DEBUG SPNegoScheme:266 - Received challenge '' from the auth server
 289313:29:01,477 DEBUG TargetAuthenticationStrategy:203 - Challenge for Kerberos authentication scheme not available
 289413:29:01,477 DEBUG TargetAuthenticationStrategy:203 - Challenge for NTLM authentication scheme not available
 289413:29:01,477 DEBUG TargetAuthenticationStrategy:203 - Challenge for Digest authentication scheme not available
 289413:29:01,477 DEBUG TargetAuthenticationStrategy:203 - Challenge for Basic authentication scheme not available
 289413:29:01,477 DEBUG SystemDefaultHttpClient:157 - Selected authentication options: [NEGOTIATE]
 289413:29:01,478 DEBUG wire:72 -  << "<html>[\n]"
 289513:29:01,478 DEBUG wire:72 -  << "<head>[\n]"
 289513:29:01,478 DEBUG wire:72 -  << "<meta http-equiv="Content-Type" content="text/html;charset=utf-8"/>[\n]"
 289513:29:01,478 DEBUG wire:72 -  << "<title>Error 401 Authentication required</title>[\n]"
 289513:29:01,478 DEBUG wire:72 -  << "</head>[\n]"
 289513:29:01,479 DEBUG wire:72 -  << "<body><h2>HTTP ERROR 401</h2>[\n]"
 289613:29:01,479 DEBUG wire:72 -  << "<p>Problem accessing /solr/test3/select. Reason:[\n]"
 289613:29:01,479 DEBUG wire:72 -  << "<pre>    Authentication required</pre></p>[\n]"
 289613:29:01,479 DEBUG wire:72 -  << "</body>[\n]"
 289613:29:01,479 DEBUG wire:72 -  << "</html>[\n]"
 289613:29:01,479 DEBUG RequestAddCookies:122 - CookieSpec selected: solr-portaware
 289613:29:01,479 DEBUG RequestAuthCache:76 - Auth cache not set in the context
 289613:29:01,479 DEBUG RequestTargetAuthentication:79 - Target auth state: CHALLENGED
 289613:29:01,480 DEBUG RequestTargetAuthentication:79 - Generating response to an authentication challenge using Negotiate scheme

**************** WORKING ****************************** ****

107718:07:10,421 DEBUG wire:72 -  >> "Authorization: Negotiate YIIFCQYGKwYBBQUCoIIE/TCCBPmgDTALBgkqhkiG9xIBAgKhBAMCAfaiggTgBIIE3GCCBNgGCSqGSIb3EgECAgEAboIExzCCBMOgAwIBBaEDAgEOogcDBQAgAAAAo4IBU2GCAU8wggFLoAMCAQWhDxsNU0VDVVJPTklYLkNPTaIqMCigAwIBAKEhMB8bBEhUVFAbFzEwLTAtMC02NC5zZWN1cm9uaXguY29to4IBBTCCAQGgAwIBEqEDAgEBooH0BIHxKKe1e1rmBivsMS7EM7QlfH/1P9AqnSEuZF/A9o2FBwt6X010f7braFpqBT7mXTU28PiUYecdtxva39av/UUQbSupla64CvR75PatuwiQYKtvnEBHINhX4/lv25Mm6SR7YiFM9o4jbxYXL9k/1bS6UitipNxA7xAX1cLLE270/j/3M4BpL3TgohZR9RVpFrL5rmGTVnrcP6nTZ0YSsaQ4js8fbldYQCxBAyTeiIwb3HGmrl68P/crWaQqv/9dkDS0yuvW26nQoTIwUuvlsTmYAB/Pgopi1hobiTaiMN7z7BIjBXjNBYUPTc6Ab9qxmtaQfqSCA1UwggNRoAMCARKiggNIBIIDROkC2Wypm9nMm60R0rZG+MkcbLLIjyqfSoliOrfCov+0Qko2nxpwOcm1ckdtNgKRnYzzCi9gWTSd9ZIisMPynY/YgCfc4a10QqKER+NwdJLxEXiNfqBKQ81xsp6llZ/I+WMpuZtmH5o8RAtAmamt4G0O31jtBwrSGFMzsxVYGi1kZeN0RqB9TvHScbFXe/TSJWoXc3qL/RzDz519MXQ/MTwYE9klvf15zXgakuEMPJbx0jJ7sVvwRTuNJ+mpv37qwX8JAKMls0045MJlc0zl/39OSh9edjnfi+IIM0K5FJ/wGgm3Vlt9t/rlCuYLlybfB5a3wZ78d/KOhmZwtBQ5arN/vXqE/UQz9rzRsbByZ7kDksphd98TgY3uPaq+SA1MBG5NrawLXxNpoxZZbgob6n6FLzesXQ2dnVMYuGLKoB/VZigAOqMR2wgLNCcNzI+dTvI7gTWf/ao/XfT5EFCd17kRH1XTvhaI6A1zULaGMNxzcoZJDlHZ0AHAnsUj4X0u4xXtVX3LaFKGkXtQNGQEsCtnQbygphl/rrgH06Mv3duhZiNqpb9TVpfbyJ6NoWzX5O4ud+meyAW4rDDKPoVUvm+kuxGLnW4W+TFXupTg2OxK6dQD/5t+INCu4nveH9sUHF+wX2K+9QfJ+hQ2MKZzQKRTdiSw7UACUcNjAQfyk5N7aZueXJE8tMeJLegVYa6Y3IQ8O6VYkg/1S719+CwlE47Z5eo5fw4mWyr0d05Et2Lt/ETomhpoMcjpajJAMwI3Nv0Y9heuwTC0uy2cv4X7ZbPOBvtTcDH936FImQRUHus40WEKA2eweEFAQliHxVhFEh6Zd07mxdOsR2LAI3gBEzdzmELd3DVtnxTsSe9hbHy7cvzFDHpKdXtJmyiAJsR38ij915/7Mv5g4np8Espcu6V0pdB/H5QM+pqrrsQ2cEpN6VExT8vRaswrZbqp6Juax4UGyG8KPS7mzV13qRsOIKn3AMA9Z2tjXXCTckqRqZhDIBPply1S+QLE8Ik4Yi3SkfkIjElEuWr4fLSPslkBSfPWeNFNOo84+rFrJnMoG2m4Rp5sLxIL811VYTicJSS6x0Kz1Uqf3LFFGi+Un8wodUahNqpb[\r][\n]"
 107818:07:10,421 DEBUG wire:72 -  >> "[\r][\n]"
 107818:07:10,421 DEBUG headers:272 - >> GET /solr/test3/select?q=*%3A*&_stateVer_=test3%3A46&wt=javabin&version=2 HTTP/1.1
 107818:07:10,421 DEBUG headers:275 - >> User-Agent: Solr[org.apache.solr.client.solrj.impl.HttpSolrClient] 1.0
 107818:07:10,421 DEBUG headers:275 - >> Host: 10-0-0-64.company.com:18987
 107818:07:10,421 DEBUG headers:275 - >> Connection: Keep-Alive
 107818:07:10,421 DEBUG headers:275 - >> Authorization: Negotiate YIIFCQYGKwYBBQUCoIIE/TCCBPmgDTALBgkqhkiG9xIBAgKhBAMCAfaiggTgBIIE3GCCBNgGCSqGSIb3EgECAgEAboIExzCCBMOgAwIBBaEDAgEOogcDBQAgAAAAo4IBU2GCAU8wggFLoAMCAQWhDxsNU0VDVVJPTklYLkNPTaIqMCigAwIBAKEhMB8bBEhUVFAbFzEwLTAtMC02NC5zZWN1cm9uaXguY29to4IBBTCCAQGgAwIBEqEDAgEBooH0BIHxKKe1e1rmBivsMS7EM7QlfH/1P9AqnSEuZF/A9o2FBwt6X010f7braFpqBT7mXTU28PiUYecdtxva39av/UUQbSupla64CvR75PatuwiQYKtvnEBHINhX4/lv25Mm6SR7YiFM9o4jbxYXL9k/1bS6UitipNxA7xAX1cLLE270/j/3M4BpL3TgohZR9RVpFrL5rmGTVnrcP6nTZ0YSsaQ4js8fbldYQCxBAyTeiIwb3HGmrl68P/crWaQqv/9dkDS0yuvW26nQoTIwUuvlsTmYAB/Pgopi1hobiTaiMN7z7BIjBXjNBYUPTc6Ab9qxmtaQfqSCA1UwggNRoAMCARKiggNIBIIDROkC2Wypm9nMm60R0rZG+MkcbLLIjyqfSoliOrfCov+0Qko2nxpwOcm1ckdtNgKRnYzzCi9gWTSd9ZIisMPynY/YgCfc4a10QqKER+NwdJLxEXiNfqBKQ81xsp6llZ/I+WMpuZtmH5o8RAtAmamt4G0O31jtBwrSGFMzsxVYGi1kZeN0RqB9TvHScbFXe/TSJWoXc3qL/RzDz519MXQ/MTwYE9klvf15zXgakuEMPJbx0jJ7sVvwRTuNJ+mpv37qwX8JAKMls0045MJlc0zl/39OSh9edjnfi+IIM0K5FJ/wGgm3Vlt9t/rlCuYLlybfB5a3wZ78d/KOhmZwtBQ5arN/vXqE/UQz9rzRsbByZ7kDksphd98TgY3uPaq+SA1MBG5NrawLXxNpoxZZbgob6n6FLzesXQ2dnVMYuGLKoB/VZigAOqMR2wgLNCcNzI+dTvI7gTWf/ao/XfT5EFCd17kRH1XTvhaI6A1zULaGMNxzcoZJDlHZ0AHAnsUj4X0u4xXtVX3LaFKGkXtQNGQEsCtnQbygphl/rrgH06Mv3duhZiNqpb9TVpfbyJ6NoWzX5O4ud+meyAW4rDDKPoVUvm+kuxGLnW4W+TFXupTg2OxK6dQD/5t+INCu4nveH9sUHF+wX2K+9QfJ+hQ2MKZzQKRTdiSw7UACUcNjAQfyk5N7aZueXJE8tMeJLegVYa6Y3IQ8O6VYkg/1S719+CwlE47Z5eo5fw4mWyr0d05Et2Lt/ETomhpoMcjpajJAMwI3Nv0Y9heuwTC0uy2cv4X7ZbPOBvtTcDH936FImQRUHus40WEKA2eweEFAQliHxVhFEh6Zd07mxdOsR2LAI3gBEzdzmELd3DVtnxTsSe9hbHy7cvzFDHpKdXtJmyiAJsR38ij915/7Mv5g4np8Espcu6V0pdB/H5QM+pqrrsQ2cEpN6VExT8vRaswrZbqp6Juax4UGyG8KPS7mzV13qRsOIKn3AMA9Z2tjXXCTckqRqZhDIBPply1S+QLE8Ik4Yi3SkfkIjElEuWr4fLSPslkBSfPWeNFNOo84+rFrJnMoG2m4Rp5sLxIL811VYTicJSS6x0Kz1Uqf3LFFGi+Un8wodUahNqpb
 107818:07:10,428 DEBUG wire:72 -  << "HTTP/1.1 200 OK[\r][\n]"
 108518:07:10,428 DEBUG wire:72 -  << "WWW-Authenticate: Negotiate oYH1MIHyoAMKAQChCwYJKoZIhvcSAQICom4EbGBqBgkqhkiG9xIBAgICAG9bMFmgAwIBBaEDAgEPok0wS6ADAgESokQEQlOz50MemlJ23yLycPU2zNSptyEvKzbtbWCzmStU9JF5m1stDqbNn/Z4z0X8Sh8hLZTxBN8Lw0it74YcSnFgBqC2CKNuBGxgagYJKoZIhvcSAQICAgBvWzBZoAMCAQWhAwIBD6JNMEugAwIBEqJEBEJTs+dDHppSdt8i8nD1NszUqbchLys27W1gs5krVPSReZtbLQ6mzZ/2eM9F/EofIS2U8QTfC8NIre+GHEpxYAagtgg=[\r][\n]"
 108518:07:10,428 DEBUG wire:72 -  << "Set-Cookie: hadoop.auth="u=company&p=company@company.COM&t=kerberos-dt&e=1525853230424&s=c7ujCu4e2i31H4l+8cDxxPnOf08="; Path=/; Domain=10-0-0-64.company.com:18987; Expires=Wed, 09-May-2018 08:07:10 GMT; Secure; HttpOnly[\r][\n]"
 108518:07:10,429 DEBUG wire:72 -  << "Content-Type: application/octet-stream[\r][\n]"
 108618:07:10,429 DEBUG wire:72 -  << "Content-Length: 243[\r][\n]"
 108618:07:10,429 DEBUG wire:72 -  << "[\r][\n]"
 108618:07:10,429 DEBUG DefaultClientConnection:253 - Receiving response: HTTP/1.1 200 OK
 108618:07:10,429 DEBUG headers:256 - << HTTP/1.1 200 OK
 108618:07:10,429 DEBUG headers:259 - << WWW-Authenticate: Negotiate oYH1MIHyoAMKAQChCwYJKoZIhvcSAQICom4EbGBqBgkqhkiG9xIBAgICAG9bMFmgAwIBBaEDAgEPok0wS6ADAgESokQEQlOz50MemlJ23yLycPU2zNSptyEvKzbtbWCzmStU9JF5m1stDqbNn/Z4z0X8Sh8hLZTxBN8Lw0it74YcSnFgBqC2CKNuBGxgagYJKoZIhvcSAQICAgBvWzBZoAMCAQWhAwIBD6JNMEugAwIBEqJEBEJTs+dDHppSdt8i8nD1NszUqbchLys27W1gs5krVPSReZtbLQ6mzZ/2eM9F/EofIS2U8QTfC8NIre+GHEpxYAagtgg=
 108618:07:10,429 DEBUG headers:259 - << Set-Cookie: hadoop.auth="u=company&p=company@company.COM&t=kerberos-dt&e=1525853230424&s=c7ujCu4e2i31H4l+8cDxxPnOf08="; Path=/; Domain=10-0-0-64.company.com:18987; Expires=Wed, 09-May-2018 08:07:10 GMT; Secure; HttpOnly
 108618:07:10,429 DEBUG headers:259 - << Content-Type: application/octet-stream
 108618:07:10,430 DEBUG headers:259 - << Content-Length: 243
 108718:07:10,430 DEBUG ResponseProcessCookies:118 - Cookie accepted [hadoop.auth=""u=company&p=company@company.COM&t=kerberos-dt&e=1525853230424&s=c7ujCu4e2i31H4l+8cDxxPnOf08="", version:0, domain:10-0-0-64.company.com:18987, path:/, expiry:Wed May 09 04:07:10 EDT 2018]
 108718:07:10,431 DEBUG SystemDefaultHttpClient:510 - Connection can be kept alive indefinitely
 108818:07:10,431 DEBUG SystemDefaultHttpClient:86 - Authentication succeeded
 108818:07:10,446 DEBUG wire:86 -  << "[0x2][0xa2][0xe0].responseHeader[0xa4][0xe0]+zkConnected[0x1][0xe0]&status[0x6][0x0][0x0][0x0][0x0][0xe0]%QTime[0x6][0x0][0x0][0x0][0x0][0xe0]&params[0xa4][0xe0]!q#*:*[0xe0]*_stateVer_(test3:46[0xe0]"wt'javabin[0xe0]'version!2[0xe0](response[0xc][0x83]b`[0x0][0x82][0xb][0xa5][0xe0]-useruniquekey#1~1[0xe0]$u_ida[0xe0])_version_[0x7][0x16]/Kk[0xab] [0x0][0x0][0xe0](tenantida[0xe0]*tenantname)company[0xb][0xa4][0xeb]!1[0xed][0x7][0x16]4[0x18][0x9c]C`[0x0][0x0][0xee]a[0xef])company"
 110318:07:10,448 DEBUG PoolingClientConnectionManager:262 - Connection [id: 0][route: {s}->https://10-0-0-64.company.com:18987][state: class org.apache.solr.client.solrj.impl.HttpSolrClient] can be kept alive indefinitely

1 个答案:

答案 0 :(得分:0)

这可能是由Kerberos客户端和服务器

之间的加密类型不匹配引起的

尝试更改/etc/krb5.conf - 在Solr服务器和客户端计算机上 - 添加[libdefaults]部分

  

[libdefaults]

     

default_tkt_ enctypes = arcfour-hmac-md5