Question

我创建了一个通过Java sdk使用Sagemaker的docker容器。该容器部署在具有多个副本的k8s集群上。

容器正在向Sagemaker发出简单的请求，以列出我们已经训练和部署的某些模型。但是，我们现在遇到一些Java证书问题。我对k8和证书不是一个新手，因此，如果您能提供一些解决问题的帮助，将不胜感激。

在尝试列出端点时，日志中有一些痕迹：

org.apache.http.conn.ssl.SSLConnectionSocketFactory.createLayeredSocket(SSLConnectionSocketFactory.java:394)
    at org.apache.http.conn.ssl.SSLConnectionSocketFactory.connectSocket(SSLConnectionSocketFactory.java:353)
    at com.amazonaws.http.conn.ssl.SdkTLSSocketFactory.connectSocket(SdkTLSSocketFactory.java:132)
    at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:141)
    at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at com.amazonaws.http.conn.ClientConnectionManagerFactory$Handler.invoke(ClientConnectionManagerFactory.java:76)
    at com.amazonaws.http.conn.$Proxy67.connect(Unknown Source)
    at org.apache.http.impl.execchain.MainClientExec.establishRoute(MainClientExec.java:380)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:236)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:184)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:55)
    at com.amazonaws.http.apache.client.impl.SdkHttpClient.execute(SdkHttpClient.java:72)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1236)
    at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1056)
    ... 70 common frames omitted
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:397)
    at sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:302)
    at sun.security.validator.Validator.validate(Validator.java:262)
    at sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:324)
    at sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:229)
    at sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:124)
    at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1621)
    ... 97 common frames omitted
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
    at sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141)
    at sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126)
    at java.security.cert.CertPathBuilder.build(CertPathBuilder.java:280)
    at sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:392)
    ... 103 common frames omitted

Answer 1

这很可能与管理员添加到网络中的某些自定义SSL认证路径有关。您可能需要通过在浏览器中打开任何受保护的网站并单击地址栏左侧的“安全”链接来检查SSL根证书（至少在chrome中是如此）。您将看到一个弹出窗口，显示证书和证明信息。转到其证书路径并查看ROOT证书，如果它是自定义证书，则需要将其添加到cacerts文件中。阅读此link了解更多详情

Answer 2

我想我找到了解决我问题的答案。我已经建立了另一个k8s集群，并将容器也部署在那里。它们工作正常，并且不会发生证书问题。在进行更多调查时，我注意到它们是第一个k8s群集上DNS解析的一些问题。实际上，例如，带有证书问题的容器无法ping google.com。我通过不依赖core-dns并在deploy.yaml文件中设置DNS配置来解决DNS问题。我不确定为什么会这样，但这似乎已经解决了证书问题。

Answer 3

当Java不知道TLS端点返回的根证书时，会出现您收到的错误消息。如果更改可用的根证书，通常会发生这种情况。

每https://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/JSSERefGuide.html#Customization：

"If a truststore named <java-home>/lib/security/jssecacerts is found, it is used. 
If not, then a truststore named <java-home>/lib/security/cacerts is searched for and used (if it exists).
Finally, if a truststore is still not found, then the truststore managed by the TrustManager will be a new empty truststore."

Openssl是调试此类证书问题的好工具。您可以使用以下命令来检索端点返回的证书。这可以帮助您确定证书链的外观。

openssl s_client -showcerts -connect www.example.com:443 </dev/null

您可以使用JRE出售的实用程序 keytool 查看Java知道的证书列表。

keytool -list -cacerts

某些系统管理员会通过将备用信任库文件写入默认位置来覆盖默认证书。其他时间，团队可以使用 javax.net.ssl.trustStore 系统属性覆盖默认设置。

最后，您可以使用还与JRE一起出售的 jps 实用程序，以查看在运行的Java进程上设置的系统属性。

jps -v

Kubernetes的Sagemaker证书问题

3 个答案: