我是新来的领事,在我们在kubernetes中新建立的领事集群中,面临一些注册和访问服务以进行领事连接的困难。
我引用了以下链接来设置集群。
https://github.com/kelseyhightower/consul-on-kubernetes
通过添加生成的证书并添加了两个包含客户端和服务器配置的configmap,创建了一个秘密。刚刚添加了一个ACL令牌,并且从其他地方引用了其他一些ACL配置。
这是我最终的服务器和客户端配置:
服务器:
{
"acl_datacenter": "dc1",
"acl_default_policy": "allow",
"acl_master_token": "root",
"acl_down_policy": "extend-cache",
"bind_addr": "0.0.0.0",
"ca_file": "/etc/tls/ca.pem",
"cert_file": "/etc/tls/consul.pem",
"client_addr": "0.0.0.0",
"datacenter": "dc1",
"data_dir": "/consul/data",
"disable_host_node_id": true,
"domain": "cluster.local",
"key_file": "/etc/tls/consul-key.pem",
"server": true,
"ui": true,
"verify_incoming": true,
"verify_outgoing": true,
"verify_server_hostname": true,
"connect": {
"enabled": true,
"proxy": {
"allow_managed_api_registration": true,
"allow_managed_root": true
}
},
"ports": {
"https": 8443
}
}
客户:
{
"acl_datacenter": "dc1",
"acl_default_policy": "allow",
"acl_down_policy": "extend-cache",
"acl_master_token": "root",
"bind_addr": "0.0.0.0",
"ca_file": "/etc/tls/ca.pem",
"cert_file": "/etc/tls/consul.pem",
"datacenter": "dc1",
"data_dir": "/consul/data",
"disable_host_node_id": true,
"domain": "cluster.local",
"key_file": "/etc/tls/consul-key.pem",
"server": false,
"ui": false,
"verify_incoming": true,
"verify_outgoing": true,
"verify_server_hostname": true,
"connect": {
"enabled": true,
"proxy": {}
},
"ports": {
"https": 8443
}
}
另一个设计决策是,我在运行服务的kubernetes容器中启动consul客户端作为sidecar进程。这只是暂时的,因为目前我们无权在kube集群中创建DaemonSet。
现在,当我们站起来时,客户端节点将成功添加到集群中。我什至可以将服务添加到领事连接,并且可以从领事UI中看到它们。但是问题是,即使我将Intentions应用于服务,我也可以直接访问它们而无需TLS。领事主的dig命令也会失败。这是下面的屏幕快照和配置以及我尝试过的一些东西。
群集:
$ consul members
Node Address Status Type Build Protocol DC Segment
consul-0 172.16.8.217:8301 alive server 1.2.3 2 dc1 <all>
consul-1 172.16.7.43:8301 alive server 1.2.3 2 dc1 <all>
consul-2 172.16.9.131:8301 alive server 1.2.3 2 dc1 <all>
gohttp1-7796bf88f-l6qbc 172.16.8.246:8301 alive client 1.2.3 2 dc1 <default>
gohttp2-5679c67c68-2s7bz 172.16.7.55:8301 alive client 1.2.3 2 dc1 <default>
Kubernetes服务:
$ kubectl get endpoints
NAME ENDPOINTS AGE
consul 172.16.7.43:8301,172.16.8.217:8301,172.16.9.131:8301 + 24 more... 30m
gohttp 172.16.71.248:8080 4d
gohttp1 172.16.8.246:8080 27m
gohttp2 172.16.7.55:9191 26m
kube-state-metrics 172.16.60.81:8080,172.16.60.81:8081 35d
我在领事代理端车中添加的服务配置:
service1.json <-- sidecar1
{
"service": {
"name": "gohttp1",
"port": 8080,
"connect": { "proxy": {} }
}
}
service2.json <-- sidecar2
{
"service": {
"name": "gohttp2",
"port": 9191,
"connect": { "proxy": {} }
}
}
证书验证:
$curl http://localhost:8500/v1/connect/ca/roots
{"ActiveRootID":"da:bb:23:e9:2c:eb:48:c2:8c:d9:a3:d3:39:5a:e2:4a:c1:03:01:16","TrustDomain":"c93eb719-883e-f3ec-91a4-318806440530.consul","Roots":[{"ID":"da:bb:23:e9:2c:eb:48:c2:8c:d9:a3:d3:39:5a:e2:4a:c1:03:01:16","Name":"Consul CA Root Cert","SerialNumber":10,"SigningKeyID":"64:36:3a:39:36:3a:66:30:3a:38:39:3a:62:36:3a:64:37:3a:37:37:3a:62:36:3a:62:35:3a:32:65:3a:62:65:3a:32:63:3a:34:38:3a:31:61:3a:39:35:3a:34:34:3a:35:63:3a:36:64:3a:65:64:3a:37:38:3a:63:39:3a:31:34:3a:61:62:3a:33:34:3a:61:30:3a:36:62:3a:65:31:3a:33:33:3a:66:32:3a:66:37:3a:65:36:3a:30:35","NotBefore":"2018-10-09T21:19:40Z","NotAfter":"2028-10-06T21:19:40Z","RootCert":"-----BEGIN CERTIFICATE-----\nMIICXDCCAgGgAwIBAgIBCjAKBggqhkjOPQQDAjAXMRUwEwYDVQQDEwxDb25zdWwg\nQ0EgMTAwHhcNMTgxMDA5MjExOTQwWhcNMjgxMDA2MjExOTQwWjAXMRUwEwYDVQQD\nEwxDb25zdWwgQ0EgMTAwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAAQqMyaMgS0p\nVUUS2nADYXQ3SguxES2wTsHKENAVKe8q15/0cFCWXaJfBRpt3gvawQZPxVprztBn\nwb6HFdYgJX0go4IBPDCCATgwDgYDVR0PAQH/BAQDAgGGMA8GA1UdEwEB/wQFMAMB\nAf8waAYDVR0OBGEEX2Q2Ojk2OmYwOjg5OmI2OmQ3Ojc3OmI2OmI1OjJlOmJlOjJj\nOjQ4OjFhOjk1OjQ0OjVjOjZkOmVkOjc4OmM5OjE0OmFiOjM0OmEwOjZiOmUxOjMz\nOmYyOmY3OmU2OjA1MGoGA1UdIwRjMGGAX2Q2Ojk2OmYwOjg5OmI2OmQ3Ojc3OmI2\nOmI1OjJlOmJlOjJjOjQ4OjFhOjk1OjQ0OjVjOjZkOmVkOjc4OmM5OjE0OmFiOjM0\nOmEwOjZiOmUxOjMzOmYyOmY3OmU2OjA1MD8GA1UdEQQ4MDaGNHNwaWZmZTovL2M5\nM2ViNzE5LTg4M2UtZjNlYy05MWE0LTMxODgwNjQ0MDUzMC5jb25zdWwwCgYIKoZI\nzj0EAwIDSQAwRgIhALNau/tIFiNz6aCu8Spa5Mj4Usksb6qDL/sW7uqgducyAiEA\npvrwfeK37epjSX7zCTnYPI7hFXXNcS02F77fAzLIAo4=\n-----END CERTIFICATE-----\n","IntermediateCerts":null,"Active":true,"CreateIndex":11,"ModifyIndex":11}]}
$ curl http://localhost:8500/v1/connect/ca/configuration
{"Provider":"consul","Config":{"LeafCertTTL":"72h","RotationPeriod":"2160h"},"CreateIndex":8,"ModifyIndex":8}
现在,当我尝试进行挖掘查找时,我得到了错误。
$dig @127.0.0.1 -p 8600 gohttp2.connect.consul
; <<>> DiG 9.10.6 <<>> @127.0.0.1 -p 8600 gohttp2.connect.consul
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached
或者,如果我登录(执行到Pod)服务1并尝试访问服务2,则可以不使用TLS直接访问。我可以从curl verbose模式中看到它。尚无证书验证。即使添加了一个Intent,我仍然可以访问该服务。我是错误地注册或访问了服务吗?
$kubectl exec -it gohttp1-7796bf88f-l6qbc -c consul /bin/sh
# curl -v gohttp2.2304613691.svc:9191
* Rebuilt URL to: gohttp2.2304613691.svc:9191/
* Trying 192.168.93.239...
* TCP_NODELAY set
* Connected to gohttp2.2304613691.svc (192.168.93.239) port 9191 (#0)
> GET / HTTP/1.1
> Host: gohttp2.2304613691.svc:9191
> User-Agent: curl/7.61.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Tue, 09 Oct 2018 22:21:53 GMT
< Content-Length: 12
< Content-Type: text/plain; charset=utf-8
<
* Connection #0 to host gohttp2.2304613691.svc left intact
Hello world!/