kubernetes节点关闭时,HA Elasticsearch群集关闭

时间:2018-09-12 06:18:44

标签: elasticsearch kubernetes high-availability downtime

我在4节点kubernetes集群中部署了Elasticsearch集群。以下是elasticsearch部署:

es-master(正在运行3个专用副本)

es-client(正在运行2个专用副本)

es-data(正在运行2个专用副本)

以上所有7个节点都在4节点kube集群中运行。但是,当我重启运行1个客户端节点的一个kubernetes集群时,elasticsearch集群没有响应请求,并且 elasticsearch集群URL已关闭

我认为在拥有多个副本之后,这应该可以正常工作。知道如何使elasticseach URL始终起作用。

Python脚本检查es-cluster的运行状况:

## Cluster health ##
url = 'http://<es-cluster>:30000/_cluster/health'
headers = {'content-type': 'application/json'}
get_reg = requests.get(url, timeout = 60, headers=headers)

if (get_req.status_code != 200):
    print ( "Error- cluster request failed" )
    print get_req.status_code
    print get_req.text
    break
print ("passed")
print get_req.status_code
print get_req.text
time.sleep(1)

一切正常后输出:

passed
200
{"cluster_name":"ebs_elastic","status":"green","timed_out":false,"number_of_nodes":7,"number_of_data_nodes":2,"active_primary_shards":46,"active_shards":92,"relocating_shards":0,"initializing_shards":0,"unassigned_shards":0,"delayed_unassigned_shards":0,"number_of_pending_tasks":0,"number_of_in_flight_fetch":0,"task_max_waiting_in_queue_millis":0,"active_shards_percent_as_number":100.0}

一个节点关闭时的输出:

hemanty-mac-0:Desktop hemanty$ python ping.py 
        Traceback (most recent call last):
  File "ping.py", line 25, in <module>
    post_req = requests.get(url, timeout = 60, headers=headers)
  File "/Library/Python/2.7/site-packages/requests-2.19.1-py2.7.egg/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/Library/Python/2.7/site-packages/requests-2.19.1-py2.7.egg/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/Library/Python/2.7/site-packages/requests-2.19.1-py2.7.egg/requests/sessions.py", line 512, in request
    resp = self.send(prep, **send_kwargs)
  File "/Library/Python/2.7/site-packages/requests-2.19.1-py2.7.egg/requests/sessions.py", line 622, in send
    r = adapter.send(request, **kwargs)
  File "/Library/Python/2.7/site-packages/requests-2.19.1-py2.7.egg/requests/adapters.py", line 513, in send
    raise ConnectionError(e, request=request)
    requests.exceptions.ConnectionError: HTTPConnectionPool(host='<es-cluster>', port=30000): Max retries exceeded with url: /_cluster/health (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x108765f90>: Failed to establish a new connection: [Errno 51] Network is unreachable',))

请注意,重新启动主,客户端或数据之类的POD时,我没有看到任何停机时间。仅在重启kubernetes节点时才会发生。

0 个答案:

没有答案