Kubernetes Python客户端错误create_namespaced_binding:(409)原因:冲突

时间:2018-06-19 16:35:26

标签: python api kubernetes http-headers scheduling

我正在使用k8 v1.7和Python Client v2.0。我的自定义调度程序检测到一个挂起的Pod并成功调度它。但是,在将Pod分配给节点之后,它抱怨说Pod已经分配给了一个节点,尽管它只是由调度程序本身分配的。这件事值得关注吗?或者我该如何解决?

错误消息

create_namespaced_binding: (409)
Reason: Conflict
HTTP response headers: HTTPHeaderDict({'Date': 'Tue, 19 Jun 2018 16:14:57 GMT', 'Content-Length': '289', 'Content-Type': 'application/json'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"Operation cannot be fulfilled on pods/binding \"ps0-16-r935x\": pod ps0-16-r935x is already assigned to node \"blipp65\"","reason":"Conflict","details":{"name":"ps0-16-r935x","kind":"pods/binding"},"code":409}

scheduler.py

from kubernetes import client, config, watch
from kubernetes.client.rest import ApiException
config.load_kube_config()
v1 = client.CoreV1Api()

scheduler_name = 'my-custom-scheduler-v1'

def nodes_available():
    ready_nodes = []
    for n in v1.list_node().items:
        for status in n.status.conditions:
            if status.status == 'True' and status.type == 'Ready':
                ready_nodes.append(n.metadata.name)
    return ready_nodes


def scheduler(name, node, namespace='default'):
    body = client.V1Binding()

    target = client.V1ObjectReference()
    target.kind = 'Node'
    target.apiVersion = 'v1'
    target.name = node

    meta = client.V1ObjectMeta()
    meta.name = name

    body.target = target
    body.metadata = meta

    return v1.create_namespaced_binding_binding(name, namespace, body)


def main():
    w = watch.Watch()
    for event in w.stream(v1.list_namespaced_pod, 'default'):
        if event['object'].status.phase == 'Pending' and event['object'].spec.scheduler_name == scheduler_name:
            print "Pending Found"
            try:
                res = scheduler(event['object'].metadata.name,random.choice(nodes_available()))
                print "success"
            except Exception as a:
                print ("Exception when calling CoreV1Api->create_namespaced_binding: %s\n" % a)

POD YML文件

apiVersion: v1
kind: Pod
metadata:
  name: shoeb-pod
spec:
  schedulerName: my-custom-scheduler-v1
  containers:
  - name: redis
    image: redis

2 个答案:

答案 0 :(得分:1)

创建广告连播后,计划程序会收到三个“待处理”事件:

  1. 尚未预定播客('node_name': None, 'status': {'conditions': None,...}
  2. 预定播客('node_name': 'some_node_name','status': {'conditions': [...,'status': True, 'type':'PodScheduled'],...}
  3. 已初始化Pod,但尚未准备好('node_name': 'minikube','status': {'conditions': [...,'status': True, 'type':'Initialized'], ... ,'status': False, 'type':'Ready']}

因此,您的自定义调度程序应将Pod绑定到第一个事件的节点上,检查Pod的状态并确保已在第二个事件出现时对Pod进行了调度,然后在第三个事件出现时检查Pod是否已初始化

如果出现问题,调度程序可能需要考虑先前的错误,并可能尝试将Pod调度到其他节点。

在您的情况下,您的调度程序会威胁到所有三个事件(如第一个事件),并试图一次又一次地调度播客。这就是为什么您看到“ pod xxx is already assigned to node yyy”错误。

答案 1 :(得分:0)

这是更新后的主要方法(根据@VAS的回答,谢谢),以找到尚未安排的正确PENDING吊舱。

def main():
    w = watch.Watch()
    for event in w.stream(v1.list_namespaced_pod, 'default'): # default == namespace name
        # All pending pods have 3 states (not scheduled, scheduled, initialized but not ready yet)
        # We look for NOT SCHEDULED pod and conditions==None
        if event['object'].status.phase == 'Pending' and event['object'].status.conditions == None and event['object'].spec.scheduler_name == CUSTOM_SCHEDULER_NAME:
            print "Pending and Not Scheduled POD Found "+event['object'].metadata.name
            try:
                res = scheduler(event['object'].metadata.name,random.choice(nodes_available())) # nodes_available() returns all available nodes
                print "success"
            except Exception as a:
                print ("Exception when calling CoreV1Api->create_namespaced_binding: %s\n" % a)