使用Airflow在AWS EKS上运行KubernetesPodOperator 获取kubernetes.client.rest.ApiException：（400），指出版本\“ v1 \”中的Pod无法作为Pod处理：v1.Pod.Spec。 aws-eks不支持这种类型的pod创建吗？
这是完整的日志
[2019-03-13 01:13:47,991] {models.py:1593} INFO - Executing
<Task(KubernetesPodOperator): failing-task> on 2019-03-
13T01:12:33.980710+00:00
[2019-03-13 01:13:47,991] {base_task_runner.py:118} INFO - Running:
['bash', '-c', 'airflow run testkuberete2 failing-task 2019-03-
13T01:12:33.980710+00:00 --job_id 20 --raw -sd DAGS_FOLDER/test-
kubepodopptest5.py --cfg_path /tmp/tmpmwb4uzk0']
[2019-03-13 01:13:48,415] {base_task_runner.py:101} INFO - Job 20:
Subtask failing-task [2019-03-13 01:13:48,415] {settings.py:174} INFO -
settings.configure_orm(): Using pool settings. pool_size=5,
pool_recycle=3600, pid=408
[2019-03-13 01:13:48,864] {base_task_runner.py:101} INFO - Job 20:
Subtask failing-task [2019-03-13 01:13:48,863] {__init__.py:51} INFO -
Using executor CeleryExecutor
[2019-03-13 01:13:49,115] {base_task_runner.py:101} INFO - Job 20:
Subtask failing-task [2019-03-13 01:13:49,114] {models.py:273} INFO -
Filling up the DagBag from /usr/local/airflow/dags/test-
kubepodopptest5.py
[2019-03-13 01:13:49,473] {base_task_runner.py:101} INFO - Job 20:
Subtask failing-task [2019-03-13 01:13:49,473] {cli.py:520} INFO -
Running <TaskInstance: testkuberete2.failing-task 2019-03-
13T01:12:33.980710+00:00 [running]> on host airflow-worker-1.airflow-
worker.rguonew.svc.cluster.local
我写了一些自定义日志来检查发送到EKS的内容：
[2019-03-13 01:13:49,782] {logging_mixin.py:95} INFO - [2019-03-13
01:13:49,781] {api_client.py:117} ERROR - header_params {Accept
application/json}
[2019-03-13 01:13:49,782] {logging_mixin.py:95} INFO - [2019-03-13
01:13:49,782] {api_client.py:117} ERROR - header_params {Content-Type
application/json}
[2019-03-13 01:13:49,782] {logging_mixin.py:95} INFO - [2019-03-13
01:13:49,782] {api_client.py:117} ERROR - header_params {User-Agent
Swagger-Codegen/8.0.1/python}
[2019-03-13 01:13:49,782] {logging_mixin.py:95} INFO - [2019-03-13
01:13:49,782] {api_client.py:128} ERROR - resource_path {namespace
rguonew}
[2019-03-13 01:13:49,782] {logging_mixin.py:95} INFO - [2019-03-13
01:13:49,782] {api_client.py:151} ERROR - body {{'apiVersion': 'v1',
'kind': 'Pod', 'metadata': {'name': 'fail-422f98fc', 'labels': {},
'annotations': {}}, 'spec': {'containers': [{'name': 'base',
'image':
'python:3.6.6-stretch', 'command': 'echo', 'imagePullPolicy':
'IfNotPresent', 'args': '10'}], 'restartPolicy': 'Never',
'nodeSelector': {}, 'volumes': [], 'serviceAccountName': 'default',
'affinity': {}}}}
[2019-03-13 01:13:49,782] {logging_mixin.py:95} INFO - [2019-03-13
01:13:49,782] {api_client.py:155} ERROR - url
{https://xxxxxxxxxxxx.xxxx.us-east-
1.eks.amazonaws.com/api/v1/namespaces/rguonew/pods}
[2019-03-13 01:13:49,838] {logging_mixin.py:95} INFO - [2019-03-13
01:13:49,823] {pod_launcher.py:58} ERROR - Exception when attempting to
create Namespaced Pod.
追踪
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-
packages/airflow/contrib/kubernetes/pod_launcher.py", line 55, in
run_pod_async
resp = self._client.create_namespaced_pod(body=req,
namespace=pod.namespace)
File "/usr/local/lib/python3.6/site-
packages/kubernetes/client/apis/core_v1_api.py", line 6115, in
create_namespaced_pod
(data) = self.create_namespaced_pod_with_http_info(namespace, body,
**kwargs)
File "/usr/local/lib/python3.6/site-
packages/kubernetes/client/apis/core_v1_api.py", line 6206, in
create_namespaced_pod_with_http_info
collection_formats=collection_formats)
File "/usr/local/lib/python3.6/site-
packages/kubernetes/client/api_client.py", line 329, in call_api
_return_http_data_only, collection_formats, _preload_content,
_request_timeout)
File "/usr/local/lib/python3.6/site-packages/kubernetes/client/ap
i_client.py", line 163, in __call_api
_request_timeout=_request_timeout)
File "/usr/local/lib/python3.6/site-
packages/kubernetes/client/api_client.py", line 372, in request
body=body)
File "/usr/local/lib/python3.6/site-
packages/kubernetes/client/rest.py", line 266, in POST
body=body)
File "/usr/local/lib/python3.6/site-
packages/kubernetes/client/rest.py", line 222, in request
raise ApiException(http_resp=r)
kubernetes.client.rest.ApiException: (400)
Reason: Bad Request
响应：
HTTP response headers: HTTPHeaderDict({'Audit-Id': '96f8ffee-3e65-47a7-
bb3c-fe519281ee45', 'Content-Type': 'application/json', 'Date': 'Wed,
13 Mar 2019 01:13:49 GMT', 'Content-Length': '491'})
HTTP response body: {"kind":"Status","apiVersion":"v1","metadata":
{},"status":"Failure","message":"Pod in version \"v1\" cannot be
handled as a Pod: v1.Pod.Spec: v1.PodSpec.Containers: []v1.Container:
v1.Container.Command: []string: decode slice: expect [ or n, but found
\", error found in #10 byte of ...|ommand\": \"echo\", \"im|..., bigger
context ...|se\", \"image\": \"python:3.6.6-stretch\", \"command\":
\"echo\", \"imagePullPolicy\": \"IfNotPresent\", \"args\":
|...","reason":"BadRequest","code":400}
[2019-03-13 01:13:49,838] {logging_mixin.py:95} INFO -
答案 0 :(得分：4)
啊，很高兴在EKS上使用Airflow找到另一个！！
我也正在运行类似的Airflow工作负载，并且可以确认AWS EKS支持KubernetesPodOperator。
从响应来看，我认为也许在实例化KubernetesPodOperator时，用于创建pod的API调用被弄乱了，或者您的python代码包含了错误的参数类型。
我已解码了您的广告连播定义（如果出错，请纠正我）：
---
apiVersion: v1
kind: Pod
metadata:
name: fail-422f98fc
labels: {}
annotations: {}
spec:
containers:
- name: base
image: python:3.6.6-stretch
command: ['echo']
imagePullPolicy: IfNotPresent
args: ['10']
restartPolicy: Never
nodeSelector: {}
volumes: []
serviceAccountName: default
affinity: {}
您应将“命令”和“参数”放在方括号中，如下所示：
public Object calculatePay() {
int potentialPayBefore10 = 12;
int potentialPayAfter10 = 8;
// $12 hour * 5 hours worked
potentialPayBefore10 = 12 * 5;
potentialPayAfter10 = 8 * 2;
// TODO Auto-generated method stub
if (potentialPayBefore10 < 60) {
return potentialPayAfter10;
} else
return potentialPayBefore10;
}
}
public class DaysWorked {
/*
* Story: As a babysitter In order to get paid for 1 night of work I want to
* calculate my nightly charge
*/
// Project Goal: Create test to show Mellie being paid
// Start with calc time for 1 hour of work
@Test
public void calculatepayforworkafterstarttimeat12hourly() {
// 5 is hours worked if start at 5 pm til 10p
MellieWageCalculator potentialPay = new MellieWageCalculator(5);
// assert equals gives -> (expected, actual)
assertEquals(60, potentialPay.calculatePay());
}
@Test
public void calculatepayforworkafter10pmat8hourly() {
// 2 hours worked if start at 10pm til 12 pm
MellieWageCalculator potentialPay = new MellieWageCalculator(2);
assertEquals(16, potentialPay.calculatePay());
}
}
希望这会有所帮助！
答案 1 :(得分：0)
当环境变量的值没有用引号引起来时，我收到此错误。
- name: sonic
image: nginx
env:
- name: NGINX_PORT
value: 8080 // "8080" fixed it