Question

我正在非kube系统名称空间资源上测试PodSecurityPolicy资源。

首先，我已经通过检查kube-apiserver进程确保启用了接纳插件PodSecurityPolicy：

kube-apiserver --advertise-address=192.168.56.4 --allow-privileged=true --authorization-mode=Node,RBAC --client-ca-file=/etc/kubernetes/pki/ca.crt --enable-admission-plugins=NodeRestriction,PodSecurityPolicy --enable-bootstrap-token-auth=true --etcd-cafile=/etc/kubernetes/pki/etcd/ca.crt --etcd-certfile=/etc/kubernetes/pki/apiserver-etcd-client.crt --etcd-keyfile=/etc/kubernetes/pki/apiserver-etcd-client.key --etcd-servers=https://127.0.0.1:2379 --insecure-port=0 --kubelet-client-certificate=/etc/kubernetes/pki/apiserver-kubelet-client.crt --kubelet-client-key=/etc/kubernetes/pki/apiserver-kubelet-client.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --proxy-client-cert-file=/etc/kubernetes/pki/front-proxy-client.crt --proxy-client-key-file=/etc/kubernetes/pki/front-proxy-client.key --requestheader-allowed-names=front-proxy-client --requestheader-client-ca-file=/etc/kubernetes/pki/front-proxy-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6443 --service-account-key-file=/etc/kubernetes/pki/sa.pub --service-cluster-ip-range=10.96.0.0/12 --tls-cert-file=/etc/kubernetes/pki/apiserver.crt --tls-private-key-file=/etc/kubernetes/pki/apiserver.key

使用以下策略创建了PodSecurityPolicy资源：

[root@master manifests]# kubectl get psp -o wide
NAME                PRIV    CAPS   SELINUX    RUNASUSER   FSGROUP    SUPGROUP   READONLYROOTFS   VOLUMES
podsecplcy          false          RunAsAny   RunAsAny    RunAsAny   RunAsAny   true

按如下方式创建clusterrole和clusterrolebinding：

[root@master manifests]# kubectl describe clusterrole non-priv-role
Name:         non-priv-role
Labels:       <none>
Annotations:  <none>
PolicyRule:
  Resources                   Non-Resource URLs  Resource Names  Verbs
  ---------                   -----------------  --------------  -----
  podsecuritypolicies.policy  []                 [podsecplcy]    [list get watch]

[root@master ~]# kubectl describe clusterrolebinding psprb
Name:         psprb
Labels:       <none>
Annotations:  <none>
Role:
  Kind:  ClusterRole
  Name:  non-priv-role
Subjects:
  Kind            Name     Namespace
  ----            ----     ---------
  ServiceAccount  default  default
[root@master ~]#

下面是我用来创建广告连播的广告连播清单：

    apiVersion: v1
    kind: Pod
    metadata:
      name: pod-privileged
    spec:
      containers:
      - name: main
        image: alpine
        command: ["/bin/sleep", "999999"]
        securityContext:
          privileged: true

我希望它不允许在默认名称空间上创建特权pod。

实际上pod已创建并运行良好：

[root@master ~]# kubectl get po
NAME                     READY   STATUS                       RESTARTS   AGE
pod-privileged           1/1     Running                      0          17s

我是否需要创建用户或组并分配此clusterrolebinding进行检查，否则它将正常工作，因为我们已将此clusterrolebinding分配给默认名称空间和默认服务帐户？

还要如何检查当前拥有的角色和特权？

请找到kubernetes版本和podsecplcy yaml文件详细信息

[root@master ~]# kubectl get no
NAME         STATUS   ROLES    AGE     VERSION
master.k8s   Ready    master   5d1h    v1.16.2
node1.k8s    Ready    <none>   5d      v1.16.3
node2.k8s    Ready    <none>   4d22h   v1.16.3
[root@master ~]#

[root@master ~]# kubectl cluster-info
Kubernetes master is running at https://192.168.56.4:6443
KubeDNS is running at https://192.168.56.4:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.
[root@master ~]#


apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"policy/v1beta1","kind":"PodSecurityPolicy","metadata":{"annotations":{},"name":"podsecplcy"},"spec":{"allowPrivilegeEscalation":false,"fsGroup":{"rule":"RunAsAny"},"hostIPC":false,"hostNetwork":false,"hostPID":false,"hostPorts":[{"max":30000,"min":10000}],"privileged":false,"readOnlyRootFilesystem":true,"runAsUser":{"rule":"RunAsAny"},"seLinux":{"rule":"RunAsAny"},"supplementalGroups":{"rule":"RunAsAny"},"volumes":["*"]}}
  creationTimestamp: "2019-11-23T21:31:36Z"
  name: podsecplcy
  resourceVersion: "626637"
  selfLink: /apis/policy/v1beta1/podsecuritypolicies/podsecplcy
  uid: f3316992-0dc7-4c19-852b-57e5babc451f
spec:
  allowPrivilegeEscalation: false
  fsGroup:
    rule: RunAsAny
  hostPorts:
  - max: 30000
    min: 10000
  readOnlyRootFilesystem: true
  runAsUser:
    rule: RunAsAny
  seLinux:
    rule: RunAsAny
  supplementalGroups:
    rule: RunAsAny
  volumes:
  - '*'

Answer 1

在这里我如何验证podSecurityPolicy podsecplcy

[root@master ~]# kubectl get psp
NAME                PRIV    CAPS   SELINUX    RUNASUSER          FSGROUP     SUPGROUP    READONLYROOTFS   VOLUMES
podsecplcy          false          RunAsAny   RunAsAny           RunAsAny    RunAsAny    true             *

问题： 即使我们创建了podsecuritypolicy podsecplcy并将其添加到clusterrole non-priv-role并将其分配给clusterrolebinding psprb，我们仍然能够创建没有错误的特权pod。但是期望它应该抛出错误

解决方案： 每当我们尝试提交特权Pod清单yaml时，我们都没有提到要创建Pod的用户，组或服务帐户。自从我以kubeadm为根安装了k8s集群后，根在主节点上，我的角色是cluster-admin，由于我的角色cluster-admin具有全部特权，因此我能够提交特权pod manifest yaml文件。

那么现在如何将其作为我们将要限制的其他用户，组或服务帐户进行测试，以创建特权吊舱？

如果我们是cluster-admin的主节点，则必须提交如下的kubectl create命令来测试podsecuritypolicy。

然后要检查我们可以在何处创建特权吊舱作为特定服务帐户

[root@master ~]# kubectl create -f kubia-priv-pod.yml   --as=system:serviceaccount:foo:default -n foo
Error from server (Forbidden): error when creating "kubia-priv-pod.yml": pods "pod-privileged" is forbidden: unable to validate against any pod security policy: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
[root@master ~]#
[root@master ~]# kubectl create -f kubia-priv-pod.yml --as=system:serviceaccount:default:default
Error from server (Forbidden): error when creating "kubia-priv-pod.yml": pods "pod-privileged" is forbidden: unable to validate against any pod security policy: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
[root@master ~]#

要检查我们在哪里可以创建特权吊舱作为服务帐户和组的组合，然后

[root@master ~]# kubectl create -f kubia-priv-pod.yml  --as-group=system:authenticated --as=system:serviceaccount:default:default
Error from server (Forbidden): error when creating "kubia-priv-pod.yml": pods "pod-privileged" is forbidden: unable to validate against any pod security policy: [spec.containers[0].securityContext.privileged: Invalid value: true: Privileged containers are not allowed]
[root@master ~]#

然后要确保我们是否能够将priv pod创建为集群管理组

[root@master ~]# kubectl get clusterrolebindings -o go-template='{{range .items}}{{range .subjects}}{{.kind}}-{{.name}} {{end}} {{" - "}} {{.metadata.name}} {{"\n"}}{{end}}' | grep "^Group-system:masters"
Group-system:masters   -  cluster-admin
[root@master ~]#

[root@master ~]# kubectl create -f kubia-priv-pod.yml  --as-group=system:masters --as=system:serviceaccount:default:default
pod/pod-privileged created
[root@master ~]#

附加说明： 如果我们想将此受限簇绑定仅应用于特定的组或用户或服务帐户，则必须创建以下内容

kubectl create clusterrolebinding psprb --clusterrole=non-priv-role --user=jaya_vkl@yahoo.co.in
kubectl create clusterrolebinding psprbgrp --clusterrole=non-priv-role --group=system:authenticated
kubectl create clusterrolebinding psprbsa --clusterrole=non-priv-role --serviceaccount=default:default

Answer 2

您尚未阻止特权升级，因此我建议您在下面设置指令。 allowPrivilegeEscalation: false

Answer 3

您的角色定义不正确，“动词”应为“使用”。请更正这一点。

如何将PodSecurityPolicy应用于非kube系统名称空间资源？

3 个答案: