我有一个带有以下(容器化)组件的Kubernetes onebox部署,全部运行为 --net=host ,其中kubelet作为特权Docker容器运行,kubernetes标志 --allow-privileged 设置为true .

gcr.io/google_containers/hyperkube-amd64:v1.7.9   "/bin/bash -c './hype"     kubelet
gcr.io/google_containers/hyperkube-amd64:v1.7.9   "/bin/bash -c './hype"     kube-proxy
gcr.io/google_containers/hyperkube-amd64:v1.7.9   "/bin/bash -c './hype"     kube-scheduler
gcr.io/google_containers/hyperkube-amd64:v1.7.9   "/bin/bash -c './hype"     kube-controller-manager
gcr.io/google_containers/hyperkube-amd64:v1.7.9   "/bin/bash -c './hype"     kube-apiserver
quay.io/coreos/etcd:v3.1.0                        "/usr/local/bin/etcd "     etcd

最重要的是,我使用 kubectl create -f https://github.com/kubernetes/kubernetes/blob/master/test/kubemark/resources/manifests/kube-addon-manager.yaml 启用了插件管理器,其中calico 2.6.1和kube-dns 1.14.5的默认yaml清单已安装到 /etc/kubernetes/addons/ . calico pod按预期提供两个节点(install-cni和calico-node) .

但是,kube-dns卡在ContainerCreating或ContainerCannotRun中,尝试启动Kubernetes暂停容器时出现以下错误:

{"log":"I1111 00:35:19.549318       1 manager.go:913] Added container: \"/kubepods/burstable/pod3173eef3-c678-11e7-ac4b-e41d2d59689e/1dd57d6f6c996d7abe061f6236fc8a0150cf6f95d16d5c3c462c9ed7158d3c54\" (aliases: [k8s_POD_kube-dns-v20-141138543-pmdww_kube-system_3173eef3-c678-11e7-ac4b-e41d2d59689e_0 1dd57d6f6c996d7abe061f6236fc8a0150cf6f95d16d5c3c462c9ed7158d3c54], namespace: \"docker\")\n","stream":"stderr","time":"2017-11-11T00:35:19.5526284Z"}
{"log":"I1111 00:35:19.549433       1 cni.go:291] About to add CNI network cni-loopback (type=loopback)\n","stream":"stderr","time":"2017-11-11T00:35:19.5526748Z"}
{"log":"I1111 00:35:19.549504       1 handler.go:325] Added event \u0026{/kubepods/burstable/pod3173eef3-c678-11e7-ac4b-e41d2d59689e/1dd57d6f6c996d7abe061f6236fc8a0150cf6f95d16d5c3c462c9ed7158d3c54 2017-11-11 00:35:19.3931718 +0000 UTC containerCreation {\u003cnil\u003e}}\n","stream":"stderr","time":"2017-11-11T00:35:19.5527217Z"}
{"log":"I1111 00:35:19.551134       1 container.go:407] Start housekeeping for container \"/kubepods/burstable/pod3173eef3-c678-11e7-ac4b-e41d2d59689e/1dd57d6f6c996d7abe061f6236fc8a0150cf6f95d16d5c3c462c9ed7158d3c54\"\n","stream":"stderr","time":"2017-11-11T00:35:19.5527441Z"}
{"log":"E1111 00:35:19.555099       1 cni.go:294] Error adding network: failed to Statfs \"/proc/54226/ns/net\": no such file or directory\n","stream":"stderr","time":"2017-11-11T00:35:19.5553606Z"}
{"log":"E1111 00:35:19.555122       1 cni.go:237] Error while adding to cni lo network: failed to Statfs \"/proc/54226/ns/net\": no such file or directory\n","stream":"stderr","time":"2017-11-11T00:35:19.5553887Z"}
{"log":"I1111 00:35:19.600281       1 manager.go:970] Destroyed container: \"/kubepods/burstable/pod3173eef3-c678-11e7-ac4b-e41d2d59689e/1dd57d6f6c996d7abe061f6236fc8a0150cf6f95d16d5c3c462c9ed7158d3c54\" (aliases: [k8s_POD_kube-dns-v20-141138543-pmdww_kube-system_3173eef3-c678-11e7-ac4b-e41d2d59689e_0 1dd57d6f6c996d7abe061f6236fc8a0150cf6f95d16d5c3c462c9ed7158d3c54], namespace: \"docker\")\n","stream":"stderr","time":"2017-11-11T00:35:19.6005722Z"}

我看到\暂停容器不断出现只是为了退出一秒钟,带有一个无害的错误消息(这个是旧的,我停止了群集,所以它不会继续产生更多的容器):

ubuntu@r172-16-6-39:~$ docker ps -a | grep 216e39defa36
216e39defa36        gcr.io/google_containers/pause-amd64:3.0          "/pause"                 About an hour ago   Exited (0) About an hour ago                         k8s_POD_kube-dns-v20-141138543-xvdmv_kube-system_0594732f-c688-11e7-9da5-e41d2d59689e_17
ubuntu@r172-16-6-39:~$ docker logs 216e39defa36
shutting down, got signal: Terminated

dir /proc/54226 在我的主机上不存在,我认为是CNI抱怨的原因 . 但Calico的暂停容器很好,运行相同的图像,因此必须要么只能在kube-dns的情况下写入,要么在Calico的情况下不能写入 . 我发现了一些类似SELinux-related error on Openshift的引用,但是我运行的是一个没有安装SELinux的裸Ubuntu 14.04 VM .

ubuntu@r172-16-6-39:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 14.04.4 LTS
Release:        14.04
Codename:       trusty
ubuntu@r172-16-6-39:~$ setenforce
The program 'setenforce' is currently not installed. You can install it by typing:
sudo apt-get install selinux-utils

我的CNI conf也非常简单,由install-cni calico容器生成:

ubuntu@r172-16-6-39:~$ cat /etc/cni/net.d/10-calico.conf
{
    "name": "k8s-pod-network",
    "cniVersion": "0.1.0",
    "type": "calico",
    "log_level": "debug",
    "datastore_type": "kubernetes",
    "nodename": "172.16.6.39",
    "mtu": 1500,
    "ipam": {
        "type": "host-local",
        "subnet": "usePodCidr"
    },
    "policy": {
        "type": "k8s",
        "k8s_auth_token": "****"
    },
    "kubernetes": {
        "k8s_api_root": "https://168.16.0.1:443",
        "kubeconfig": "/etc/kubernetes/kubeconfig"
    }
}

有没有人碰过类似的东西?