首页 文章

K8s Ingress，启动入口控制器nginx错误？

提问于
浏览 次
0

我有两个 spring 启动容器，我想设置入口服务 . 正如文件here所说，ingress有两部分，一部分是控制器，另一部分是资源 .

我的两个资源是两个容器：gearbox-rack-eureka-server和gearbox-rack-config-server . 区别在于端口，因此入口可以通过不同的端口路由流量 . 我的yaml文件如下：

eureka_pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: gearbox-rack-eureka-server
  labels:
    app: gearbox-rack-eureka-server
    purpose: platform_eureka_demo
spec:
  containers:
  - name:  gearbox-rack-eureka-server
    image: 192.168.1.229:5000/gearboxrack/gearbox-rack-eureka-server
    ports:
        - containerPort: 8761

eureka_svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: gearbox-rack-eureka-server
  labels:
    name: gearbox_rack_eureka_server
spec:
  selector:
    app: gearbox-rack-eureka-server
  type: NodePort
  ports:
    - port: 8761
      nodePort: 31501
      name: tcp

config_pod.yaml

apiVersion: v1
kind: Pod
metadata:
  name: gearbox-rack-config-server
  labels:
    app: gearbox-rack-config-server
    purpose: platform-demo
spec:
  containers:
  - name:  gearbox-rack-config-server
    image: 192.168.1.229:5000/gearboxrack/gearbox-rack-config-server
    ports:
    - containerPort: 8888
    env:
      - name: EUREKA_SERVER
        value: http://172.16.100.83:8761

config_svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: gearbox-rack-config-server
  labels:
    name: gearbox-rack-config-server
spec:
  selector:
    app: gearbox-rack-config-server
  type: NodePort
  ports:
    - port: 8888
      nodePort: 31502
      name: tcp

我的ingress-nginx控制器大多被复制from the link above，

ingress_nginx_ctl.yaml：

kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
spec:
  type: LoadBalancer
  selector:
    app: ingress-nginx
  ports:
  - name: http
    port: 80
    targetPort: http
  - name: https
    port: 443
    targetPort: https
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: ingress-nginx
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: ingress-nginx
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - image: nginx:1.13.12
        name: ingress-nginx
        imagePullPolicy: Always
        ports:
          - name: http
            containerPort: 80
            protocol: TCP
          - name: https
            containerPort: 443
            protocol: TCP
        livenessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
        args:
        - /nginx-ingress-controller
        - --default-backend-service=$(POD_NAMESPACE)/nginx-default-backend

我做了以下命令，他们成功了 .

kubectl apply -f eureka_pod.yaml
kubectl apply -f eureka_svc.yaml
kubectl apply -f config_pod.yaml
kubectl apply -f config_svc.yaml

然后我从执行 kubectl apply -f ingress_nginx_ctl.yaml 收到错误，pod没有启动，日志列在下面：

[root@master3 nginx-ingress-controller]# kubectl get pods
NAME                             READY     STATUS             RESTARTS   AGE
gearbox-rack-config-server       1/1       Running            0          39m
gearbox-rack-eureka-server       1/1       Running            0          40m
ingress-nginx-686c9975d5-7d464   0/1       CrashLoopBackOff   6          7m
[root@master3 nginx-ingress-controller]# kubectl logs -f ingress-nginx-686c9975d5-7d464
container_linux.go:247: starting container process caused "exec: \"/nginx-ingress-controller\": stat /nginx-ingress-controller: no such file or directory"

我在root下创建了一个目录/ nginx-ingress-controller，并再次重复这些步骤，它仍然说同样的错误 . 有人能指出我的问题吗？

我把我的ingress_nginx_res.yaml作为参考，它也可能有错误 .

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: my-ingress
spec:
  rules:
  - host: 172.16.100.83
    http:
      paths:
      - backend:
          serviceName: gearbox-rack-eureka-server
          servicePort: 8761
  - host: 172.16.100.83
    http:
      paths:
      - path:
        backend:
          serviceName: gearbox-rack-config-server
          servicePort: 8888

==========================================

第二版

更改图像链接后，之前的错误消失，但仍有以下权限问题：

[root@master3 ingress]# kubectl get pods
NAME                             READY     STATUS             RESTARTS   AGE
gearbox-rack-config-server       1/1       Running            0          15m
gearbox-rack-eureka-server       1/1       Running            0          15m
ingress-nginx-8679f9c8ff-5sxw7   0/1       CrashLoopBackOff   5          12m

日志消息如下：

[root@master3 kube]# kubectl logs ingress-nginx-8679f9c8ff-5sxw7
W0530 07:54:22.290114       5 client_config.go:533] Neither --kubeconfig nor --master was specified.  Using the inClusterConfig.  This might not work.
I0530 07:54:22.290374       5 main.go:158] Creating API client for https://10.96.0.1:443
-------------------------------------------------------------------------------
NGINX Ingress controller
  Release:    0.15.0
  Build:      git-df61bd7
  Repository: https://github.com/kubernetes/ingress-nginx
-------------------------------------------------------------------------------
I0530 07:54:22.298248       5 main.go:202] Running in Kubernetes Cluster version v1.9 (v1.9.2) - git (clean) commit 5fa2db2bd46ac79e5e00a4e6ed24191080aa463b - platform linux/amd64
F0530 07:54:22.298610       5 main.go:80] ✖ It seems the cluster it is running with Authorization enabled (like RBAC) and there is no permissions for the ingress controller. Please check the configuration

这是RBAC问题 . 我检查从论坛下载的安装脚本：

heapster-rbac.yaml：

kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: heapster
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: system:heapster
subjects:
- kind: ServiceAccount
  name: heapster
  namespace: kube-system

相关的kubelet开始参数之一如下:(我不知道它是否相关） .

Environment="KUBELET_AUTHZ_ARGS=--authorization-mode=Webhook --client-ca-file=/etc/kubernetes/pki/ca.crt"

通过哪种方式，我可以授予入口控制器的权限？只需将namespace kube-system放入ingress_nginx_ctl.yaml？

================================================== ==============

第三版

我将Kun Li的代码放入 ingress_nginx_role_rb.yaml ，并运行以下命令：

kubectl apply -f eureka_pod.yaml
kubectl apply -f eureka_svc.yaml
kubectl apply -f config_pod.yaml
kubectl apply -f config_svc.yaml
kubectl apply -f ingress_nginx_role_rb.yaml (just copy paste from Kun Li's answer)
kubectl apply -f nginx_default_backend.yaml
kubectl apply -f ingress_nginx_ctl.yaml

nginx_default_backend.yaml文件如下：

kind: Service
apiVersion: v1
metadata:
  name: nginx-default-backend
  namespace: kube-system
spec:
  ports:
  - port: 80
    targetPort: http
  selector:
    app: nginx-default-backend
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: nginx-default-backend
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: nginx-default-backend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: default-http-backend
        image: chenliujin/defaultbackend
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
          requests:
            cpu: 10m
            memory: 20Mi
        ports:
        - name: http
          containerPort: 8080
          protocol: TCP

ingress_nginx_ctl.yaml如下：

kind: Service
apiVersion: v1
metadata:
  name: ingress-nginx
spec:
  type: LoadBalancer
  selector:
    app: ingress-nginx
  ports:
  - name: http
    port: 80
    targetPort: http
  - name: https
    port: 443
    targetPort: https
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: ingress-nginx
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: ingress-nginx
    spec:
      terminationGracePeriodSeconds: 60
      serviceAccount: lb
      containers:
      - image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.15.0
        name: ingress-nginx
        imagePullPolicy: Always
        ports:
          - name: http
            containerPort: 80
            protocol: TCP
          - name: https
            containerPort: 443
            protocol: TCP
        livenessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
        args:
        - /nginx-ingress-controller
        - --default-backend-service=$(POD_NAMESPACE)/nginx-default-backend

从这里，我们可以看到服务ingress-nginx命名空间是默认的，而不是kube-system . 但无论如何，控制器已经启动了 .

[root@master3 ingress]# kubectl get pods -n kube-system
NAME                                      READY     STATUS    RESTARTS   AGE
calico-etcd-cdn8z                         1/1       Running   0          11m
calico-kube-controllers-d554689d5-tzdq5   1/1       Running   0          11m
calico-node-dz4d6                         2/2       Running   1          11m
coredns-65dcdb4cf-h62bh                   1/1       Running   0          11m
etcd-master3                              1/1       Running   0          10m
heapster-5c448886d-swp58                  1/1       Running   0          11m
ingress-nginx-6ccc799fbc-hq2rm            1/1       Running   0          9m
kube-apiserver-master3                    1/1       Running   0          10m

ingress-nginx pod的命名空间是kube-system（如上所示），但其服务的命名空间是默认的 . （如下所示） .

[root@master3 ingress]# kubectl get service
NAME                         TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
gearbox-rack-config-server   NodePort       10.97.211.136   <none>        8888:31502/TCP               43m
gearbox-rack-eureka-server   NodePort       10.106.69.13    <none>        8761:31501/TCP               43m
ingress-nginx                LoadBalancer   10.105.114.64   <pending>     80:30646/TCP,443:31332/TCP   42m
kubernetes                   ClusterIP      10.96.0.1       <none>        443/TCP                      44m

正如评论中所提到的，专家的回应帮助我向前迈进 .

kubernetes kubernetes-ingress

1 回答

  • 1

    对于入口控制器，应使用image quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.15.0 . 您需要设置nginx-default-backend pod和服务 .

    关于RBAC，我认为您需要一个seviceaccount来部署您的nginx-ingress-controller，具有以下角色和绑定：

    apiVersion: v1
kind: ServiceAccount
metadata:
  name: lb
  namespace: kube-system

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: nginx-ingress-normal
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - endpoints
      - nodes
      - pods
      - secrets
    verbs:
      - list
      - watch
  - apiGroups:
      - ""
    resources:
      - nodes
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - services
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - ""
    resources:
        - events
    verbs:
        - create
        - patch
  - apiGroups:
      - "extensions"
    resources:
      - ingresses/status
    verbs:
      - update

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: nginx-ingress-minimal
  namespace: kube-system
rules:
  - apiGroups:
      - ""
    resources:
      - configmaps
      - pods
      - secrets
      - namespaces
    verbs:
      - get
  - apiGroups:
      - ""
    resources:
      - configmaps
    resourceNames:
      - "ingress-controller-leader-dev"
      - "ingress-controller-leader-prod"
    verbs:
      - get
      - update
  - apiGroups:
      - ""
    resources:
      - configmaps
    verbs:
      - create
  - apiGroups:
      - ""
    resources:
      - endpoints
    verbs:
      - get

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: nginx-ingress-minimal
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: nginx-ingress-minimal
subjects:
  - kind: ServiceAccount
    name: lb
    namespace: kube-system

---

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: nginx-ingress-normal
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: nginx-ingress-normal
subjects:
  - kind: ServiceAccount
    name: lb
    namespace: kube-system
    回复于

相关问题