PangYeon`s IT Log

kubernetes dashboard를 설치하게 되면 기본적으로 kong proxy pod가 생성되면서,

이 kong proxy를 통해 사용하게되는데, 이를 비활성화하고 따로 ingress 적용을 통해 kubernetes dashboard에 접속하고

사용할 수 있다.

일반 ingress 적용 가이드 -

2024.10.22 - [Develop/k8s] - kubernetes dashboard ingress 연결 방법

1. 설치시 kong proxy disabled

helm upgrade --install kubernetes-dashboard kubernetes-dashboard/kubernetes-dashboard \
--create-namespace --namespace kubernetes-dashboard \
--set kong.enabled=false

2. ingress 적용

아래 URL과 service 조회를 통해 ingress를 생성한다

https://github.com/kubernetes/dashboard/blob/master/hack/gateway/prod.kong.yml

kubectl get svc -n kubernetes-dashboard
NAME                                   TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
kubernetes-dashboard-api               ClusterIP   10.106.25.34     <none>        8000/TCP   6m13s
kubernetes-dashboard-auth              ClusterIP   10.101.168.28    <none>        8000/TCP   6m13s
kubernetes-dashboard-metrics-scraper   ClusterIP   10.104.27.35     <none>        8000/TCP   6m13s
kubernetes-dashboard-web               ClusterIP   10.109.176.153   <none>        8000/TCP   6m13s

dashboard ui에 로그인하기위해선 tls 적용이 필요하다.

Kubernetes Dashboard 아래 소스를 보면 setTokenCookie 부분이 있는데, 이 부분 때문에 tls가 필요해보인다.
아님말고...?
dashboard-master/modules/web/src/common/services/global/authentication.ts
---
private setTokenCookie_(token: string): void { if (this.isCurrentProtocolSecure_()) { this.cookies_.set(this.config_.authTokenCookieName, token, null, null, null, true, 'Strict'); return; } if (this.isCurrentDomainSecure_()) { this.cookies_.set(this.config_.authTokenCookieName, token, null, null, location.hostname, false, 'Strict'); } }
---

tls 적용을 위한 인증서 및 secret 생성

# 인증서 생성
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=dashboard.wky.kr/O=Kubernetes" -addext "subjectAltName = DNS:dashboard.wky.kr"

# secret 생성
kubectl create secret tls tls-dashboard --key tls.key --cert tls.crt -n kubernetes-dashboard

ingress 생성

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  ingressClassName: nginx
  rules:
  - host: dashboard.wky.kr
    http:
      paths:
      - path: /api/v1/login
        pathType: Prefix
        backend:
          service:
            name: kubernetes-dashboard-auth
            port:
              number: 8000
      - path: /api/v1/csrftoken/login
        pathType: Prefix
        backend:
          service:
            name: kubernetes-dashboard-auth
            port:
              number: 8000
      - path: /api/v1/me
        pathType: Prefix
        backend:
          service:
            name: kubernetes-dashboard-auth
            port:
              number: 8000
      - path: /api
        pathType: Prefix
        backend:
          service:
            name: kubernetes-dashboard-api
            port:
              number: 8000
      - path: /metrics
        pathType: Prefix
        backend:
          service:
            name: kubernetes-dashboard-api
            port:
              number: 8000
      - path: /
        pathType: Prefix
        backend:
          service:
            name: kubernetes-dashboard-web
            port:
              number: 8000
  tls:
  - hosts:
    - dashboard.wky.kr
    secretName: tls-dashboard

3. 로그인테스트를 위한 계정 및 토큰 발급

계정생성

apiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kubernetes-dashboard

토큰발급

kubectl -n kubernetes-dashboard create token admin-user

4. 로그인 및 확인

tls 적용을 했으므로 반드시 https로 접속 후 테스트한다.

kubernetes-dashboard 는 기본적으로 localhost 에서 https 를 권장하지만 Ingress 를 적용하여 접속할 수도 있다.

kubernetes-dashboard-kong-proxy를 기준으로 https ingress 접속하는 방법이다.

1. 인증서 생성

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=dashboard.wky.kr/O=Kubernetes"
또는
openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=dashboard.wky.kr/O=Kubernetes" -addext "subjectAltName = DNS:dashboard.wky.kr"

2. secret 생성

kubectl create secret tls tls-dashboard --key tls.key --cert tls.crt -n kubernetes-dashboard

3. Ingress 생성 및 적용

# vi ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kubernetes-dashboard
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
spec:
  ingressClassName: nginx
  rules:
  - host: dashboard.wky.kr
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: kubernetes-dashboard-kong-proxy
            port:
              number: 443
  tls:
  - hosts:
    - dashboard.wky.kr
    secretName: tls-dashboard
 
 # 적용
 kubectl apply -f ingress.yaml -n kubernetes-dashboard

4. 접속할 pc 의 hosts에 등록

vi /etc/hosts

172.1.1.1 dashboard.wky.kr

5. 확인

기타. ssl-passthrough

ssl-passthrough 를 적용하기 위해선 deployment에 --enable-ssl-passthrough 플래그를 추가한다.

kubectl edit deploy -n ingress-nginx ingress-nginx-controller

...
spec:
  containers:
  - args:
    - /nginx-ingress-controller
    - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
    - --election-id=ingress-nginx-leader
    - --controller-class=k8s.io/ingress-nginx
    - --ingress-class=nginx
    - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
    - --validating-webhook=:8443
    - --validating-webhook-certificate=/usr/local/certificates/cert
    - --validating-webhook-key=/usr/local/certificates/key
    - --enable-ssl-passthrough=true
...

ingress 설정

# vi ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: kubernetes-dashboard
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
spec:
  ingressClassName: nginx
  rules:
  - http:
      paths:
      - path: /dash(/|$)(.*)
        pathType: ImplementationSpecific
        backend:
          service:
            name: kubernetes-dashboard-kong-proxy
            port:
              number: 443
 
 # 적용
 kubectl apply -f ingress.yaml -n kubernetes-dashboard

확인