반응형
간단하게 인증서 설정없이 Docker Private Registry를 k8s에 설치하는 방법이다. 설치 후 UI 적용할 예정.
1. id, password 생성 (굳이 안 해도 됨)
nerdctl run --entrypoint htpasswd httpd:2 -Bbn testuser testpassword > auth/htpasswd
2. namespace 생성 및 secret 생성
$ kubectl create ns registry
$ kubectl create secret generic registry-auth --from-file=idpass=./htpasswd -n registry
3. pv, pvc, deployment, service 생성
storageclass가 없을 경우 pv를 생성하고 workernode에 registry폴더를 생성한다.(해당 경로는 적절히 수정해서 사용하면 됨)
1번에 id, password 생성 하지 않았으면, deployment 내 registry-auth 부분들 삭제하고 진행
아래 yaml 생성 후 kubectl apply -f registry.yaml -n registry 명령어 실행
$ vi registry.yaml
$ kubectl apply -f registry.yaml -n registry
apiVersion: v1
kind: PersistentVolume
metadata:
name: registry-pv
labels:
reg: registry
spec:
capacity:
storage: 10Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
hostPath:
path: "/root/registry"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: registry-pvc
labels:
reg: registry
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 10Gi
selector:
matchLabels:
reg: registry
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: registry
labels:
reg: registry
spec:
replicas: 1
selector:
matchLabels:
reg: registry
template:
metadata:
labels:
reg: registry
spec:
containers:
- name: registry
image: registry:2
imagePullPolicy: IfNotPresent
ports:
- containerPort: 5000
volumeMounts:
- name: registry-auth
mountPath: /auth # 이미지 내 기본 registry auth 경로
- name: registry-data
mountPath: /var/lib/registry
env:
- name: REGISTRY_AUTH_HTPASSWD_PATH # 기본 설정
value: /auth/idpass # secert에 들어가는 파일을 idpass로 생성
- name: REGISTRY_AUTH_HTPASSWD_REALM # 기본 설정
value: "Registry Realm"
- name: REGISTRY_STORAGE_DELETE_ENABLED
value: "true"
volumes:
- name: registry-auth
secret:
secretName: registry-auth
- name: registry-data
persistentVolumeClaim:
claimName: registry-pvc
---
apiVersion: v1
kind: Service
metadata:
name: registry
spec:
type: NodePort
ports:
- port: 5000
targetPort: 5000
nodePort: 30050
protocol: TCP
selector:
reg: registry
4. 테스트
curl localhost:30050/v2/_catalog -u "testuser:testpassword"
5. UI 적용 (UI가 굳이 필요없으면 안 해도 됨)
UI 는 아래 오픈소스를 사용할 예정이다.
https://github.com/Joxit/docker-registry-ui
우선 위에 생성한 registry 에 ingress 를 등록해야한다. (Domain이 일치해야 UI에서 인증을 사용할 수 있음.)
$ vi registry-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: registry
nginx.ingress.kubernetes.io/proxy-body-size: 4096m # 이미지 업로드 용량으로 반드시 설정필요하며 필요에 따라 설정
spec:
ingressClassName: nginx
rules:
- host: registry.wky.kr
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: registry
port:
number: 5000
또한, 위에 생성한 Deployment에 환경변수를 추가해야한다.
$ kubectl edit deploy -n registry registry
...
- env:
- name: REGISTRY_HTTP_HEADERS_Access-Control-Allow-Origin # 추가
value: '[http://registry-ui.wky.kr:30080]'
- name: REGISTRY_HTTP_HEADERS_Access-Control-Allow-Methods # 추가
value: '[HEAD,GET,OPTIONS,DELETE]'
- name: REGISTRY_HTTP_HEADERS_Access-Control-Allow-Credentials # 추가
value: '[true]'
- name: REGISTRY_HTTP_HEADERS_Access-Control-Allow-Headers # 추가
value: '[Authorization,Accept]'
- name: REGISTRY_HTTP_HEADERS_Access-Control-Expose-Headers # 추가
value: '[Docker-Content-Digest]'
- name: REGISTRY_STORAGE_DELETE_ENABLED
value: "true"
- name: REGISTRY_AUTH_HTPASSWD_PATH
value: /auth/idpass
- name: REGISTRY_AUTH_HTPASSWD_REALM
value: Registry Realm
...
UI 설치
특히 아래 yaml의 NGINX_PROXY_PASS_URL은 k8s service domain을 넣거나 nodeport를 넣거나 ingress를 넣어주면된다.
$ vi registry-ui.yaml
$ kubectl apply -f registry-ui -n registry
apiVersion: apps/v1
kind: Deployment
metadata:
name: registry-ui
labels:
reg: registry-ui
spec:
replicas: 1
selector:
matchLabels:
reg: registry-ui
template:
metadata:
labels:
reg: registry-ui
spec:
containers:
- name: registry-ui
image: joxit/docker-registry-ui:main
imagePullPolicy: IfNotPresent
ports:
- containerPort: 80
env:
- name: SINGLE_REGISTRY
value: "true"
- name: REGISTRY_TITLE
value: Docker Registry UI
- name: DELETE_IMAGES
value: "true"
- name: SHOW_CONTENT_DIGEST
value: "true"
- name: SHOW_CATALOG_NB_TAGS
value: "true"
- name: CATALOG_MIN_BRANCHES
value: "1"
- name: CATALOG_MAX_BRANCHES
value: "1"
- name: TAGLIST_PAGE_SIZE
value: "100"
- name: REGISTRY_SECURED
value: "true"
- name: CATALOG_ELEMENTS_LIMIT
value: "1000"
- name: NGINX_PROXY_PASS_URL
value: http://registry.registry:5000 # http://registry.wky.kr:30050 또는 http://registry.wky.kr:30080
- name: REGISTRY_SECURED
value: "true"
---
apiVersion: v1
kind: Service
metadata:
name: registry-ui
spec:
type: NodePort
ports:
- port: 80
targetPort: 80
nodePort: 30051
protocol: TCP
selector:
reg: registry-ui
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: registry-ui
spec:
ingressClassName: nginx
rules:
- host: registry-ui.wky.kr
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: registry-ui
port:
number: 80
결과 확인
기타 : https 로 여는법
사설 인증서를 생성하고, 사설인증서이기 때문에 해당 registry를 실제 사용할 서버를 위해 ca파일도 생성한다.
# 사설 인증서 생성
$ openssl req -newkey rsa:4096 -nodes -sha256 -keyout domain.key -subj "/CN=registry.wky.kr" -addext "subjectAltName=DNS:registry.wky.kr" -x509 -days 365 -out domain.crt
# 이미지 push 또는 pull 위한 ca 파일 생성
$ cat domain.crt domain.key > domain.pem
registry에 마운트할 secret과 ingress에 등록할 secert 2개를 생성한다.
# registry 인증서용
$ kubectl create secret generic https-registry --from-file=domain.crt=./domain.crt --from-file=domain.key=./domain.key -n registry
# registry ingress용
$ kubectl create secret tls tls-registry --key ./domain.key --cert ./domain.crt -n registry
deployment와 service, ingress 수정
# deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: registry
labels:
reg: registry
spec:
replicas: 1
selector:
matchLabels:
reg: registry
template:
metadata:
labels:
reg: registry
spec:
containers:
- name: registry
image: registry:2
imagePullPolicy: IfNotPresent
ports:
- containerPort: 443 # 443으로 수정
volumeMounts:
- name: registry-auth
mountPath: /auth
- name: registry-data
mountPath: /var/lib/registry
- name: https-registry # 인증서파일 마운트
mountPath: /tmp
env:
- name: REGISTRY_HTTP_HEADERS_Access-Control-Allow-Origin
value: '[http://registry-ui.wky.kr:30080]'
- name: REGISTRY_HTTP_HEADERS_Access-Control-Allow-Methods
value: '[HEAD,GET,OPTIONS,DELETE]'
- name: REGISTRY_HTTP_HEADERS_Access-Control-Allow-Credentials
value: '[true]'
- name: REGISTRY_HTTP_HEADERS_Access-Control-Allow-Headers
value: '[Authorization,Accept]'
- name: REGISTRY_HTTP_HEADERS_Access-Control-Expose-Headers
value: '[Docker-Content-Digest]'
- name: REGISTRY_STORAGE_DELETE_ENABLED
value: "true"
- name: REGISTRY_AUTH_HTPASSWD_PATH
value: /auth/idpass
- name: REGISTRY_AUTH_HTPASSWD_REALM
value: Registry Realm
- name: REGISTRY_STORAGE_DELETE_ENABLED
value: "true"
- nmae: REGISTRY_HTTP_TLS_CERTIFICATE # 인증서 설정
value: /tmp/domain.crt
- name: REGISTRY_HTTP_TLS_KEY # 인증서 설정
value: /tmp/domain.key
- name: REGISTRY_HTTP_ADDR # 443으로 포트 열기
value: 0.0.0.0:443
volumes:
- name: registry-auth
secret:
secretName: registry-auth
- name: registry-data
persistentVolumeClaim:
claimName: registry-pvc
- name: https-registry # 인증서 secret 마운트
secret:
secretName: https-registry
# serivce
apiVersion: v1
kind: Service
metadata:
name: registry
spec:
type: ClusterIP # NodePort도 관계없음.
ports:
- port: 443 # 443으로 수정
targetPort: 443 # 443으로 수정
protocol: TCP
selector:
reg: registry
# ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: registry
nginx.ingress.kubernetes.io/backend-protocol: HTTPS # HTTPS 통신
nginx.ingress.kubernetes.io/proxy-body-size: 4096m
spec:
ingressClassName: nginx
rules:
- host: registry.wky.kr
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: registry
port:
number: 443
tls: # tls secret 마운트
- hosts:
- registry.wky.kr
secretName: tls-registry
참고 :
https://distribution.github.io/distribution/
https://github.com/Joxit/docker-registry-ui
반응형
'Develop > k8s' 카테고리의 다른 글
| Containerd x509: certificate signed by unknown authority 설정 (0) | 2024.11.19 |
|---|---|
| Container 실행없이 이미지 내부 확인 ctr image mount (0) | 2024.11.18 |
| kubernetes dashboard ingress 연결 방법 (0) | 2024.10.22 |
| k8s config 파일 만료일자 확인 (0) | 2024.10.17 |
| serviceAccount 사용중인 pod 조회 (field-selector) (0) | 2024.09.23 |
