이러다 설치만 장인되겠다
해당 글은 helm chart로 설치하는 정리 가이드이다.
가이드 작성시 상세 설명은 따로 하지 않는다.
OpenSearch Operator로 설치는 가이드보면 설치자체는 어려운것은 없으나, 인증서 부분에서 상당히 복잡하기 때문에 아래 글 참고한다.
2024.11.28 - [Develop/OpenSearch ／ FluentBit] - OpenSearch Operator Certificate 설정
Single Node 로 설치는 아래 글 참고한다.
2025.03.06 - [Develop/OpenSearch ／ FluentBit] - OpenSearch SingleNode
OpenSearch FluentBit 연동은 아래 글 참고한다.
2024.11.03 - [Develop/OpenSearch ／ FluentBit] - OpenSearch, Dashboard 설치 및 Fluent-Bit OpenSeach 연동
OpenSearch Operator Coordinator 설치는 아래 글 참고한다.
2025.02.18 - [Develop/OpenSearch ／ FluentBit] - OpenSearch Operator Coordinator
1. Demo 인증서가아닌 self 인증서를 생성한다. 인증서가 있으면 건너 뛴다.
# root ca 생성
openssl genrsa -out root-ca-key.pem 2048
openssl req -new -x509 -sha256 -key root-ca-key.pem -subj "/CN=opensearch" -out root-ca.pem -days 3650
# OpenSearch Operator가 OpenSearch Cluster를 관리하는 관리자 인증서
openssl genrsa -out admin-key-temp.pem 2048
openssl pkcs8 -inform PEM -outform PEM -in admin-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out admin-key.pem
openssl req -new -key admin-key.pem -subj "/CN=opensearch-admin" -out admin.csr
openssl x509 -req -in admin.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out admin.pem -days 3650
# Node간 DNS 통신을 위해 subjectAltName 생성
echo 'subjectAltName=DNS:opensearch-cluster-master,DNS:opensearch-cluster-master.logging,DNS:opensearch-cluster-master.logging.svc,DNS:opensearch-cluster-master.logging.svc.cluster.local' > san.ext
# OpenSearch Node간 인증서
openssl genrsa -out node-key-temp.pem 2048
openssl pkcs8 -inform PEM -outform PEM -in node-key-temp.pem -topk8 -nocrypt -v1 PBE-SHA1-3DES -out node-key.pem
openssl req -new -key node-key.pem -subj "/CN=opensearch-cluster-master" -out node.csr
openssl x509 -req -in node.csr -CA root-ca.pem -CAkey root-ca-key.pem -CAcreateserial -sha256 -out node.pem -days 3650 -extfile san.ext
# OpenSearch yaml에 등록할 인증서 파일들 secret으로 생성
kubectl create ns logging
kubectl create secret generic opensearch-cert --from-file=root-ca.pem=root-ca.pem --from-file=admin.pem=admin.pem --from-file=admin-key.pem=admin-key.pem \
--from-file=root-ca.pem=root-ca.pem --from-file=node.pem=node.pem --from-file=node-key.pem=node-key.pem -n logging
2. helm repo 추가
공식 가이드에 나온대로 진행한다.
helm repo add opensearch https://opensearch-project.github.io/helm-charts
helm repo upadte
3. OpenSearch에 사용할 관리자 계정 secret 생성
아래 링크에서 가져온 샘플이다. 필요에 따라 수정해서 사용하면 됨.
https://github.com/opensearch-project/opensearch-k8s-operator/blob/main/opensearch-operator/examples/securityconfig-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: securityconfig-secret
type: Opaque
stringData:
action_groups.yml: |-
_meta:
type: "actiongroups"
config_version: 2
internal_users.yml: |-
_meta:
type: "internalusers"
config_version: 2
admin:
hash: "$2y$12$lJsHWchewGVcGlYgE3js/O4bkTZynETyXChAITarCHLz8cuaueIyq"
reserved: true
backend_roles:
- "admin"
description: "admin user"
dashboarduser:
hash: "$2a$12$4AcgAt3xwOWadA5s5blL6ev39OXDNhmOesEoo33eZtrq2N0YrU3H."
reserved: true
description: "OpenSearch Dashboards user"
nodes_dn.yml: |-
_meta:
type: "nodesdn"
config_version: 2
whitelist.yml: |-
_meta:
type: "whitelist"
config_version: 2
tenants.yml: |-
_meta:
type: "tenants"
config_version: 2
roles_mapping.yml: |-
_meta:
type: "rolesmapping"
config_version: 2
all_access:
reserved: false
backend_roles:
- "admin"
description: "Maps admin to all_access"
own_index:
reserved: false
users:
- "*"
description: "Allow full access to an index named like the username"
readall:
reserved: false
backend_roles:
- "readall"
manage_snapshots:
reserved: false
backend_roles:
- "snapshotrestore"
dashboard_server:
reserved: true
users:
- "dashboarduser"
roles.yml: |-
_meta:
type: "roles"
config_version: 2
dashboard_read_only:
reserved: true
security_rest_api_access:
reserved: true
# Allows users to view monitors, destinations and alerts
alerting_read_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/alerting/alerts/get'
- 'cluster:admin/opendistro/alerting/destination/get'
- 'cluster:admin/opendistro/alerting/monitor/get'
- 'cluster:admin/opendistro/alerting/monitor/search'
# Allows users to view and acknowledge alerts
alerting_ack_alerts:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/alerting/alerts/*'
# Allows users to use all alerting functionality
alerting_full_access:
reserved: true
cluster_permissions:
- 'cluster_monitor'
- 'cluster:admin/opendistro/alerting/*'
index_permissions:
- index_patterns:
- '*'
allowed_actions:
- 'indices_monitor'
- 'indices:admin/aliases/get'
- 'indices:admin/mappings/get'
# Allow users to read Anomaly Detection detectors and results
anomaly_read_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/ad/detector/info'
- 'cluster:admin/opendistro/ad/detector/search'
- 'cluster:admin/opendistro/ad/detectors/get'
- 'cluster:admin/opendistro/ad/result/search'
- 'cluster:admin/opendistro/ad/tasks/search'
- 'cluster:admin/opendistro/ad/detector/validate'
- 'cluster:admin/opendistro/ad/result/topAnomalies'
# Allows users to use all Anomaly Detection functionality
anomaly_full_access:
reserved: true
cluster_permissions:
- 'cluster_monitor'
- 'cluster:admin/opendistro/ad/*'
index_permissions:
- index_patterns:
- '*'
allowed_actions:
- 'indices_monitor'
- 'indices:admin/aliases/get'
- 'indices:admin/mappings/get'
# Allows users to read Notebooks
notebooks_read_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/notebooks/list'
- 'cluster:admin/opendistro/notebooks/get'
# Allows users to all Notebooks functionality
notebooks_full_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/notebooks/create'
- 'cluster:admin/opendistro/notebooks/update'
- 'cluster:admin/opendistro/notebooks/delete'
- 'cluster:admin/opendistro/notebooks/get'
- 'cluster:admin/opendistro/notebooks/list'
# Allows users to read observability objects
observability_read_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opensearch/observability/get'
# Allows users to all Observability functionality
observability_full_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opensearch/observability/create'
- 'cluster:admin/opensearch/observability/update'
- 'cluster:admin/opensearch/observability/delete'
- 'cluster:admin/opensearch/observability/get'
# Allows users to read and download Reports
reports_instances_read_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/reports/instance/list'
- 'cluster:admin/opendistro/reports/instance/get'
- 'cluster:admin/opendistro/reports/menu/download'
# Allows users to read and download Reports and Report-definitions
reports_read_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/reports/definition/get'
- 'cluster:admin/opendistro/reports/definition/list'
- 'cluster:admin/opendistro/reports/instance/list'
- 'cluster:admin/opendistro/reports/instance/get'
- 'cluster:admin/opendistro/reports/menu/download'
# Allows users to all Reports functionality
reports_full_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/reports/definition/create'
- 'cluster:admin/opendistro/reports/definition/update'
- 'cluster:admin/opendistro/reports/definition/on_demand'
- 'cluster:admin/opendistro/reports/definition/delete'
- 'cluster:admin/opendistro/reports/definition/get'
- 'cluster:admin/opendistro/reports/definition/list'
- 'cluster:admin/opendistro/reports/instance/list'
- 'cluster:admin/opendistro/reports/instance/get'
- 'cluster:admin/opendistro/reports/menu/download'
# Allows users to use all asynchronous-search functionality
asynchronous_search_full_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/asynchronous_search/*'
index_permissions:
- index_patterns:
- '*'
allowed_actions:
- 'indices:data/read/search*'
# Allows users to read stored asynchronous-search results
asynchronous_search_read_access:
reserved: true
cluster_permissions:
- 'cluster:admin/opendistro/asynchronous_search/get'
# Allows user to use all index_management actions - ism policies, rollups, transforms
index_management_full_access:
reserved: true
cluster_permissions:
- "cluster:admin/opendistro/ism/*"
- "cluster:admin/opendistro/rollup/*"
- "cluster:admin/opendistro/transform/*"
index_permissions:
- index_patterns:
- '*'
allowed_actions:
- 'indices:admin/opensearch/ism/*'
# Allows users to use all cross cluster replication functionality at leader cluster
cross_cluster_replication_leader_full_access:
reserved: true
index_permissions:
- index_patterns:
- '*'
allowed_actions:
- "indices:admin/plugins/replication/index/setup/validate"
- "indices:data/read/plugins/replication/changes"
- "indices:data/read/plugins/replication/file_chunk"
# Allows users to use all cross cluster replication functionality at follower cluster
cross_cluster_replication_follower_full_access:
reserved: true
cluster_permissions:
- "cluster:admin/plugins/replication/autofollow/update"
index_permissions:
- index_patterns:
- '*'
allowed_actions:
- "indices:admin/plugins/replication/index/setup/validate"
- "indices:data/write/plugins/replication/changes"
- "indices:admin/plugins/replication/index/start"
- "indices:admin/plugins/replication/index/pause"
- "indices:admin/plugins/replication/index/resume"
- "indices:admin/plugins/replication/index/stop"
- "indices:admin/plugins/replication/index/update"
- "indices:admin/plugins/replication/index/status_check"
config.yml: |-
_meta:
type: "config"
config_version: "2"
config:
dynamic:
http:
anonymous_auth_enabled: false
authc:
basic_internal_auth_domain:
http_enabled: true
transport_enabled: true
order: "4"
http_authenticator:
type: basic
challenge: true
authentication_backend:
type: intern
4. OpenSearch Dashboards에 등록할 관리자 계정 Secret생성
vi admin-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: admin-credentials
namespace: logging
type: Opaque
data:
username: YWRtaW4= # admin
password: YWRtaW4xMjM= # admin123
kubectl apply -f admin-secret.yaml
5. OpenSearch 설치
기본적으로 helm chart에 values.yaml 을 수정해서 적용하는 것을 추천하나, 이번엔 바로 적용을 해본다.
인증서를 mount하고, opensearch.yml을 정의하고 securityConfigSecret을 추가하여 관리자 계정을 생성한다.
helm install opensearch-cluster opensearch/opensearch \
--namespace logging --create-namespace
--set clusterName='opensearch-cluster'
--set persistence.storageClass=nfs \ # StorgeClass 가 있으면 설정 없으면 삭제
--set persistence.enabled=false \ # EmptyDir로 실행하고 싶으면 설정 그게아니라면 따로 PV설정필요
# 기본적으로 아래로 적용
--set-json 'extraEnvs=[{"name": "DISABLE_INSTALL_DEMO_CONFIG", "value": "true"}] \
# SingleNode로 설치시 lifecycle 이용할려면 아래로 적용
--set-json 'extraEnvs=[{"name": "DISABLE_INSTALL_DEMO_CONFIG", "value": "true"}
{"name": "OPENSEARCH_USERNAME", "valueFrom": {"secretKeyRef": {"key": "username",
"name": "admin-credentials-secret"}}},
{"name": "OPENSEARCH_PASSWORD", "valueFrom": {"secretKeyRef": {"key": "password",
"name": "admin-credentials-secret"}}}]' \
--set-json 'extraVolumes=[{"name": "certs-secret" "secret": {"defaultMode": 384,
"secretName": "opensearch-cert"}}] \
--set-json 'extraVolumeMounts=[{"mountPath": "/usr/share/opensearch/config/certs",
"name": "certs-secret"}]' \
--set config.opensearch.yml="cluster.name: opensearch-cluster
plugins.security.ssl.transport.pemcert_filepath: certs/node.pem
plugins.security.ssl.transport.pemkey_filepath: certs/node-key.pem
plugins.security.ssl.transport.pemtrustedcas_filepath: certs/root-ca.pem
plugins.security.ssl.http.pemcert_filepath: certs/node.pem
plugins.security.ssl.http.pemkey_filepath: certs/node-key.pem
plugins.security.ssl.http.pemtrustedcas_filepath: certs/root-ca.pem
plugins.security.allow_unsafe_democertificates: false
plugins.security.allow_default_init_securityindex: true
plugins.security.authz.admin_dn: ['CN=opensearch-admin']
plugins.security.nodes_dn: ['CN=opensearch*']
plugins.security.audit.type: internal_opensearch
plugins.security.enable_snapshot_restore_privilege: true
plugins.security.check_snapshot_restore_write_privilege: true
plugins.security.restapi.roles_enabled: [all_access, security_rest_api_access]
plugins.security.system_indices.enabled: true
plugins.security.system_indices.indices: ['.opendistro-alerting-config', '.opendistro-alerting-alert*', '.opendistro-anomaly-results*', '.opendistro-anomaly-detector*', '.opendistro-anomaly-checkpoints', '.opendistro-anomaly-detection-state', '.opendistro-reports-*", ".opendistro-notifications-*', '.opendistro-notebooks', '.opendistro-asynchronous-search-response*']
node.max_local_storage_nodes: 3' # Single Node 이면 삭제하고 따옴표 주의
--set securityConfig.config.securityConfigSecret=securityconfig-secret"
기타.
https://opensearch.org/docs/latest/security/configuration/system-indices/
를 확인하면 관리자 계정으로도 사용할 수 없는 api들이 있음.
5. OpenSearch Dashboards 설치
4번에서 생성한 secret을 넣어준다.
helm install opensearch-dashboards opensearch/opensearch-dashboards \
-n logging --create-namespace \
--set-json 'extraEnvs=[{"name": "OPENSEARCH_USERNAME", "valueFrom": {"secretKeyRef":
{"key": "username", "name": "admin-credentials"}}},
{"name": "OPENSEARCH_PASSWORD", "valueFrom": {"secretKeyRef":
{"key": "password", "name": "admin-credentials"}}}]'
Ingress 설정은 맨위에 OpenSearch FluentBit 연동글을 참고한다.
