2020-10-13
在k8s中搭建es集群的思路如下圖：
k8s搭建es服務集群結構示意圖
service1、service2、service3用于集群內服務互通，service用于集群服務對外提供訪問。
方式一：使用自定義存儲卷
statefulset的部署文件 es-statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: es-sts-01
spec:
selector:
matchLabels:
app: es
es-cluster: es01
serviceName: "elasticsearch-svc"
replicas: 1
template:
metadata:
labels:
app: es
es-cluster: es01
spec:
terminationGracePeriodSeconds: 10
volumes:
- name: data-storage
persistentVolumeClaim:
claimName: es-nfs-pvc-1 #使用第一塊存儲卷
containers:
- name: elasticsearch
image: docker.elastic.co/elasticsearch/elasticsearch:7.9.2
ports:
- containerPort: 9200
name: es-cli
- containerPort: 9300
name: es-inner
env:
- name: cluster.name
value: es-luster
- name: node.name
value: es01
- name: cluster.initial_master_nodes #用于第一次啟動時引導集群的創建，集群組建成功后就可以不需要了
value: es01,es02,es03 #對應各個es的node.name
- name: discovery.seed_hosts
value: es-svc-inner-02,es-svc-inner-02
- name: http.cors.enabled
value: "true"
- name: http.cors.allow-origin
value: "*"
volumeMounts:
- mountPath: "/usr/share/elasticsearch/data"
name: data-storage
---
kind: StatefulSet
metadata:
name: es-sts-02
spec:
selector:
matchLabels:
app: es
es-cluster: es02
serviceName: "elasticsearch-svc"
replicas: 1
template:
metadata:
labels:
app: es
es-cluster: es02
spec:
terminationGracePeriodSeconds: 10
volumes:
- name: data-storage
persistentVolumeClaim:
claimName: es-nfs-pvc-2 #使用第二塊存儲卷
containers:
- name: es-02
image: docker.elastic.co/elasticsearch/elasticsearch:7.9.2
ports:
- containerPort: 9200
name: es-cli
- containerPort: 9300
name: es-inner
env:
- name: cluster.name
value: es-cluster
- name: node.name
value: es02
- name: cluster.initial_master_nodes # 最好每個都要配置這個列表，而且要與es01配置的一致
value: es01,es02,es03
- name: discovery.seed_hosts
value: es-svc-inner-01,es-svc-inner-03
- name: http.cors.enabled
value: "true"
- name: http.cors.allow-origin
value: "*"
volumeMounts:
- mountPath: "/usr/share/elasticsearch/data"
name: data-storage
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: es-sts-03
spec:
selector:
matchLabels:
app: es
es-cluster: es03
serviceName: "elasticsearch-svc"
replicas: 1
template:
metadata:
labels:
app: es
es-cluster: es03
spec:
terminationGracePeriodSeconds: 10
volumes:
- name: data-storage
persistentVolumeClaim:
claimName: es-nfs-pvc-3 #使用第三塊存儲卷
containers:
- name: es-03
image: docker.elastic.co/elasticsearch/elasticsearch:7.9.2
ports:
- containerPort: 9200
name: es-cli
- containerPort: 9300
name: es-iner
env:
- name: cluster.name
value: es-cluster
- name: node.name
value: es03
- name: cluster.initial_master_nodes # 最好每個都要配置這個列表，而且要與es01配置的一致
- name: discovery.seed_hosts
value: es-svc-inner-01,es-svc-inner-02
- name: http.cors.enabled
value: "true"
- name: http.cors.allow-origin
value: "*"
volumeMounts:
- mountPath: "/usr/share/elasticsearch/data"
name: data-storage
存儲卷的部署：persistant-voulmes.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
name: es-nfs-pv-1
labels:
pv: es-nfs-pv-1
spec:
capacity:
storage: 1Gi
#volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
storageClassName: nfs
nfs:
path: /data/nfs/es
server: k8snode-01
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: es-nfs-pv-2
labels:
pv: es-nfs-pv-2
spec:
capacity:
storage: 1Gi
#volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
storageClassName: nfs
nfs:
path: /data/nfs/es
server: k8snode-02
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: es-nfs-pv-3
labels:
pv: es-nfs-pv-3
spec:
capacity:
storage: 1Gi
#volumeMode: Filesystem
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Recycle
storageClassName: nfs
nfs:
path: /data/nfs/es
server: k8snode-03
##claim部署
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: es-nfs-pvc-1
spec:
accessModes:
- ReadWriteOnce
#volumeMode: Filesystem
resources:
requests:
storage: 1Gi
storageClassName: nfs
selector:
matchLabels:
pv: "es-nfs-pv-1"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: es-nfs-pvc-2
spec:
accessModes:
- ReadWriteOnce
#volumeMode: Filesystem
resources:
requests:
storage: 1Gi
storageClassName: nfs
selector:
matchLabels:
pv: "es-nfs-pv-2"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: es-nfs-pvc-3
spec:
accessModes:
- ReadWriteOnce
#volumeMode: Filesystem
resources:
requests:
storage: 1Gi
storageClassName: nfs
selector:
matchLabels:
pv: "es-nfs-pv-3"
注：存儲卷的部署不成功的問題，提供下思路：需要對應的服務器上啟動nfs服務和相關路徑的讀寫權限。
Service的部署：es-service.yaml
##用于對外提供服務的Service
apiVersion: v1
kind: Service
metadata:
name: elasticsearch-svc
labels:
app: elasticsearch-svc
spec:
type: NodePort
ports:
- port: 9200
name: es-cli
nodePort: 30001
selector:
app: es
---
##用于集群內部服務間互通的Service
apiVersion: v1
kind: Service
metadata:
name: es-svc-inner-01
labels:
app: es-svc-inner-01
spec:
type: ClusterIP
ports:
- port: 9300
name: es-port
selector:
es-cluster: es01
---
apiVersion: v1
kind: Service
metadata:
name: es-svc-inner-02
labels:
app: es-svc-inner-02
spec:
type: ClusterIP
ports:
- port: 9300
name: es-port
selector:
es-cluster: es02
---
apiVersion: v1
kind: Service
metadata:
name: es-svc-inner-03
labels:
app: es-svc-inner-03
spec:
type: ClusterIP
ports:
- port: 9300
name: es-port
selector:
es-cluster: es03
方式二：使用阿里云存儲
使用阿里云存儲就不需要上面方試一那樣麻創建PV和PVC了。
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: es-sts-01
namespace: elasticsearch
spec:
selector:
matchLabels:
app: es
es-cluster: es01
serviceName: "elasticsearch-svc"
replicas: 1 # by default is 1
template:
metadata:
labels:
app: es
es-cluster: es01
spec:
securityContext:
fsGroup: 1000 #使用這個獲得讀寫權限
terminationGracePeriodSeconds: 10
volumes:
- name: aliyun-nas-01
flexVolume:
driver: "alicloud/nas"
options:
server: "xxxxxx.xxxxxx.nas.aliyuncs.com"
path: "/elasticsearch/test/node01"
vers: "3"
containers:
- name: elasticsearch
image: docker.elastic.co/elasticsearch/elasticsearch:7.9.2
ports:
- containerPort: 9200
name: es-cli
- containerPort: 9300
name: es-iner
env:
- name: cluster.name
value: es-cluster
- name: node.name
value: es01
- name: cluster.initial_master_nodes
value: es01,es02,es03
- name: discovery.seed_hosts
value: es-svc-inner-02,es-svc-inner-02
- name: http.cors.enabled
value: "true"
- name: http.cors.allow-origin
value: "*"
volumeMounts:
- mountPath: "/usr/share/elasticsearch/data"
name: aliyun-nas-01
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: es-sts-02
namespace: elasticsearch
spec:
selector:
matchLabels:
app: es
es-cluster: es02
serviceName: "elasticsearch-svc"
replicas: 1
template:
metadata:
labels:
app: es
es-cluster: es02
spec:
securityContext:
fsGroup: 1000
terminationGracePeriodSeconds: 10
volumes:
- name: aliyun-nas-02
flexVolume:
driver: "alicloud/nas"
options:
server: "xxxxxx.xxxxxx.nas.aliyuncs.com"
path: "/elasticsearch/test/node02"
vers: "3"
containers:
- name: es-02
image: docker.elastic.co/elasticsearch/elasticsearch:7.9.2
ports:
- containerPort: 9200
name: es-cli
- containerPort: 9300
name: es-iner
env:
- name: cluster.name
value: es-cluster
- name: node.name
value: es02
- name: cluster.initial_master_nodes
value: es01,es02,es03
- name: discovery.seed_hosts
value: es-svc-inner-01,es-svc-inner-03
- name: http.cors.enabled
value: "true"
- name: http.cors.allow-origin
value: "*"
volumeMounts:
- mountPath: "/usr/share/elasticsearch/data"
name: aliyun-nas-02
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: es-sts-03
namespace: elasticsearch
spec:
selector:
matchLabels:
app: es
es-cluster: es03
serviceName: "elasticsearch-svc"
replicas: 1
template:
metadata:
labels:
app: es
es-cluster: es03
spec:
securityContext:
fsGroup: 1000
terminationGracePeriodSeconds: 10
volumes:
- name: aliyun-nas-03
flexVolume:
driver: "alicloud/nas"
options:
server: "xxxxxx.xxxxxx.nas.aliyuncs.com"
path: "/elasticsearch/test/node03"
vers: "3"
containers:
- name: es-03
image: docker.elastic.co/elasticsearch/elasticsearch:7.9.2
ports:
- containerPort: 9200
name: es-cli
- containerPort: 9300
name: es-iner
env:
- name: cluster.name
value: es-cluster
- name: node.name
value: es03
- name: cluster.initial_master_nodes
value: es01,es02,es03
- name: discovery.seed_hosts
value: es-svc-inner-01,es-svc-inner-02
- name: http.cors.enabled
value: "true"
- name: http.cors.allow-origin
value: "*"
volumeMounts:
- mountPath: "/usr/share/elasticsearch/data"
name: aliyun-nas-03
經驗證，需要搭建三個es實例才能保證其中一個掛了之后集群還能正常運行，官方也推薦最好配置三個以上的mater node，以防數據的丟失。
ps: Elasticsearch默認單機情況下會自動引導組建集群，但是只要配置了
discovery.seed_providers
discovery.seed_hosts
cluster.initial_master_nodes
三個中的一個Elasticsearch就不會自動引導集群了。
