参考: https://kubernetes.io/zh/docs/concepts/workloads/controllers/statefulset/
StatefulSet 是用来管理有状态应用的控制器。
无状态应用与有状态应用无状态应用: 如nginx
请求本身包含了响应端为响应这一请求所需的全部信息。每一个请求都像首次执行一样，不会依赖之前的数据进行响应。不需要持久化的数据无状态应用的多个实例之间互不依赖，可以无序的部署、删除或伸缩
有状态应用: 如mysql
前后请求有关联与依赖需要持久化的数据有状态应用的多个实例之间有依赖，不能相互替换：无论怎么调度，每个 Pod 都有一个永久不变的 ID。 StatefulSet的特点
稳定的、唯一的网络标识符。 (通过headless服务实现)稳定的、持久的存储。 (通过PV，PVC，storageclass实现)有序的、优雅的部署和缩放。有序的、自动的滚动更新。 StatefulSet的YAML组成
需要三个组成部分:
headless service: 实现稳定，唯一的网络标识statefulset类型资源: 写法和deployment几乎一致，就是类型不一样volumeClaimTemplate : 指定存储卷 二、nginx+statefulset+nfs案例 创建StatefulSet应用参考: https://kubernetes.io/zh/docs/tutorials/stateful-application/basic-stateful-set/
创建statelfulset应用来调用名为managed-nfs-storage的storageclass,以实现动态供给
[root@master1 ~]# vim nginx-storageclass-nfs.ymlapiVersion: v1kind: Servicemetadata: name: nginxspec: ports: - port: 80 name: web clusterIP: None # 无头服务 selector: app: nginx---apiVersion: apps/v1kind: StatefulSetmetadata: name: web# statefulset的名称spec: serviceName: "nginx"# 服务名与上面的无头服务名要一致 replicas: 3# 3个副本 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.15-alpine volumeMounts: - name: www mountPath: /usr/share/nginx/html volumeClaimTemplates: - metadata: name: www spec: accessModes: [ "ReadWriteOnce" ] storageClassName: "managed-nfs-storage"# 与前面定义的storageclass名称对应 resources: requests: storage: 1Gi
[root@master1 ~]# kubectl apply -f nginx-storageclass-nfs.ymlservice/nginx createdstatefulset.apps/web created
[root@master1 ~]# kubectl get statefulsets# 可以简写成stsNAME READY AGEweb 3/3 1m
验证pod,pv,pvc产生了3个pod
[root@master1 ~]# kubectl get pods |grep webweb-0 1/1 Running 0 1m15sweb-1 1/1 Running 0 1m7sweb-2 1/1 Running 0 57s
自动产生了3个pv
[root@master1 ~] # kubectl get pvpvc-2436b20d-1be3-4c2e-87a9-5533e5c5e2c6 1Gi RWO Delete Bound default/www-web-0 managed-nfs-storage 3mpvc-3114be74-5969-40eb-aeb3-87a3b9ae17bc 1Gi RWO Delete Bound default/www-web-1 managed-nfs-storage 2mpvc-43afb71d-1d02-4699-b00c-71679fd75fc3 1Gi RWO Delete ound default/www-web-2 managed-nfs-storage 2m
自动产生了3个PVC
[root@master1 ~]# kubectl get pvc |grep webwww-web-0 Bound pvc-2436b20d-1be3-4c2e-87a9-5533e5c5e2c6 1Gi RWO managed-nfs-storage 3mwww-web-1 Bound pvc-3114be74-5969-40eb-aeb3-87a3b9ae17bc 1Gi RWO managed-nfs-storage 2mwww-web-2 Bound pvc-43afb71d-1d02-4699-b00c-71679fd75fc3 1Gi RWO managed-nfs-storage 2m
验证nfs服务目录在nfs服务器（这里为hostos)的共享目录中发现自动产生了3个子目录
[root@hostos ~]# ls /data/nfs/default-www-web-0-pvc-2436b20d-1be3-4c2e-87a9-5533e5c5e2c6 default-www-web-2-pvc-43afb71d-1d02-4699-b00c-71679fd75fc3default-www-web-1-pvc-3114be74-5969-40eb-aeb3-87a3b9ae17bc
3个子目录默认都为空目录
[root@hostos ~]# tree /data/nfs//data/nfs/├── default-www-web-0-pvc-2436b20d-1be3-4c2e-87a9-5533e5c5e2c6├── default-www-web-1-pvc-3114be74-5969-40eb-aeb3-87a3b9ae17bc└── default-www-web-2-pvc-43afb71d-1d02-4699-b00c-71679fd75fc3
验证存储持久性在3个pod中其中一个创建一个主页文件
[root@master1 ~]# kubectl exec -it web-0 -- /bin/sh/ # echo "haha" > /usr/share/nginx/html/index.html/ # exit
在nfs服务器上发现文件被创建到了对应的目录中
[root@hostos ~]# tree /data/nfs//data/nfs/├── default-www-web-0-pvc-2436b20d-1be3-4c2e-87a9-5533e5c5e2c6│ └── index.html# 此目录里多了index.html文件，对应刚才在web-0的pod中的创建├── default-www-web-1-pvc-3114be74-5969-40eb-aeb3-87a3b9ae17bc└── default-www-web-2-pvc-43afb71d-1d02-4699-b00c-71679fd75fc3[root@hostos ~]# cat /data/nfs/default-www-web-0-pvc-2436b20d-1be3-4c2e-87a9-5533e5c5e2c6/index.htmlhaha# 文件内的内容也与web-0的pod中创建的一致
删除web-0这个pod,再验证
[root@master1 ~]# kubectl delete pod web-0pod "web-0" deleted[root@master1 ~]# kubectl get pods |grep web# 因为控制器的原因，会迅速再拉起web-0这个podweb-0 1/1 Running 0 9s # 时间上看到是新拉起的podweb-1 1/1 Running 0 37mweb-2 1/1 Running 0 37m[root@master1 ~]# kubectl exec -it web-0 -- cat /usr/share/nginx/html/index.htmlhaha# 新拉起的pod仍然是相同的存储数据[root@hostos ~]# cat /data/nfs/default-www-web-0-pvc-2436b20d-1be3-4c2e-87a9-5533e5c5e2c6/index.htmlhaha# nfs服务器上的数据还在
结论: 说明数据可持久化
验证pod唯一名称回顾域名格式:
service: ..svc.cluster.local.
pod: ...svc.cluster.local.
可以看到在web-0这个pod中，nslookup查询service的域名，直接解析成了3个pod的域名
[root@master1 ~]# kubectl exec -it web-0 -- /bin/sh/ # nslookup nginx-svc.default.svc.cluster.local.Name: nginx-svc.default.svc.cluster.local.Address 1: 10.3.104.48 web-0.nginx-svc.default.svc.cluster.local.Address 2: 10.3.104.47 web-2.nginx-svc.default.svc.cluster.localAddress 3: 10.3.166.185 web-1.nginx-svc.default.svc.cluster.local
ping这三个pod的域名都可以ping通
/ # ping web-0.nginx-svc.default.svc.cluster.local.PING web-0.nginx-svc.default.svc.cluster.local、(10.3.104.48): 56 data bytes64 bytes from 10.3.104.48: seq=0 ttl=64 time=0.095 ms64 bytes from 10.3.104.48: seq=1 ttl=64 time=0.159 ms......
/ # ping web-1.nginx-svc.default.svc.cluster.local.PING web-1.nginx-svc.default.svc.cluster.local、(10.3.166.185): 56 data bytes64 bytes from 10.3.166.185: seq=0 ttl=62 time=0.700 ms64 bytes from 10.3.166.185: seq=1 ttl=62 time=0.493 ms......
/ # ping web-2.nginx-svc.default.svc.cluster.local.PING web-2.nginx-svc.default.svc.cluster.local、(10.3.104.47): 56 data bytes64 bytes from 10.3.104.47: seq=0 ttl=63 time=0.101 ms64 bytes from 10.3.104.47: seq=1 ttl=63 time=0.086 ms......
补充: 当pod被删除后，重新拉起来，pod-IP可能会变，但上面的pod域名仍然可以ping通(请自行验证)
验证statefulset的伸缩 扩容[root@master1 ~]# kubectl scale sts web --replicas=4statefulset.apps/web scaled
[root@master1 ~]# kubectl get podsNAME READY STATUS RESTARTS AGEnfs-client-provisioner-5b5ddcd6c8-c2gbl 1/1 Running 0 3h3mweb-0 1/1 Running 0 7m31sweb-1 1/1 Running 0 21mweb-2 1/1 Running 0 21mweb-3 1/1 Running 0 10s有序地扩展了一个pod,名称为web-3
裁剪[root@master1 ~]# kubectl scale sts web --replicas=1statefulset.apps/web scaled
[root@master1 ~]# kubectl get pods |grep webweb-0 1/1 Running 0 31mweb-1 1/1 Running 0 45mweb-2 1/1 Running 0 22mweb-3 0/1 Terminating 0 22m先裁剪web-3
[root@master1 ~]# kubectl get pods |grep webweb-0 1/1 Running 0 31mweb-1 1/1 Running 0 45mweb-2 1/1 Terminating 0 22m再裁剪web-2
[root@master1 ~]# kubectl get pods |grep webweb-0 1/1 Running 0 32mweb-1 0/1 Terminating 0 46m最后裁剪web-1
[root@master1 ~]# kubectl get pods |grep webweb-0 1/1 Running 0 32m只留下web-0
三、mysql-statefulset-nfs案例 编写statefulset[root@master1 ~]# vim statefulset-mysql-nfs.yamlapiVersion: v1kind: Servicemetadata: name: mysql-svcspec: clusterIP: None# 无头服务 ports: - port: 3306 protocol: TCP targetPort: 3306 selector: app: mysql---apiVersion: apps/v1kind: StatefulSetmetadata: name: mysqlspec: serviceName: mysql-svc# 与上面服务名一致 replicas: 1# 副本数为1就OK，这里不做mysql集群 selector: matchLabels: app: mysql template: metadata: labels: app: mysql spec: containers: - name: c1 image: mysql:5.7 env: - name: MYSQL_ROOT_PASSWORD# mysql5.7的镜像必须要设置一下mysql的root密码 value: "123456" - name: MYSQL_DATAbase# 给它建一个库名为daniel，用于验证 value: daniel volumeMounts: - name: mysql-data mountPath: /var/lib/mysql volumeClaimTemplates: - metadata: name: mysql-data spec: accessModes: [ "ReadWriteOnce" ] storageClassName: "managed-nfs-storage" resources: requests: storage: 3Gi
应用YAML[root@master1 ~]# kubectl apply -f statefulset-mysql-nfs.yamlservice/mysql-svc createdstatefulset.apps/mysql created
验证资源[root@master1 ~]# kubectl get pods |grep mysqlmysql-0 1/1 Running 0 28s
[root@master1 ~]# kubectl get sts |grep mysqlmysql 1/1 1m
[root@master1 ~]# kubectl get pv |grep mysqlpvc-bc72e7ac-d65b-42ae-853f-14b5b1c9d30c 3Gi RWO Delete Bound default/mysql-data-mysql-0 managed-nfs-storage 6m
[root@master1 ~]# kubectl get pvc |grep mysqlmysql-data-mysql-0 Bound pvc-bc72e7ac-d65b-42ae-853f-14b5b1c9d30c 3Gi RWO managed-nfs-storage 7m
验证mysql[root@master1 ~]# kubectl exec -it mysql-0 -- /bin/bashroot@mysql-0:/# mysql -p123456mysql: [Warning] Using a password on the command line interface can be insecure.Welcome to the MySQL monitor、 Commands end with ; or g.Your MySQL connection id is 2Server version: 5.7.32 MySQL Community Server (GPL)Copyright (c) 2000, 2020, Oracle and/or its affiliates、All rights reserved.Oracle is a registered trademark of Oracle Corporation and/or itsaffiliates、Other names may be trademarks of their respectiveowners.Type 'help;' or 'h' for help、Type 'c' to clear the current input statement.mysql> show databases;+--------------------+| Database |+--------------------+| information_schema || daniel |# 这里daneil库就是帮我们创建的| mysql || performance_schema || sys |+--------------------+5 rows in set (0.01 sec)mysql> exit
验证nfs上的数据
[root@hostos ~]# ls /data/nfs/default-mysql-data-mysql-0-pvc-bc72e7ac-d65b-42ae-853f-14b5b1c9d30c/auto.cnf client-cert.pem ib_buffer_pool ib_logfile1 private_key.pem server-key.pemca-key.pem client-key.pem ibdata1 mysql public_key.pem sysca.pem daniel ib_logfile0 performance_schema server-cert.pem
删除mysql-0这个pod，会帮我们再次启动，并且数据还是用原来的数据（请自行验证)