背景
本文主要講解kubernetes 如何使用helm chart安裝 ingress-nginx
kubernetes 使用helm chart安裝 ingress-nginx
1. ingres nginx controller 和 k8s版本 兼容性要求
https://github.com/kubernetes/ingress-nginx/blob/main/README.md#supported-versions-table
Supported Ingress-NGINX version k8s supported version Alpine Version Nginx Version Helm Chart Version
?? v1.11.2 1.30, 1.29, 1.28, 1.27, 1.26 3.20.0 1.25.5 4.11.2
?? v1.11.1 1.30, 1.29, 1.28, 1.27, 1.26 3.20.0 1.25.5 4.11.1
?? v1.11.0 1.30, 1.29, 1.28, 1.27, 1.26 3.20.0 1.25.5 4.11.0
?? v1.10.4 1.30, 1.29, 1.28, 1.27, 1.26 3.20.0 1.25.5 4.10.4
?? v1.10.3 1.30, 1.29, 1.28, 1.27, 1.26 3.20.0 1.25.5 4.10.3
2.安裝環(huán)境
注: linux amd64 替換相關(guān)鏡像版本 & helm3安裝包即可-安裝流程同理
linux arm64: 內(nèi)核版本 4.18.0-348.20.1.el7.aarch64 #1 SMP Wed Apr 13 20:57:50 UTC 2022 aarch64 aarch64 aarch64 GNU/Linux
Kubernetes: v1.28.0
Docker: 26.1.4
ingress-nginx: 4.11.2
3.安裝離線鏡像準(zhǔn)備
# 下載 ingress-controller依賴鏡像(國內(nèi)機(jī)器有墻無法拉取)
docker pull registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.3
docker pull registry.k8s.io/ingress-nginx/controller:v1.11.2
# 導(dǎo)出為離線鏡像
docker save -o kube-webhook-certgen-v1.4.3.tar registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.3
docker save -o controller-v1.11.2.tar registry.k8s.io/ingress-nginx/controller:v1.11.2
# 安裝k8s所有機(jī)器節(jié)點(diǎn)導(dǎo)入離線鏡像
docker load -i controller-v1.11.2.tar
docker load -i kube-webhook-certgen-v1.4.3.tar
# docker images|grep ingress
registry.k8s.io/ingress-nginx/controller v1.11.2 289a818c8d9c 2 weeks ago 294MB
registry.k8s.io/ingress-nginx/kube-webhook-certgen v1.4.3 420193b27261 3 weeks ago 53.3MB
# 鏡像打tag & push到本地倉庫[可選]
#docker tag registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.3 sealos.hub:5000/ingress-nginx/kube-webhook-certgen:v1.4.3
#docker push sealos.hub:5000/ingress-nginx/kube-webhook-certgen:v1.4.3
#docker tag registry.k8s.io/ingress-nginx/controller:v1.11.2 sealos.hub:5000/ingress-nginx/controller:v1.11.2
#docker push sealos.hub:5000/ingress-nginx/controller:v1.11.2
4.linux(amd64)安裝 helm3
參考 https://helm.sh/zh/docs/intro/install/
https://github.com/helm/helm/releases
wget https://get.helm.sh/helm-v3.15.4-linux-amd64.tar.gz
tar -xvf helm-v3.15.4-linux-amd64.tar.gz
mv linux-amd64/helm /usr/local/bin/helm
helm version
5.創(chuàng)建k8s拉取鏡像-鏡像倉庫驗(yàn)證鑒權(quán)信息
# 創(chuàng)建鏡像倉庫驗(yàn)證鑒權(quán)信息: k8s拉取驗(yàn)證 結(jié)合 imagePullSecrets: imagePullSecrets: - name: scr 引用使用
kubectl create secret docker-registry scr \
-n ingress-nginx \
--docker-server=http://sealos.hub:5000 \
--docker-username=admin \
--docker-password=123456 \
--docker-email=jinze@ali.com
# 刪除鏡像倉庫驗(yàn)證鑒權(quán)信息
kubectl delete secret -n ingress-nginx scr
# 查看解密secret內(nèi)容
kubectl get secret -n ingress-nginx scr --output="jsonpath={.data.\.dockerconfigjson}" | base64 --decode
4.解壓安裝ingress-nginx
# helmchart 安裝 ingrss nginx
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update
# helm列出所有版本:
helm search repo ingress-nginx/ingress-nginx -l
NAME CHART VERSION APP VERSION DESCRIPTION
ingress-nginx/ingress-nginx 4.11.2 1.11.2 Ingress controller for Kubernetes using NGINX a...
ingress-nginx/ingress-nginx 4.11.1 1.11.1 Ingress controller for Kubernetes using NGINX a...
ingress-nginx/ingress-nginx 4.11.0 1.11.0 Ingress controller for Kubernetes using NGINX a...
ingress-nginx/ingress-nginx 4.10.4 1.10.4 Ingress controller for Kubernetes using NGINX a...
ingress-nginx/ingress-nginx 4.10.3 1.10.3 Ingress controller for Kubernetes using NGINX a...
ingress-nginx/ingress-nginx 4.10.2 1.10.2 Ingress controller for Kubernetes using NGINX a...
ingress-nginx/ingress-nginx 4.10.1 1.10.1 Ingress controller for Kubernetes using NGINX a...
ingress-nginx/ingress-nginx 4.10.0 1.10.0 Ingress controller for Kubernetes using NGINX a...
# helm 下載指定版本:4.11.2
helm fetch ingress-nginx/ingress-nginx --version 4.11.2
# 解壓ingress-nginx4.11.2 版本安裝包
tar -xvf ingress-nginx-4.11.2.tgz
# 編輯 ingress-nginx 配置 values.yaml
vi ingress-nginx/values.yaml
# 配置controller鏡像
controller:
image:
chroot: false
registry: registry.k8s.io
image: ingress-nginx/controller
tag: "v1.11.2"
#digest: sha256:d5f8217feeac4887cb1ed21f27c2674e58be06bd8f5184cacea2a69abaf78dce
# digest 需配置為空才能拉取 registry.k8s.io/ingress-nginx/controller:v1.11.2 鏡像
digest:
# 配置admissionWebhooks鏡像
controller:
admissionWebhooks:
patch:
image:
digest: sha256:a320a50cc91bd15fd2d6fa6de58bd98c1bd64b9a6f926ce23a600d87043455a3
# digest 需配置為空才能拉取 registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.3 鏡像
digest:
# ingress 暴露為NodePort
controller:
service:
#type: LoadBalancer
type: NodePort
# k8s 拉取鏡像倉庫驗(yàn)證secret
#imagePullSecrets: []
imagePullSecrets:
- name: scr
# helm chart 安裝(存在就更新) ingress-nginx
cd /root/ingress-nginx && helm upgrade --install ingress-nginx . --namespace ingress-nginx --create-namespace
# 卸載 ingress-nginx
helm uninstall ingress-nginx -n ingress-nginx
# 查看安裝 ingress
helm list -A|grep ingress
# 驗(yàn)證ingress 組件狀態(tài),是否正常拉起
kubectl get svc -A |grep ingress
kubectl get pod -A |grep ingress
kubectl get deploy -n ingress-nginx ingress-nginx-controller -oyaml
# ingress-nginx 成功安裝效果
[root@bj-arm-master ingress-nginx]# helm upgrade --install ingress-nginx . --namespace ingress-nginx --create-namespace
Release "ingress-nginx" has been upgraded. Happy Helming!
NAME: ingress-nginx
LAST DEPLOYED: Fri Sep 6 11:20:42 2024
NAMESPACE: ingress-nginx
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
The ingress-nginx controller has been installed.
Get the application URL by running these commands:
export HTTP_NODE_PORT=$(kubectl get service --namespace ingress-nginx ingress-nginx-controller --output jsonpath="{.spec.ports[0].nodePort}")
export HTTPS_NODE_PORT=$(kubectl get service --namespace ingress-nginx ingress-nginx-controller --output jsonpath="{.spec.ports[1].nodePort}")
export NODE_IP="$(kubectl get nodes --output jsonpath="{.items[0].status.addresses[1].address}")"
echo "Visit http://${NODE_IP}:${HTTP_NODE_PORT} to access your application via HTTP."
echo "Visit https://${NODE_IP}:${HTTPS_NODE_PORT} to access your application via HTTPS."
An example Ingress that makes use of the controller:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example
namespace: foo
spec:
ingressClassName: nginx
rules:
- host: www.example.com
http:
paths:
- pathType: Prefix
backend:
service:
name: exampleService
port:
number: 80
path: /
# This section is only required if TLS is to be enabled for the Ingress
tls:
- hosts:
- www.example.com
secretName: example-tls
If TLS is enabled for the Ingress, a Secret containing the certificate and key must also be provided:
apiVersion: v1
kind: Secret
metadata:
name: example-tls
namespace: foo
data:
tls.crt: <base64 encoded cert>
tls.key: <base64 encoded key>
type: kubernetes.io/tls
6.配置 ingress 轉(zhuǎn)發(fā)規(guī)則:
-- ingress配置demo1: 訪問 路徑/ 轉(zhuǎn)發(fā)到后端 namespace為default的 bte-service的8080端口
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: bte
namespace: default
#annotations:
# 無論客戶端請求的是哪個(gè)路徑,Ingress 控制器都會將目標(biāo)請求路徑重寫為根路徑 /
# 當(dāng)用戶訪問 http://example.com/foo 時(shí),NGINX Ingress Controller 會將請求重寫為 http://my-service:80/。也就是說,任何通過 /foo 訪問的請求都會轉(zhuǎn)發(fā)到 my-service 服務(wù),并且請求路徑會被重寫為根路徑 /
#nginx.ingress.kubernetes.io/rewrite-target: /
spec:
# 指定 Ingress Controller 的類型 為 nginx 類型:告訴 Kubernetes,這個(gè) Ingress 由 NGINX Ingress Controller 處理
ingressClassName: nginx
rules:
#- host: "*"
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: bte-service
port:
number: 8080
-- ingress配置demo2: 訪問 路徑/layout 轉(zhuǎn)發(fā)到后端 namespace為default的 layout-service的8080端口
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: layout
namespace: default
annotations:
# 無論客戶端請求的是哪個(gè)路徑,Ingress 控制器都會將目標(biāo)請求路徑重寫為根路徑 /
# 當(dāng)用戶訪問 http://example.com/foo 時(shí),NGINX Ingress Controller 會將請求重寫為 http://my-service:80/。也就是說,任何通過 /foo 訪問的請求都會轉(zhuǎn)發(fā)到 my-service 服務(wù),并且請求路徑會被重寫為根路徑 /
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
# k8s 1.18版本后 指定 Ingress Controller 的類型 為 nginx 類型配置:告訴 Kubernetes,這個(gè) Ingress 由 NGINX Ingress Controller 處理;
ingressClassName: nginx
rules:
#- host: "*"
- http:
paths:
- path: /layout
pathType: Prefix
backend:
service:
name: layout-service
port:
number: 8080
3.安裝過程遇到安裝失敗問題處理
安裝遇到問題: 無法正常拉取鏡像ImagePullBackOff
# kubectl get pod -A |grep ingress
ingress-nginx ingress-nginx-admission-create-nz6hv 0/1 ImagePullBackOff 0 64s
# kubectl describe pod -n ingress-nginx ingress-nginx-admission-create-nz6hv
問題1: 報(bào)錯(cuò): 無法正常拉取鏡像 registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.3@sha256:a320a50cc91bd15fd2d6fa6de58bd98c1bd64b9a6f926ce23a600d87043455a3
此鏡像版本比我們離線導(dǎo)入的image tag 多了 @sha256:a320a50cc91bd15fd2d6fa6de58bd98c1bd64b9a6f926ce23a600d87043455a3 信息
查看helm chart 源碼分析問題原因
無法正常拉取鏡像 registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.4.3@sha256:a320a50cc91bd15fd2d6fa6de58bd98c1bd64b9a6f926ce23a600d87043455a3
vi /root/ingress-nginx/templates/admission-webhooks/job-patch/job-createSecret.yaml & job-patchWebhook.yaml
digest: sha256:a320a50cc91bd15fd2d6fa6de58bd98c1bd64b9a6f926ce23a600d87043455a3
** fix: digest 設(shè)置為空即可 **
重啟 ingress-nginx
helm upgrade --install ingress-nginx . --namespace ingress-nginx --create-namespace
問題2: 無法正常拉取鏡像 registry.k8s.io/ingress-nginx/controller:v1.11.2@sha256:d5f8217feeac4887cb1ed21f27c2674e58be06bd8f5184cacea2a69abaf78dce
# kubectl get pod -A |grep ingress
ingress-nginx ingress-nginx-controller-5bddfb7dbf-gzjsx 0/1 ImagePullBackOff 0 49s 100.78.46.152 bj-arm-node1 <none> <none>
# kubectl describe pod -n ingress-nginx ingress-nginx-controller-5bddfb7dbf-gzjsx
Failed to pull image "registry.k8s.io/ingress-nginx/controller:v1.11.2@sha256:d5f8217feeac4887cb1ed21f27c2674e58be06bd8f5184cacea2a69abaf78dce": Error response from daemon: Get "https://registry.k8s.io/v2/": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
此鏡像版本比我們離線導(dǎo)入的image tag 多了 @sha256:d5f8217feeac4887cb1ed21f27c2674e58be06bd8f5184cacea2a69abaf78dce 信息
查看helm chart 源碼分析問題原因
cat /root/ingress-nginx/templates/controller-deployment.yaml
###### 檢查當(dāng)前目錄下所有yaml文件中包含 字符串 ingress-nginx.imageDigest
grep -o "ingress-nginx.imageDigest" ./*.*
# 輸出
/root/ingress-nginx/templates/_helpers.tpl
fix: digest 設(shè)置為空即可
重啟 ingress-nginx
helm upgrade --install ingress-nginx . --namespace ingress-nginx --create-namespace
ingress-nginx pod正常拉起,問題fix
# kubectl get pod -A |grep ingress
ingress-nginx ingress-nginx-controller-785fcc99b-2zdhx 1/1 Running 0 22s
-- 問題fix!
參考文檔
https://kubernetes.io/docs/concepts/services-networking/ingress/ ingress文檔
https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/ Ingress Controllers 文檔
-- ingress-nginx相關(guān)文檔
https://github.com/kubernetes/ingress-nginx/blob/main/README.md#readme ingres nginx controller github文檔
https://github.com/kubernetes/ingress-nginx Ingress-nginx 文檔(支持 helm chart部署) use
https://github.com/kubernetes/ingress-nginx/tree/main/charts/ingress-nginx ingress-nginx helm chart文檔
https://github.com/kubernetes/ingress-nginx/tree/main/charts/ingress-nginx#values ingress-nginx helm chart values.yaml 配置說明
https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/nginx-configuration/index.md
https://kubernetes.github.io/ingress-nginx/user-guide/tls/ ngress-nginx 配置文檔