Keepalived
1.Keepalived定義:Keepalived 是一個(gè)基于VRRP協(xié)議來實(shí)現(xiàn)的LVS服務(wù)高可用方案,可以利用其來避免單點(diǎn)故障。一個(gè)LVS服務(wù)會(huì)有2臺(tái)服務(wù)器運(yùn)行Keepalived,一臺(tái)為主服務(wù)器(MASTER),一臺(tái)為備份服務(wù)器(BACKUP),但是對外表現(xiàn)為一個(gè)虛擬IP,主服務(wù)器會(huì)發(fā)送特定的消息給備份服務(wù)器,當(dāng)備份服務(wù)器收不到這個(gè)消息的時(shí)候,即主服務(wù)器宕機(jī)的時(shí)候, 備份服務(wù)器就會(huì)接管虛擬IP,繼續(xù)提供服務(wù),從而保證了高可用性。Keepalived是VRRP的完美實(shí)現(xiàn),因此在介紹keepalived之前,先介紹一下VRRP的原理。
2.VRRP 協(xié)議簡介
VRRP將局域網(wǎng)內(nèi)的一組路由器劃分在一起,形成一個(gè)VRRP備份組,它在功能上
相當(dāng)于一臺(tái)虛擬路由器,使用虛擬路由器號(hào)進(jìn)行標(biāo)識(shí)。以下使用虛擬路由器代替
VRRP備份組進(jìn)行描述。
虛擬路由器有自己的虛擬IP地址和虛擬MAC地址,它的外在表現(xiàn)形式和實(shí)際的物
理路由器完全一樣。局域網(wǎng)內(nèi)的主機(jī)將虛擬路由器的IP地址設(shè)置為默認(rèn)網(wǎng)關(guān),通過
虛擬路由器與外部網(wǎng)絡(luò)進(jìn)行通信。
虛擬路由器是工作在實(shí)際的物理路由器之上的。它由多個(gè)實(shí)際的路由器組成,包括
一個(gè)Master路由器和多個(gè)Backup路由器。Master路由器正常工作時(shí),局域網(wǎng)內(nèi)的
主機(jī)通過Master與外界通信。當(dāng)Master路由器出現(xiàn)故障時(shí),Backup路由器中的一
臺(tái)設(shè)備將成為新的Master路由器,接替轉(zhuǎn)發(fā)報(bào)文的工作.
3.VRRP工作過程
(1) 虛擬路由器中的路由器根據(jù)優(yōu)先級選舉出Master。Master 路由器通過發(fā)送免
費(fèi)ARP 報(bào)文,將自己的虛擬MAC 地址通知給與它連接的設(shè)備或者主機(jī),從
而承擔(dān)報(bào)文轉(zhuǎn)發(fā)任務(wù);
(2) Master 路由器周期性發(fā)送VRRP 報(bào)文,以公布其配置信息(優(yōu)先級等)和工
作狀況;
(3) 如果Master 路由器出現(xiàn)故障,虛擬路由器中的Backup 路由器將根據(jù)優(yōu)先級
重新選舉新的Master;
(4) 虛擬路由器狀態(tài)切換時(shí),Master 路由器由一臺(tái)設(shè)備切換為另外一臺(tái)設(shè)備,新
的Master 路由器只是簡單地發(fā)送一個(gè)攜帶虛擬路由器的MAC 地址和虛擬IP
地址信息的免費(fèi)ARP 報(bào)文,這樣就可以更新與它連接的主機(jī)或設(shè)備中的
ARP 相關(guān)信息。網(wǎng)絡(luò)中的主機(jī)感知不到Master 路由器已經(jīng)切換為另外一臺(tái)
設(shè)備。
(5) Backup 路由器的優(yōu)先級高于Master 路由器時(shí),由Backup 路由器的工作方
式(搶占方式和非搶占方式)決定是否重新選舉Master。
實(shí)現(xiàn)雙主模型的ipvs高可用集群;
一、實(shí)驗(yàn)環(huán)境:
操作系統(tǒng)
centos 7
keepalived master 192.168.18.97
keepalived bachup 192.168.18.98
Real Server 1 192.168.18.99
Real Server 2 192.168.18.100
前提
1.各節(jié)點(diǎn)時(shí)間必須同步;ntp 或者 chrony;
- 確保iptables及selinux不會(huì)成為阻礙
3.各節(jié)點(diǎn)之間可通過主機(jī)名互相通信(對Keepalived并非必須);
簡單拓?fù)?/p>
二.配置
1.先安裝各主機(jī)相應(yīng)的軟件:
MASTER與BACKUP主機(jī):
yum -y install keepalived
RS1與RS2主機(jī),安裝相應(yīng)的web服務(wù),這里安裝的是Nginx
yum -y install niginx
2.配置兩web服務(wù)的簡單測試頁
RS1:
vim usr/share/nginx/html/index.html
<h1> SR1 </h1>
RS2:
vim usr/share/nginx/html/index.html
<h1> SR1 </h1>
3.啟動(dòng)兩臺(tái)相應(yīng)的Nginx服務(wù)
systemctl start nginx
4.測試
[root@centos7 ~]#curl http://192.168.18.99
<h1> SR1 host </h1>
[root@centos7 ~]#curl http://192.168.18.100
<h1> SR2 host </h1>
這里可正常訪問
三.配置各主機(jī)的VIP
因?yàn)橐獙?shí)現(xiàn)LVS+Keepalived 實(shí)現(xiàn)高可用的前端負(fù)載均衡器這里就是用LVS-DR模式
dr模型中,各主機(jī)上均需要配置VIP,解決地址沖突的方式有三種:
(1) 在前端網(wǎng)關(guān)做靜態(tài)綁定
(2) 在各RS使用arptables
(3) 在各RS修改內(nèi)核參數(shù),來限制arp響應(yīng)和通告的級別
限制響應(yīng)級別:arp_ignore
0:默認(rèn)值,表示可使用本地任意接口上配置的任意地址進(jìn)行響應(yīng)
1: 僅在請求的目標(biāo)IP配置在本地主機(jī)的接收到請求報(bào)文的接口上時(shí),才給予響應(yīng)
限制通告級別:arp_announce
0:默認(rèn)值,把本機(jī)所有接口的所有信息向每個(gè)接口的網(wǎng)絡(luò)進(jìn)行通告
1:盡量避免將接口信息向非直接連接網(wǎng)絡(luò)進(jìn)行通告
2:必須避免將接口信息向非本網(wǎng)絡(luò)進(jìn)行通告
1.在RS1于RS2上設(shè)置:這里可以設(shè)置成腳本,使用起來更加方便。
#!/bin/bash
#
vip1=192.168.18.68
vip2=192.168.18.168
mask=255.255.255.255
iface1="lo:1"
iface2="lo:2"
case $1 in
start)
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
ifconfig $iface1 $vip1 netmask $mask broadcast $vip1 up
ifconfig $iface2 $vip2 netmask $mask broadcast $vip2 up
route add -host $vip1 dev $iface1 //此步驟非必須
;;
stop)
ifconfig $iface1 down
ifconfig $iface2 down
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
;;
*)
echo "Usage:$(basename $0) start|stop"
exit 1
;;
esac
2.在主機(jī)RS1和RS2運(yùn)行腳本,查看VIP是否 已經(jīng)添加成功。
[root@centos7 ~]#bash lvs_dr.sh start
[root@centos7 ~]#ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 192.168.18.68/32 brd 192.168.18.68 scope global lo:1
valid_lft forever preferred_lft forever
inet 192.168.18.168/32 brd 192.168.18.168 scope global lo:2
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
...
2: ens34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:90:2e:2a brd ff:ff:ff:ff:ff:ff
inet 192.168.18.100/24 brd 192.168.18.255 scope global ens34
valid_lft forever preferred_lft forever
inet6 fe80::8c8e:8ee1:9668:4e8e/64 scope link
valid_lft forever preferred_lft forever
inet6 fe80::6a3c:6:139c:cc3b/64 scope link tentative dadfailed
valid_lft forever preferred_lft forever
3.測試兩臺(tái)準(zhǔn)備做MASTER和BAVKUP的主機(jī),LVS-DR是否可用:
MASTER:
yum -y install ipvsadm
ifconfig ens34:0 192.168.18.68 netmask 255.255.255.255 broadcast 192.168.18.68 up
ifconfig ens34:1192.168.18.168 netmask 255.255.255.255 broadcast 192.168.18.168 up
ipvsadm -A -t 192.168.18.68:80 -s rr
ipvsadm -a -t 192.168.18.68:80 -r 192.168.18.99:80 -g
ipvsadm -a -t 192.168.18.68:80 -r 192.168.18.100:80 -g
ipvsadm -A -t 192.168.18.168:80 -s rr
ipvsadm -a -t 192.168.18.168:80 -r 192.168.18.99:80 -g
ipvsadm -a -t 192.168.18.168:80 -r 192.168.18.100:80 -g
[root@cnetos7 ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.18.68:80 rr
-> 192.168.18.99:80 Route 1 0 0
-> 192.168.18.100:80 Route 1 0 0
TCP 192.168.18.168:80 rr
-> 192.168.18.99:80 Route 1 0 0
-> 192.168.18.100:80 Route 1 0 0
4.測試MASTER和BACKUP是否可用。
[root@centos7 ~]#for i in {1..10};do curl http://192.168.18.68;done
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
[root@centos7 ~]#for i in {1..10};do curl http://192.168.18.168;done
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
可正常訪問
5.清除MASTER和BACKUP的VIP和集群
[root@cnetos7 ~]#ifconfig ens34:1 down //down掉VIP
[root@cnetos7 ~]#ifconfig ens34:0 down
[root@cnetos7 ~]#ipvsadm -C //清空定義的所有內(nèi)容
[root@cnetos7 ~]#ipvsadm -Ln //查看
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
[root@cnetos7 ~]#ip a //查看ip確保ip恢復(fù)到最初狀態(tài)
四、配置實(shí)現(xiàn)雙主模型的ipvs高可用集群
1.在MASTER主機(jī)上:
[root@centos7 ~]#vim /etc/keepalived/keepalived.conf
global_defs {
notification_email {
keepalived@jie.com //配置管理員郵箱
}
notification_email_from ka_admin@jie.com //配置發(fā)件人
smtp_server 127.0.0.1 //p配置郵件服務(wù)器
smtp_connect_timeout 30 //郵件連接超時(shí)時(shí)長s為單位
router_id centos7.3 //主機(jī)名
vrrp_mcast_group4 224.27.27.18 //vrrp組
}
vrrp_instance VI_1 { //組1
state MASTER //當(dāng)前節(jié)點(diǎn)在此虛擬路由器上的初始狀態(tài);只能有一個(gè)是MASTER,余下的都應(yīng)該為BACKUP
interface ens34 //綁定為當(dāng)前虛擬路由器使用的物理接口;
virtual_router_id 27 // 前虛擬路由器的惟一標(biāo)識(shí),范圍是0-255;
priority 100 //當(dāng)前主機(jī)在此虛擬路徑器中的優(yōu)先級;范圍1-254;
advert_int 1 //vrrp通告的時(shí)間間隔;
authentication { //認(rèn)證
auth_type PASS
auth_pass eHTQgK1n
}
virtual_ipaddress {
192.168.18.168 //配置虛擬ip地址
}
//定義通知腳本:
notify_master "/etc/keepalived/notify.sh master" //當(dāng)前節(jié)點(diǎn)成為主節(jié)點(diǎn)時(shí)觸發(fā)的腳本;
notify_backup "/etc/keepalived/notify.sh backup" //當(dāng)前節(jié)點(diǎn)轉(zhuǎn)為備節(jié)點(diǎn)時(shí)觸發(fā)的腳本;
notify_fault "/etc/keepalived/notify.sh fault" //當(dāng)前節(jié)點(diǎn)轉(zhuǎn)為“失敗”狀態(tài)時(shí)觸發(fā)的腳本;
}
vrrp_instance VI_2 {
state BACKUP
interface ens34
virtual_router_id 37
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass eHTQgK0n
}
virtual_ipaddress {
192.168.18.68
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
virtual_server 192.168.18.168 80 { //虛擬服務(wù)器
delay_loop 3 //服務(wù)輪詢的時(shí)間間隔;
lb_algo rr //定義調(diào)度方法;
lb_kind DR //集群的類型;
protocol TCP //服務(wù)協(xié)議,僅支持TCP;
sorry_server 127.0.0.1 80 //備用服務(wù)器地址;
real_server 192.168.18.100 80 {
weight 1
HTTP_GET { 應(yīng)用層檢測
url {
path / //定義要監(jiān)控的URL;
status_code 200 //
}
connect_timeout 1 //連接請求的超時(shí)時(shí)長;
nb_get_retry 3 //重試次數(shù);
delay_before_retry 1 //重試之前的延遲時(shí)長;
}
}
real_server 192.168.18.99 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
}
virtual_server 192.168.18.68 80 {
delay_loop 3
lb_algo rr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 192.168.18.100 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
real_server 192.168.18.99 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
}
2.在BACKUP主機(jī)上:
global_defs {
notification_email {
keepalived@jie.com
}
notification_email_from ka_admin@jie.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id cnetos7.localdomain
vrrp_mcast_group4 224.27.27.18 //保持一致
}
vrrp_instance VI_1 {
state BACKUP
interface ens34
virtual_router_id 27 //保持一致
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass eHTQgK1n //保持一致
}
virtual_ipaddress {
192.168.18.168
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
vrrp_instance VI_2 {
state MASTER
interface ens34
virtual_router_id 37 //保持一致
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass eHTQgK0n //保持一致
}
virtual_ipaddress {
192.168.18.68
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
virtual_server 192.168.18.168 80 {
delay_loop 3
lb_algo rr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 192.168.18.100 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
real_server 192.168.18.99 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
}
virtual_server 192.168.18.68 80 {
delay_loop 3
lb_algo rr
lb_kind DR
protocol TCP
sorry_server 127.0.0.1 80
real_server 192.168.18.100 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
real_server 192.168.18.99 80 {
weight 1
HTTP_GET {
url {
path /
status_code 200
}
connect_timeout 1
nb_get_retry 3
delay_before_retry 1
}
}
}
3.配置通知腳本
[root@cnetos7 keepalived]#vim notify.sh
#!bin/bash
contact='root@localhost'
notify() {
local mailsubject="$(hostname) to be $1, vip floating"
local mailbody="$(date +'%F %T'): vrrp transition, $(hostname) changed to be $1"
echo "$mailbody" | mail -s "$mailsubject" $contact
}
case $1 in
master)
notify master
;;
backup)
notify backup
;;
fault)
notify fault
;;
*)
echo "Usage: $(basename $0) {master|backup|fault}"
exit 1
;;
esac
4.測試:
1).先在BACKUP上啟用
[root@cnetos7 keepalived]#systemctl start keepalived
[root@cnetos7 keepalived]#systemctl status keepalived //查看狀態(tài)
● keepalived.service - LVS and VRRP High Availability Monitor
Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
Active: active (running) since 四 2017-09-07 04:58:27 CST; 42s ago
Process: 2859 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 2860 (keepalived)
CGroup: /system.slice/keepalived.service
├─2860 /usr/sbin/keepalived -D
├─2861 /usr/sbin/keepalived -D
└─2862 /usr/sbin/keepalived -D
9月 07 04:58:29 cnetos7.localdomain Keepalived_vrrp[2862]: Opening script file /etc/keepalived/notify.sh
9月 07 04:58:29 cnetos7.localdomain Keepalived_healthcheckers[2861]: Netlink reflector reports IP 192.168.18.68 added
9月 07 04:58:31 cnetos7.localdomain Keepalived_vrrp[2862]: VRRP_Instance(VI_1) Transition to MASTER STATE
9月 07 04:58:32 cnetos7.localdomain Keepalived_vrrp[2862]: VRRP_Instance(VI_1) Entering MASTER STATE
9月 07 04:58:32 cnetos7.localdomain Keepalived_vrrp[2862]: VRRP_Instance(VI_1) setting protocol VIPs.
9月 07 04:58:32 cnetos7.localdomain Keepalived_vrrp[2862]: VRRP_Instance(VI_1) Sending gratuitous ARPs on ens34 for 192.168.18.168
9月 07 04:58:32 cnetos7.localdomain Keepalived_vrrp[2862]: Opening script file /etc/keepalived/notify.sh
9月 07 04:58:32 cnetos7.localdomain Keepalived_healthcheckers[2861]: Netlink reflector reports IP 192.168.18.168 added
9月 07 04:58:34 cnetos7.localdomain Keepalived_vrrp[2862]: VRRP_Instance(VI_2) Sending gratuitous ARPs on ens34 for 192.168.18.68
9月 07 04:58:37 cnetos7.localdomain Keepalived_vrrp[2862]: VRRP_Instance(VI_1) Sending gratuitous ARPs on ens34 for 192.168.18.168
您在 /var/spool/mail/root 中有新郵件 //設(shè)置了郵件通知
因?yàn)橹鱉ASTER的Keepalived沒有啟動(dòng),所以先在兩VIP都在BACHUP主機(jī)上
[root@cnetos7 keepalived]#ip a
....
3: ens34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:50:56:2e:b2:ba brd ff:ff:ff:ff:ff:ff
inet 192.168.18.98/24 brd 192.168.18.255 scope global ens34
valid_lft forever preferred_lft forever
inet 192.168.18.68/32 scope global ens34
valid_lft forever preferred_lft forever
inet 192.168.18.168/32 scope global ens34
valid_lft forever preferred_lft forever
inet6 fe80::c4db:7bad:474f:7c55/64 scope link
valid_lft forever preferred_lft forever
[root@cnetos7 keepalived]#tcpdump -nn -i ens34 host 224.27.27.18 //組播信心查看
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens34, link-type EN10MB (Ethernet), capture size 65535 bytes
05:04:45.963370 IP 192.168.18.98 > 224.27.27.18: VRRPv2, Advertisement, vrid 27, prio 98, authtype simple, intvl 1s, length 20
05:04:46.228313 IP 192.168.18.98 > 224.27.27.18: VRRPv2, Advertisement, vrid 37, prio 100, authtype simple, intvl 1s, length 20
2).啟動(dòng)MASTER主機(jī)
[root@centos7 ~]#systemctl start keepalived
[root@centos7 ~]#systemctl status keepalived
● keepalived.service - LVS and VRRP High Availability Monitor
Loaded: loaded (/usr/lib/systemd/system/keepalived.service; disabled; vendor preset: disabled)
Active: active (running) since 三 2017-09-06 21:06:56 CST; 7s ago
Process: 19901 ExecStart=/usr/sbin/keepalived $KEEPALIVED_OPTIONS (code=exited, status=0/SUCCESS)
Main PID: 19902 (keepalived)
CGroup: /system.slice/keepalived.service
├─19902 /usr/sbin/keepalived -D
├─19903 /usr/sbin/keepalived -D
└─19904 /usr/sbin/keepalived -D
9月 06 21:06:57 centos7.3 Keepalived_vrrp[19904]: VRRP_Instance(VI_1) Transition to MASTER STATE
9月 06 21:06:57 centos7.3 Keepalived_vrrp[19904]: VRRP_Instance(VI_1) Received lower prio advert, forcing new election
9月 06 21:06:58 centos7.3 Keepalived_vrrp[19904]: VRRP_Instance(VI_1) Entering MASTER STATE
9月 06 21:06:58 centos7.3 Keepalived_vrrp[19904]: VRRP_Instance(VI_1) setting protocol VIPs.
9月 06 21:06:58 centos7.3 Keepalived_vrrp[19904]: VRRP_Instance(VI_1) Sending gratuitous ARPs on ens34 for 192.168.18.168
9月 06 21:06:58 centos7.3 Keepalived_vrrp[19904]: Opening script file /etc/keepalived/notify.sh
[root@centos7 ~]#ip a
....
3: ens34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:2b:e9:62 brd ff:ff:ff:ff:ff:ff
inet 192.168.18.97/24 brd 192.168.18.255 scope global ens34
valid_lft forever preferred_lft forever
inet 192.168.18.168/32 scope global ens34 //獲取了ip
valid_lft forever preferred_lft forever
inet6 fe80::6a3c:6:139c:cc3b/64 scope link
valid_lft forever preferred_lft forever
[root@cnetos7 keepalived]#tcpdump -nn -i ens34 host 224.27.27.18
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens34, link-type EN10MB (Ethernet), capture size 65535 bytes
05:08:44.608354 IP 192.168.18.97 > 224.27.27.18: VRRPv2, Advertisement, vrid 27, prio 100, authtype simple, intvl 1s, length 20
05:08:44.740453 IP 192.168.18.98 > 224.27.27.18: VRRPv2, Advertisement, vrid 37, prio 100, authtype simple, intvl 1s, length 20
05:08:45.611090 IP 192.168.18.97 > 224.27.27.18: VRRPv2, Advertisement, vrid 27, prio 100, authtype simple, intvl 1s, length 20
05:08:45.744621 IP 192.168.18.98 > 224.27.27.18: VRRPv2, Advertisement, vrid 37, prio 100, authtype simple, intvl 1s, length 20
3)此時(shí)兩臺(tái)主機(jī)都會(huì)顯示:
[root@centos7 ~]#ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.18.68:80 rr
-> 192.168.18.99:80 Route 1 0 0
-> 192.168.18.100:80 Route 1 0 0
TCP 192.168.18.168:80 rr
-> 192.168.18.99:80 Route 1 0 0
-> 192.168.18.100:80 Route 1 0 0
5在其它主機(jī)上測試:
[root@centos7 ~]#for i in {1..10};do curl http://192.168.18.168;done
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
[root@centos7 ~]#for i in {1..10};do curl http://192.168.18.68;done
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
<h1> SR1 host </h1>
<h1> SR2 host </h1>
6.設(shè)置sorry.server,當(dāng)后端兩臺(tái)web服務(wù)器都宕機(jī)時(shí),能顯示信息提醒.
1)在MASTER和BACKUP上安裝Nginx
[root@centos7 ~]#yum -y install nginx
[root@centos7 ~]#vim /usr/share/nginx/html/index.html
Website is currently under maintenance, please come back later!
[root@centos7 ~]#systenctl start nginx
2)停止RS1和RS2web服務(wù)。
systenctl stop nginx
[root@centos7 ~]#for i in {1..10};do curl http://192.168.18.168;done
Website is currently under maintenance, please come back later!
Website is currently under maintenance, please come back later!
Website is currently under maintenance, please come back later!
Website is currently under maintenance, please come back later!
Website is currently under maintenance, please come back later!
Website is currently under maintenance, please come back later!
Website is currently under maintenance, please come back later!
Website is currently under maintenance, please come back later!
Website is currently under maintenance, please come back later!
Website is currently under maintenance, please come back later!
總結(jié),到這里keepalived的雙主模型的ipvs高可用集群已經(jīng)實(shí)現(xiàn),極大的提高負(fù)載均衡,高可用。
雙主模型的nginx proxy高可用集群;
簡單拓?fù)?/p>
一、配置MASTER:
因?yàn)槭褂肗ginx proxy,只需在上面的基礎(chǔ)上稍加修改就可以了,在后端的RS1和RS2上就不需要配置VIP,只需提供web服務(wù)即可。
global_defs {
notification_email {
keepalived@jie.com
}
notification_email_from ka_admin@jie.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id centos7.3
vrrp_mcast_group4 224.27.27.18
}
vrrp_script chk_down { //keepalived調(diào)用外部的輔助腳本進(jìn)行資源監(jiān)控,并根據(jù)監(jiān)控的結(jié)果狀態(tài)能實(shí)現(xiàn)優(yōu)先動(dòng)態(tài)調(diào)整;
script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0" //檢測是否存在down文件,有,權(quán)重減5
interval 1
weight -5
}
vrrp_script chk_nginx {
script "killall -0 nginx && exit 0 || exit 1" //檢測是否Nginx進(jìn)程正常在運(yùn)行,如沒有Nginx進(jìn)程,權(quán)重減5
interval 1
weight -5
fall 2
rise 1
}
vrrp_instance VI_1 {
state MASTER
interface ens34
virtual_router_id 27
priority 100
advert_int 1
authentication {
auth_type PASS
! Configuration File for keepalived
global_defs {
notification_email {
keepalived@jie.com
}
notification_email_from ka_admin@jie.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id centos7.3
vrrp_mcast_group4 224.27.27.18
}
vrrp_script chk_down {
script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"
interval 1
weight -5
}
vrrp_script chk_nginx {
script "killall -0 nginx && exit 0 || exit 1"
interval 1
weight -5
fall 2
rise 1
}
vrrp_instance VI_1 {
state MASTER
interface ens34
virtual_router_id 27
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass eHTQgK1n
}
virtual_ipaddress {
192.168.18.168
}
track_script {
chk_down
chk_nginx
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup" //如需必要可在相應(yīng)的郵件通知腳本中添加一旦Nginx進(jìn)程中斷,可重啟Nginx服務(wù)。
notify_fault "/etc/keepalived/notify.sh fault"
}
vrrp_instance VI_2 {
state BACKUP
interface ens34
virtual_router_id 37
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass eHTQgK0n
}
virtual_ipaddress {
192.168.18.68
}
track_script {
chk_down
chk_nginx
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
二、設(shè)置BACKUP
global_defs {
notification_email {
keepalived@jie.com
}
notification_email_from ka_admin@jie.com
smtp_server 127.0.0.1
smtp_connect_timeout 30
router_id cnetos7.localdomain
vrrp_mcast_group4 224.27.27.18
}
vrrp_script chk_down {
script "[[ -f /etc/keepalived/down ]] && exit 1 || exit 0"
interval 1
weight -5
}
vrrp_script chk_nginx {
script "killall -0 nginx && exit 0 || exit 1"
interval 1
weight -5
fall 2
rise 1
}
vrrp_instance VI_1 {
state BACKUP
interface ens34
virtual_router_id 27
priority 98
advert_int 1
authentication {
auth_type PASS
auth_pass eHTQgK1n
}
virtual_ipaddress {
192.168.18.168
}
track_script {
chk_down
chk_nginx
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
vrrp_instance VI_2 {
state MASTER
interface ens34
virtual_router_id 37
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass eHTQgK0n
}
virtual_ipaddress {
192.168.18.68
}
track_script {
chk_down
chk_nginx
}
notify_master "/etc/keepalived/notify.sh master"
notify_backup "/etc/keepalived/notify.sh backup"
notify_fault "/etc/keepalived/notify.sh fault"
}
三、配置Nginx的代理
在MASTER和BACKUP主機(jī)上配置相同的Nginx服務(wù)。
vim /etc/nginx/nginx.cong
http { //在http下添加組
upstream websrvs {
server 192.168.18.99:80;
server 192.168.18.100:80;
}
server {
location / {
proxy_pass http://websrvs; //添加為代理
}
}
}
如果不想在注配置文件修改可在/etc/nginx/conf.d/目錄下另外創(chuàng)建一個(gè)虛擬主機(jī)文件。
#######四.、測試
在MASTER和BACHUP主機(jī)上重啟Nginx
systemctl restart Nginx
systemctl start keepalived //如果此前已經(jīng)開啟了Keepalived服務(wù),則需停止,在啟動(dòng)。
1.測試down文件存在,看ip是否轉(zhuǎn)移。
MASTER:
[root@centos7 keepalived]#ip a
3: ens34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:2b:e9:62 brd ff:ff:ff:ff:ff:ff
inet 192.168.18.97/24 brd 192.168.18.255 scope global ens34
valid_lft forever preferred_lft forever
inet 192.168.18.168/32 scope global ens34
valid_lft forever preferred_lft forever
inet6 fe80::6a3c:6:139c:cc3b/64 scope link
valid_lft forever preferred_lft forever
[root@centos7 keepalived]#touch down
[root@centos7 keepalived]#ip a
3: ens34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:0c:29:2b:e9:62 brd ff:ff:ff:ff:ff:ff
inet 192.168.18.97/24 brd 192.168.18.255 scope global ens34
valid_lft forever preferred_lft forever
inet6 fe80::6a3c:6:139c:cc3b/64 scope link
valid_lft forever preferred_lft forever
BACHUP:
[root@centos7 keepalived]#ip a
3: ens34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:50:56:2e:b2:ba brd ff:ff:ff:ff:ff:ff
inet 192.168.18.98/24 brd 192.168.18.255 scope global ens34
valid_lft forever preferred_lft forever
inet 192.168.18.68/32 scope global ens34
valid_lft forever preferred_lft forever
inet 192.168.18.168/32 scope global ens34
valid_lft forever preferred_lft forever
inet6 fe80::c4db:7bad:474f:7c55/64 scope link
valid_lft forever preferred_lft forever
2.測試Nginx服務(wù)進(jìn)程是否正常工作
MASTER:
[root@centos7 ~]#tcpdump -nn -i ens34 host 224.27.27.18
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on ens34, link-type EN10MB (Ethernet), capture size 65535 bytes
10:04:49.240856 IP 192.168.18.97 > 224.27.27.18: VRRPv2, Advertisement, vrid 27, prio 100, authtype simple, intvl 1s, length 20
10:04:49.268780 IP 192.168.18.98 > 224.27.27.18: VRRPv2, Advertisement, vrid 37, prio 100, authtype simple, intvl 1s, length 20
10:04:50.242388 IP 192.168.18.97 > 224.27.27.18: VRRPv2, Advertisement, vrid 27, prio 100, authtype simple, intvl 1s, length 20
10:04:50.273757 IP 192.168.18.98 > 224.27.27.18: VRRPv2, Advertisement, vrid 37, prio 100, authtype simple, intvl 1s, length 20
root@centos7 keepalived]#systemctl stop nginx //停止Nginx服務(wù)地址轉(zhuǎn)移到BACKUP主機(jī)上。
10:05:33.393194 IP 192.168.18.98 > 224.27.27.18: VRRPv2, Advertisement, vrid 37, prio 100, authtype simple, intvl 1s, length 20
10:05:34.382548 IP 192.168.18.98 > 224.27.27.18: VRRPv2, Advertisement, vrid 27, prio 98, authtype simple, intvl 1s, length 20
10:05:34.394344 IP 192.168.18.98 > 224.27.27.18: VRRPv2, Advertisement, vrid 37, prio 100, authtype simple, intvl 1s, length 20
10:05:35.390452 IP 192.168.18.98 > 224.27.27.18: VRRPv2, Advertisement, vrid 27, prio 98, authtype simple, intvl 1s, length 20
BACKUP:
[root@cnetos7 keepalived]#ip a
3: ens34: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:50:56:2e:b2:ba brd ff:ff:ff:ff:ff:ff
inet 192.168.18.98/24 brd 192.168.18.255 scope global ens34
valid_lft forever preferred_lft forever
inet 192.168.18.68/32 scope global ens34
valid_lft forever preferred_lft forever
inet 192.168.18.168/32 scope global ens34
valid_lft forever preferred_lft forever
inet6 fe80::c4db:7bad:474f:7c55/64 scope link
valid_lft forever preferred_lft forever
到這里雙主模型的nginx proxy高可用集群也設(shè)置完畢!