【zookeeper學習筆記二】安裝搭建

zookeeper的搭建方式

zookeeper有三種安裝方式:

  1. 單機模式:zookeeper只運行在一臺服務器上,適用于測試環境
  2. 偽集群模式:在一臺物理機上運行多個zookeeper實例
  3. 集群模式:zookeeper運行于一個集群上,適用于生產環境,這個計算機集群被稱為一個"集合體"(ensemble)

單機模式搭建

下載zookeeper

官網:http://mirror.bit.edu.cn/apache/zookeeper/

解壓到安裝目錄

更改配置文件

在conf目錄下,新建zoo.cfg

tickTime=2000
dataDir=/usr/local/zk/data
dataLogDir=/usr/local/zk/dataLog
clientPort=2181

配置環境變量

vi .bash_profile

添加后面兩行的內容

export M2_HOME=/Users/sanbian/Documents/soft/apache-maven-3.3.9/
export PATH=$PATH:$M2_HOME/bin

export ZOOKEEPER_HOME=/usr/local/zk
export PATH=$PATH:$ZOOKEEPER_HOME/bin

啟動zookeeper

在zookeeper的bin目錄下,管理員啟動

sudo sh zkServer.sh start

響應的,關閉:

sudo sh zkServer.sh stop

偽集群模式搭建

要在一臺機器上的部署3個server,我們使用的是:每個配置文檔模擬一臺機器,實現單臺機器上運行多個zookeeper實例。

必須保證的是:每個配置文檔的端口號、dataDir都不能沖突!!!

搭建3個server

按照一下路徑建立3個server:

sanbiandeMacBook-Pro:zookeeper sanbian$ tree -d -L 2
.
├── server1
│   ├── data
│   ├── dataLog
│   └── zk
├── server2
│   ├── data
│   ├── dataLog
│   └── zk
└── server3
    ├── data
    ├── dataLog
    └── zk

12 directories

配置文檔 zoo.cfg

dir相對配置,clientPort也響應配置為2181,2182,2183

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/Users/sanbian/Documents/soft/zookeeper/server3/data

dataLogDir=/Users/sanbian/Documents/soft/zookeeper/server3/dataLog
# the port at which the clients will connect
clientPort=2183
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

server.1=127.0.0.1:8881:7771
server.2=127.0.0.1:8882:7772
server.3=127.0.0.1:8883:7773

各個配置項的含義

tickTime=2000

zookeeper服務器之間或客戶端與服務器之間維持心跳的時間間隔,每隔tickTime時間就會發送一個心跳。

initLimit

配置zookeeper接受客戶端(zookeeper服務器集群中連接到Leader的Follower服務器)初始化連接時最長能忍受多少個心跳時間間隔數。當已超過initLimit*tickTime長度后,zookeeperr服務器若還沒有收到客戶端的返回信息,則表明客戶端連接失敗。

syncLimit

配置Leader與Follower之間發送消息,請求和應答時間長度,最長不能超過syncLimit*tickTime的時間長度

dataDir

zookeeper保存數據的目錄,默認情況下,寫數據的日志也保存在這個目錄

dataLogDir

若沒有的話,則用dataDir.

zookeeper的持久化都存儲在這個目錄里。

dataLogDir放的是順序日志(WAL),而dataDir放的是內存數據結構的snapshot,便于快速回復和達到性能最大化。

clientPort

zookeeper服務器監聽的端口,以接受客戶端的訪問請求。

server.A=B:C:D
  1. A:是一個數字,表示這是第幾號服務器
  2. B:這個服務器的ip地址。
  3. C:這個服務器與集群中的Leader服務器交換信息的端口
  4. D:萬一集群中的Leader服務偽集群器掛了,需要一個端口來重新進行選舉新的Leader,此端口就是用來執行選舉時服務器相互通信的端口。如果是偽集群的配置方式,由于B都是一樣的,所以不同zookeeper實例通信端口號不能一樣,所以要給它們分配不同的端口號。

在data下面新建myid文檔,對應到server.n

sanbiandeMacBook-Pro:zookeeper sanbian$ cd server1/data
sanbiandeMacBook-Pro:data sanbian$ echo "1" myid

啟動服務

sanbiandeMacBook-Pro:zookeeper sanbian$ server3/zk/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server3/zk/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
sanbiandeMacBook-Pro:zookeeper sanbian$ server2/zk/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server2/zk/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
sanbiandeMacBook-Pro:zookeeper sanbian$ server1/zk/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server1/zk/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

查看啟動狀態

sanbiandeMacBook-Pro:zookeeper sanbian$ server1/zk/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server1/zk/bin/../conf/zoo.cfg
Mode: follower
sanbiandeMacBook-Pro:zookeeper sanbian$ server2/zk/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server2/zk/bin/../conf/zoo.cfg
Mode: follower
sanbiandeMacBook-Pro:zookeeper sanbian$ server3/zk/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server3/zk/bin/../conf/zoo.cfg
Mode: leader
sanbiandeMacBook-Pro:zookeeper sanbian$ 

關閉一個server,看其他server狀態

sanbiandeMacBook-Pro:zookeeper sanbian$ server1/zk/bin/zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server1/zk/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
sanbiandeMacBook-Pro:zookeeper sanbian$ server1/zk/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server1/zk/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.
sanbiandeMacBook-Pro:zookeeper sanbian$ server2/zk/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server2/zk/bin/../conf/zoo.cfg
Mode: leader
sanbiandeMacBook-Pro:zookeeper sanbian$ server3/zk/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server3/zk/bin/../conf/zoo.cfg
Mode: follower

可以看到,關閉掉Server1后,server2自動成為了leader

關閉2個server,啟動一個server

sanbiandeMacBook-Pro:zookeeper sanbian$ server3/zk/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server3/zk/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
sanbiandeMacBook-Pro:zookeeper sanbian$ server3/zk/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server3/zk/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.

產生異常信息的原因是由于Zookeeper 服務的每個實例都擁有全局配置信息,他們在啟動的時候會隨時隨地的進行Leader選舉操作。此時,第一個啟動的Zookeeper需要和另外兩個 Zookeeper實例進行通信。但是,另外兩個Zookeeper實例還沒有啟動起來,因此就產生了這的異樣信息。

如果將配置文件zoo.cfg里的關聯其他server的配置項去掉,再重新啟動查看狀態

sanbiandeMacBook-Pro:zookeeper sanbian$ server3/zk/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server3/zk/bin/../conf/zoo.cfg
Mode: standalone

這樣就變成了單機模式(獨立模式)。

所以在啟動三個服務時,先啟動一個服務也會出現這樣的異常,等到第二個啟動成功后,便正常了。

sanbiandeMacBook-Pro:zookeeper sanbian$ server3/zk/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server3/zk/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
sanbiandeMacBook-Pro:zookeeper sanbian$ server3/zk/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server3/zk/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.
sanbiandeMacBook-Pro:zookeeper sanbian$ server2/zk/bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server2/zk/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
sanbiandeMacBook-Pro:zookeeper sanbian$ server3/zk/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server3/zk/bin/../conf/zoo.cfg
Mode: leader
sanbiandeMacBook-Pro:zookeeper sanbian$ server2/zk/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server2/zk/bin/../conf/zoo.cfg
Mode: follower

這個時候再關掉一個Server

sanbiandeMacBook-Pro:zookeeper sanbian$ server3/zk/bin/zkServer.sh stop
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server3/zk/bin/../conf/zoo.cfg
Stopping zookeeper ... STOPPED
sanbiandeMacBook-Pro:zookeeper sanbian$ server2/zk/bin/zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /Users/sanbian/Documents/soft/zookeeper/server2/zk/bin/../conf/zoo.cfg
Error contacting service. It is probably not running.

在選舉leader時,找不到可以通信的Server,就又異常了。

如果先關閉掉follower,也是一樣的狀態。

遇到的坑

Error contacting service. It is probably not running.

網上查了很多辦法,都沒有解決。后來一句話點醒了我,看日志呀!!!日志在哪里呢???zookeeper.out

sanbiandeMacBook-Pro:zookeeper sanbian$ vi server2/zk/bin/zookeeper.out 

2017-09-06 16:49:41,767 [myid:] - INFO  [main:QuorumPeerConfig@134] - Reading configuration from: ../conf/zoo3.cfg
2017-09-06 16:49:41,777 [myid:] - INFO  [main:QuorumPeer$QuorumServer@167] - Resolved hostname: localhost to address: localhost/127.0.0.1
2017-09-06 16:49:41,778 [myid:] - INFO  [main:QuorumPeer$QuorumServer@167] - Resolved hostname: localhost to address: localhost/127.0.0.1
2017-09-06 16:49:41,778 [myid:] - INFO  [main:QuorumPeer$QuorumServer@167] - Resolved hostname: localhost to address: localhost/127.0.0.1
2017-09-06 16:49:41,779 [myid:] - ERROR [main:QuorumPeerMain@85] - Invalid config, exiting abnormally
org.apache.zookeeper.server.quorum.QuorumPeerConfig$ConfigException: Error processing ../conf/zoo3.cfg
        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:154)
        at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:101)
        at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:78)
Caused by: java.lang.IllegalArgumentException: initLimit is not set
        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parseProperties(QuorumPeerConfig.java:355)
        at org.apache.zookeeper.server.quorum.QuorumPeerConfig.parse(QuorumPeerConfig.java:150)
        ... 2 more
Invalid config, exiting abnormally
~                                   

很明顯了吧。我之前的zoo.cfg是自己寫進去的,少了initLimit

tickTime=2000
dataDir=/Users/sanbian/Documents/soft/zookeeper/server3/data
dataLogDir=/Users/sanbian/Documents/soft/zookeeper/server3/dataLog
clientPort=2183
server.1=127.0.0.1:8881:7771
server.2=127.0.0.1:8882:7772
server.3=127.0.0.1:8883:7773

按照zoo_sample.cfg更改:

# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/tmp/zookeeper
# the port at which the clients will connect
clientPort=2181
# the maximum number of client connections.
# increase this if you need to handle more clients
#maxClientCnxns=60
#
# Be sure to read the maintenance section of the
# administrator guide before turning on autopurge.
#
# http://zookeeper.apache.org/doc/current/zookeeperAdmin.html#sc_maintenance
#
# The number of snapshots to retain in dataDir
#autopurge.snapRetainCount=3
# Purge task interval in hours
# Set to "0" to disable auto purge feature
#autopurge.purgeInterval=1

然后。。就解決了!所以有錯還是要先看自己的報錯日志啊

最后編輯于
?著作權歸作者所有,轉載或內容合作請聯系作者
平臺聲明:文章內容(如有圖片或視頻亦包括在內)由作者上傳并發布,文章內容僅代表作者本人觀點,簡書系信息發布平臺,僅提供信息存儲服務。

推薦閱讀更多精彩內容