- 創(chuàng)建 increase-replication-factor.json 文件
$ cat increase-replication-factor.json
{"version":1, "partitions":[
{"topic":"testTopic","partition":0,"replicas":[0,1,2]},
{"topic":"testTopic","partition":1,"replicas":[0,1,2]},
{"topic":"testTopic","partition":2,"replicas":[0,1,2]},
{"topic":"testTopic","partition":3,"replicas":[0,1,2]},
{"topic":"testTopic","partition":4,"replicas":[0,1,2]},
{"topic":"testTopic","partition":5,"replicas":[0,1,2]},
{"topic":"testTopic","partition":6,"replicas":[0,1,2]},
{"topic":"testTopic","partition":7,"replicas":[0,1,2]},
{"topic":"testTopic","partition":8,"replicas":[0,1,2]},
{"topic":"testTopic","partition":9,"replicas":[0,1,2]},
{"topic":"testTopic","partition":10,"replicas":[0,1,2]},
{"topic":"testTopic","partition":11,"replicas":[0,1,2]},
{"topic":"testTopic","partition":12,"replicas":[0,1,2]},
{"topic":"testTopic","partition":13,"replicas":[0,1,2]},
{"topic":"testTopic","partition":14,"replicas":[0,1,2]},
{"topic":"testTopic","partition":15,"replicas":[0,1,2]}]
}
- 執(zhí)行增加副本操作
$ bin/kafka-reassign-partitions.sh --zookeeper xx.xx.xx.xx:2181 --reassignment-json-file increase-replication-factor.json --execute
- 查看執(zhí)行進度
$ /bin/kafka-reassign-partitions.sh --zookeeper xx.xx.xx.xx:2181 --reassignment-json-file increase-replication-factor.json --verify
Status of partition reassignment:
Reassignment of partition testTopic-15 completed successfully
Reassignment of partition testTopic-2 completed successfully
Reassignment of partition testTopic-7 completed successfully
Reassignment of partition testTopic-12 completed successfully
Reassignment of partition testTopic-4 completed successfully
Reassignment of partition testTopic-11 completed successfully
Reassignment of partition testTopic-10 completed successfully
Reassignment of partition testTopic-8 completed successfully
Reassignment of partition testTopic-5 completed successfully
Reassignment of partition testTopic-13 completed successfully
Reassignment of partition testTopic-1 completed successfully
Reassignment of partition testTopic-14 completed successfully
Reassignment of partition testTopic-9 completed successfully
Reassignment of partition testTopic-6 completed successfully
Reassignment of partition testTopic-0 completed successfully
Reassignment of partition testTopic-3 completed successfully
要熟練使用 Kafka 自帶的 kafka-reassign-partitions.sh 腳本工具來完成對 topic 的分區(qū)分配、分區(qū)副本增加等操作。
該腳本有三個參數(shù)：
--generate：配合著 --topics-to-move-json-file 可以生成分區(qū)分配策略，該參數(shù)適用于分區(qū)多的情況
--execute： 配合著 --reassignment-json-file 可以執(zhí)行分區(qū)分配策略
--verify： 配合著 --reassignment-json-file 可以檢查分區(qū)分配進度
通過以上命令，既可以分配分區(qū)，也可以增加分區(qū)副本數(shù)，非常方便。
參考
kafka 遷移分區(qū)創(chuàng)建規(guī)則json
https://www.orchome.com/454#item-8
kafka命令大全
https://www.orchome.com/454#item-8
kafka集群中有多個offsets.topic.replication.factor？
https://cloud.tencent.com/developer/ask/sof/1540657/answer/2102855
kafka高可用失敗問題 3broker，單殺一個broker就不能消費的問題探討
https://www.orchome.com/805
