kafka优选副本切换办法
1. 以topic test为例,假设test的分布为以下状态。
Topic:test PartitionCount:3 ReplicationFactor:2 Configs:
Topic: test Partition: 0 Leader: 1 Replicas: 1,2 Isr: 1,2
Topic: test Partition: 1 Leader: 2 Replicas: 2,3 Isr: 2,3
Topic: test Partition: 2 Leader: 3 Replicas: 3,1 Isr: 1,3
2. 假设现在broker-1节点出现问题,希望parititon-0的优先副本发生切换,即由Replicas: 1,2变为
Replicas: 2,1,于此同时,leader由1切换到2。
首先编写迁移文件move.json,内容如下:
{"partitions": [{"topic": "test", "partition": 0, "replicas": [2,1] }],"version":1}
其中,文件的意义是:修改调换partition 0 的replicas的顺序为[2,1]。
3. 执行迁移脚本
sh kafka-reassign-partitions.sh --zookeeper zookeeper_IP:24002/kafka --reassignment-json-file move.json --execute
执行完成后的结果为:
Topic:test PartitionCount:3 ReplicationFactor:2 Configs:
Topic: test Partition: 0 Leader: 1 Replicas: 2,1 Isr: 1,2
Topic: test Partition: 1 Leader: 2 Replicas: 2,3 Isr: 2,3
Topic: test Partition: 2 Leader: 3 Replicas: 3,1 Isr: 1,3
此时,partition-0的副本顺序变为了Replicas: 2,1
构建副本切换文件
创建excution.json,内容为:
{"partitions":[{"topic": "test", "partition": 0}]}
执行副本切换脚本
kafka-preferred-replica-election.sh --zookeeper zookeeper_IP:24002/kafka --path-to-json-file excution.json
执行后结果如下:
Topic:test PartitionCount:3 ReplicationFactor:2 Configs:
Topic: test Partition: 0 Leader: 2 Replicas: 2,1 Isr: 1,2
Topic: test Partition: 1 Leader: 2 Replicas: 2,3 Isr: 2,3
Topic: test Partition: 2 Leader: 3 Replicas: 3,1 Isr: 1,3
此时,leader已切换broker-2
kafka-preferred-replica-election.sh 可以不加后面的副本切换脚本而直接执行,例如:
kafka-preferred-replica-election.sh --zookeeper zookeeper_IP:2181/kafka,这样会将真个集群的leader全部调整到优选副本