Kafka不同進程“溝通機制”
Kafka服務的定位是一種高吞吐量的分布式消息訂閱系統(tǒng)。服務再運行過程中，不同的進程（broker和controller之間，客戶端和集群）之間需要進行“溝通”來保證功能的可用性。
Kafka主要通過兩種方式進行“溝通”，以保證自身狀態(tài)或需求被感知：
通過nio方式完成“溝通”
controller通過ControllerChannelManager發(fā)送消息到各broker，主要有ApiKeys.LEADER_AND_ISR, ApiKeys.STOP_REPLICA, ApiKeys.UPDATE_METADATA_KEY 三種消息。其中僅有ApiKeys.STOP_REPLICA有相關callback操作。broker通過KafkaRequestHandlerPool接收消息并調用kafkaApi進行消息相應。其他如producer發(fā)送消息，consumser消費消息，獲取集群信息等操作，也是客戶端通過nio的方式發(fā)送請求。Broker可處理的消息有如下：
| 消息 | 備注 |
|---|---|
| ApiKeys.LEADER_AND_ISR | Controller 發(fā)送給broker |
| ApiKeys.STOP_REPLICA | Controller 發(fā)送給broker |
| ApiKeys.UPDATE_METADATA_KEY | Controller 發(fā)送給broker |
| ApiKeys.PRODUCE | producer發(fā)送生產(chǎn)數(shù)據(jù)請求給broker |
| ApiKeys.FETCH | consumer發(fā)送消費數(shù)據(jù)請求給broker |
| ApiKeys.LIST_OFFSETS | 客戶端／consumer 發(fā)送查詢offset請求 |
| ApiKeys.METADATA | 獲取集群topic信息 |
| ApiKeys.CONTROLLED_SHUTDOWN_KEY | brokershutdown時向controller發(fā)送消息 |
| ApiKeys.OFFSET_COMMIT | consumer向coordinator發(fā)送的offset commit 請求 |
| ApiKeys.OFFSET_FETCH | consumer發(fā)送的請求partitions的當前的offset |
| ApiKeys.GROUP_COORDINATOR | consumer向broker發(fā)送group的coordinator是哪個節(jié)點的請求 |
| ApiKeys.JOIN_GROUP | consumer向groupcoordinator發(fā)送 join group的請求 |
| ApiKeys.HEARTBEAT | consumer定期發(fā)送heartbeat請求到groupcoordinator |
| ApiKeys.LEAVE_GROUP | consumer停止消費時發(fā)送退出group請求到groupcoordinator |
| ApiKeys.SYNC_GROUP | consumer變?yōu)閘eader獲取follower時發(fā)送的已同步請求到groupcoordinator |
| ApiKeys.DESCRIBE_GROUPS | 客戶端工具kafka-consumer-groups.sh ConsumerGroupCommand中使用describe group信息所用 |
| ApiKeys.LIST_GROUPS | 客戶端工具kafka-consumer-groups.sh ConsumerGroupCommand中使用list group信息所用 |
| ApiKeys.SASL_HANDSHAKE | nio的select發(fā)送消息/請求時，需要先發(fā)送sasl——handshake請求 |
| ApiKeys.API_VERSIONS | 客戶端發(fā)送的kafka版本請求 |
通過zookeeper注冊監(jiān)聽器完成“溝通”
客戶端（broker／client）如果需要將訴求（如topic創(chuàng)建，刪除，reblance，partition reassign等）告知服務，所采取的方案是再ZooKeeper中創(chuàng)建目錄，寫入數(shù)據(jù)，服務端（此處指controller）通過listener監(jiān)控到zk中的變化，并作出響應
Zookeeper的watch機制與第三方封裝
Zookeeper是分布式應用程序協(xié)調服務。不同的進程通過約定的目錄的寫入和監(jiān)控操作完成消息的互相感知。開源大數(shù)據(jù)多數(shù)服務的高可用性實現(xiàn)都是依靠zookeeper的協(xié)調功能完成。zookeeper客戶端通過watcher機制完成對zookeeper的集群的監(jiān)控。
由于zookeeper應用開發(fā)的復雜性，出現(xiàn)了第三方對zk的應用開發(fā)的封裝。常用的有Curator FrameWork，I0Itec-zkclient。其中hive，hbase，spark等都是通過Curator FrameWork完成和zk的交互。Kafka則選用的是I0Itec-zkclient與zk交互。
Kafka中實現(xiàn)listener的方法：
- Kafka通過zkUtils中調用org.I0Itec.zkclient.zkCLient完成listener(listener中相關的handle方法有相關的處理邏輯)的注冊。
- zkClient繼承Watcher，再監(jiān)控的對象發(fā)生變化時，zkCLient中的process方法會被調用
- 在該方法中會完成對不同的listener的handle方法的調用
I0Itec-zkclient主要有如下三種listener，分別實現(xiàn)不同的監(jiān)聽方式。
| 消息 | handle的方法 | 備注 |
|---|---|---|
| IZkChildListener | handleChildChange | listening on zk child changes for a given path |
| IZkDataListener | handleDataChange | listening on zk data changes for a given path |
| IZkDataListener | handleDataDeleted | listening on zk data changes for a given path |
| IZkStateListener | handleStateChanged | Called when the zookeeper connection state has changed. |
| IZkStateListener | handleNewSession | Called after the zookeeper session has expired and a new session has been created |
| IZkStateListener | handleSessionEstablishmentError | Called when a session cannot be re-established |
| listener接口 | listener | listener 模塊 | 監(jiān)控對象 | 備注 |
|---|---|---|---|---|
| IZkChildListener | DeleteTopicsListener | PartitionStateMachine | DeleteTopicsPath = "/admin/delete_topics" | |
| IZkChildListener | IsrChangeNotificationListener | KafkaController | IsrChangeNotificationPath = "/isr_change_notification" | |
| IZkChildListener | zkTopicEventListener | ZookeeperTOpicEventWatcher | BrokerTopicsPath = "/brokers/topics" | |
| IZkChildListener | NodeChangeListener | ZkNodeCHangeNotificaionListener in SimpleAclAuthorizer | AclChangedZkPath = "/kafka-acl-changes" AclChangedPrefix = "acl_changes_" | |
| IZkChildListener | NodeChangeListener | ZkNodeCHangeNotificaionListener In DynamicConfigManager | EntityConfigChangesPath = "/config/changes" EntityConfigChangeZnodePrefix = "config_change_" | |
| IZkChildListener | ZKRebalancerListener | ZookeeperConsumerConnector | BrokerIdsPath = "/brokers/ids" | 同時也會被由ZKTopicPartitionChangeListener觸發(fā)，然后調用syncedRebalance進行rebalance |
| IZkChildListener | TopicChangeListener | PartitionStateMachine | BrokerTopicsPath = "/brokers/topics" | |
| IZkChildListener | BrokerChangeListener | ReplicaStateMachine | BrokerIdsPath = "/brokers/ids" | |
| IZKDateListener | PreferredReplicaElectionListener | KafkaController | PreferredReplicaLeaderElectionPath = "/admin/preferred_replica_election" | |
| IZKDateListener | ZKTopicPartitionChangeListener | ZookeeperConsumerConnector | "/brokers/topics/${topic_name}" | consumer監(jiān)控的partition的變化并作出處理。 高級api |
| IZKDateListener | PartitionsReassignedListener | KafkaController | ReassignPartitionsPath = "/admin/reassign_partitions" | |
| IZKDateListener | LeaderChangeListener | ZookeeperLeaderElector | ControllerPath = "/controller" | Kafka的broker服務的升controller的競選目錄 |
| IZKDateListener | ReassignedPartitionsIsrChangeListener | KafkaController | "/brokers/topics/${topic_name}/partitions/${partition_id}/state" | |
| IZKDateListener | PartitionModificationsListener | PartitionStateMachine | "/brokers/topics/${topic_name} | |
| IZkStateListener | SessionExpireListener | KafkaHealthcheck | kafka健康檢查會檢查進程和zk的session狀態(tài) | |
| IZkStateListener | ZKSessionExpireListener | ZookeeperConsumerConnector | consumser初始化時注冊，在新建session時，也會調用到用syncedRebalance進行rebalance | |
| IZkStateListener | SessionExpirationListener | KafkaController | kafkacontroller會監(jiān)聽與zk的鏈接狀態(tài)，如果新建session，會調用onControllerResignation，并重新競選leader | |
| IZkStateListener | ZkStateChangeListener | ZkNodeChangeNotificationListener In SimpleAclAuthorizer | AclChangedZkPath = "/kafka-acl-changes" AclChangedPrefix = "acl_changes_" | 在新建seesion時會通過注冊的notificaionhandler處理所有notifications |
| IZkStateListener | ZkStateChangeListener | ZkNodeChangeNotificationListener In DynamicConfigManager | EntityConfigChangesPath = "/config/changes" EntityConfigChangeZnodePrefix = "config_change_" | 在新建seesion時會通過注冊的notificaionhandler處理所有notifications |
| IZkStateListener | ZkSessionExpireListener | ZookeeperTopicEventWatcher | 消費時監(jiān)控和zk的鏈接狀態(tài)，低級api有效 |
Kafka內部有大量的listener機制完成不同的功能。
| listener接口 | listener | listener 模塊 | 監(jiān)控對象 | 備注 |
|---|---|---|---|---|
| IZkChildListener | DeleteTopicsListener | PartitionStateMachine | DeleteTopicsPath = "/admin/delete_topics" | |
| IZkChildListener | IsrChangeNotificationListener | KafkaController | IsrChangeNotificationPath = "/isr_change_notification" | |
| IZkChildListener | zkTopicEventListener | ZookeeperTOpicEventWatcher | BrokerTopicsPath = "/brokers/topics" | |
| IZkChildListener | NodeChangeListener | ZkNodeCHangeNotificaionListener in SimpleAclAuthorizer | AclChangedZkPath = "/kafka-acl-changes" AclChangedPrefix = "acl_changes_" | |
| IZkChildListener | NodeChangeListener | ZkNodeCHangeNotificaionListener In DynamicConfigManager | EntityConfigChangesPath = "/config/changes" EntityConfigChangeZnodePrefix = "config_change_" | |
| IZkChildListener | ZKRebalancerListener | ZookeeperConsumerConnector | BrokerIdsPath = "/brokers/ids" | 同時也會被由ZKTopicPartitionChangeListener觸發(fā)，然后調用syncedRebalance進行rebalance |
| IZkChildListener | TopicChangeListener | PartitionStateMachine | BrokerTopicsPath = "/brokers/topics" | |
| IZkChildListener | BrokerChangeListener | ReplicaStateMachine | BrokerIdsPath = "/brokers/ids" | |
| IZKDateListener | PreferredReplicaElectionListener | KafkaController | PreferredReplicaLeaderElectionPath = "/admin/preferred_replica_election" | |
| IZKDateListener | ZKTopicPartitionChangeListener | ZookeeperConsumerConnector | "/brokers/topics/${topic_name}" | consumer監(jiān)控的partition的變化并作出處理。 高級api |
| IZKDateListener | PartitionsReassignedListener | KafkaController | ReassignPartitionsPath = "/admin/reassign_partitions" | |
| IZKDateListener | LeaderChangeListener | ZookeeperLeaderElector | ControllerPath = "/controller" | Kafka的broker服務的升controller的競選目錄 |
| IZKDateListener | ReassignedPartitionsIsrChangeListener | KafkaController | "/brokers/topics/${topic_name}/partitions/${partition_id}/state" | |
| IZKDateListener | PartitionModificationsListener | PartitionStateMachine | "/brokers/topics/${topic_name} | |
| IZkStateListener | SessionExpireListener | KafkaHealthcheck | kafka健康檢查會檢查進程和zk的session狀態(tài) | |
| IZkStateListener | ZKSessionExpireListener | ZookeeperConsumerConnector | consumser初始化時注冊，在新建session時，也會調用到用syncedRebalance進行rebalance | |
| IZkStateListener | SessionExpirationListener | KafkaController | kafkacontroller會監(jiān)聽與zk的鏈接狀態(tài)，如果新建session，會調用onControllerResignation，并重新競選leader | |
| IZkStateListener | ZkStateChangeListener | ZkNodeChangeNotificationListener In SimpleAclAuthorizer | AclChangedZkPath = "/kafka-acl-changes" AclChangedPrefix = "acl_changes_" | 在新建seesion時會通過注冊的notificaionhandler處理所有notifications |
| IZkStateListener | ZkStateChangeListener | ZkNodeChangeNotificationListener In DynamicConfigManager | EntityConfigChangesPath = "/config/changes" EntityConfigChangeZnodePrefix = "config_change_" | 在新建seesion時會通過注冊的notificaionhandler處理所有notifications |
| IZkStateListener | ZkSessionExpireListener | ZookeeperTopicEventWatcher | 消費時監(jiān)控和zk的鏈接狀態(tài)，低級api有效 |
