一個消息中間件最核心的東西就是消息存儲結構。
這是kafka的消息存儲：
clipboard.png
每個topic_partition對應一個日志文件，Producer對該日志文件進行“順序寫”，Consumer對該文件進行“順序讀”。這種存儲方式，對于每個文件來說是順序IO，但是當并發的讀寫多個partition的時候，對應多個文件的順序IO，表現在文件系統的磁盤層面，還是隨機IO。
因此出現了當partition或者topic個數過多時，Kafka的性能急劇下降。參見http://blog.csdn.net/chunlongyu/article/details/53913758
RocketMq的消息存儲：
RocketMQ采用了單一的日志文件，即把同1臺機器上面所有topic的所有queue的消息，存放在一個文件里面，從而避免了隨機的磁盤寫入。
clipboard.png
如上圖所示，所有消息都存在一個單一的CommitLog文件里面，然后有后臺線程異步的同步到ConsumeQueue，再由Consumer進行消費。
這里至所以可以用“異步線程”，也是因為消息隊列天生就是用來“緩沖消息”的。只要消息到了CommitLog，發送的消息也就不會丟。只要消息不丟，那就有了“充足的回旋余地”，用一個后臺線程慢慢同步到ConsumeQueue，再由Consumer消費。
可以說，這也是在消息隊列內部的一個典型的“最終一致性”的案例：Producer發了消息，進了CommitLog，此時Consumer并不可見。但沒關系，只要消息不丟，消息最終肯定會進入ConsumeQueue，讓Consumer可見。
深入看一下這兩個結構。
ConsumeQueue
ConsumeQueue的存儲位置
默認的存儲位置：${user.home} \store\consumequeue${topicName}${queueId}${fileName}
可以修改：配置文件的
storePathRootDir=/home/haieradmin/mqstore/rocketmqstore
storePathCommitLog=/home/haieradmin/mqstore/rocketmqstore/commitlog
這兩個參數。
測試先發送100個消息：
public class SyncProducer {
public static void main(String[] args) throws Exception {
DefaultMQProducer producer = new
DefaultMQProducer("produce1");
producer.setNamesrvAddr("10.135.17.26:9876;10.135.17.27:9876");
producer.start();
for (int i = 0; i < 100; i++) {
Message msg = new Message("DemoTopic", "TagA",
("Hello RocketMQ").getBytes()
);
SendResult sendResult = producer.send(msg);
System.out.printf("%s%n", sendResult);
}
producer.shutdown();
}
發送100個完全一樣的消息
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128085F6B4, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=0], queueOffset=0]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128085FD31, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=1], queueOffset=0]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F00000012808606D9, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=2], queueOffset=0]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128086075F, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=3], queueOffset=0]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270BE8E7, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=0], queueOffset=0]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270BE96D, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=1], queueOffset=0]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270BEC87, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=2], queueOffset=0]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270BED0D, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=3], queueOffset=0]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128086CB34, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=0], queueOffset=1]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128086CFB4, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=1], queueOffset=1]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128086D811, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=2], queueOffset=1]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128086DA94, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=3], queueOffset=1]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270CA577, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=0], queueOffset=1]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270CD5A9, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=1], queueOffset=1]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270CD62F, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=2], queueOffset=1]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270CE170, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=3], queueOffset=1]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F0000001280889405, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=0], queueOffset=2]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F0000001280889FDF, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=1], queueOffset=2]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088A262, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=2], queueOffset=2]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088A976, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=3], queueOffset=2]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270CFEF4, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=0], queueOffset=2]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270CFF7A, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=1], queueOffset=2]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D0000, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=2], queueOffset=2]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D031A, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=3], queueOffset=2]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088AE8D, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=0], queueOffset=3]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088AF13, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=1], queueOffset=3]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088AF99, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=2], queueOffset=3]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088B01F, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=3], queueOffset=3]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D064E, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=0], queueOffset=3]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D06D4, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=1], queueOffset=3]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D075A, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=2], queueOffset=3]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D07E0, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=3], queueOffset=3]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088B354, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=0], queueOffset=4]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088B3DA, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=1], queueOffset=4]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088B460, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=2], queueOffset=4]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088B795, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=3], queueOffset=4]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D0DC4, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=0], queueOffset=4]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D0E4A, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=1], queueOffset=4]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D117F, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=2], queueOffset=4]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D14B4, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=3], queueOffset=4]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088B81B, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=0], queueOffset=5]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088B8A1, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=1], queueOffset=5]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088B927, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=2], queueOffset=5]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088B9AD, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=3], queueOffset=5]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D17E9, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=0], queueOffset=5]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D186F, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=1], queueOffset=5]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D1BA4, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=2], queueOffset=5]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D1C2A, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=3], queueOffset=5]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088BA33, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=0], queueOffset=6]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088BAB9, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=1], queueOffset=6]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088BB3F, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=2], queueOffset=6]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088BBC5, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=3], queueOffset=6]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D1CB0, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=0], queueOffset=6]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D1D36, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=1], queueOffset=6]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D1DBC, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=2], queueOffset=6]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D20F1, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=3], queueOffset=6]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088BC4B, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=0], queueOffset=7]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088BCD1, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=1], queueOffset=7]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088BD57, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=2], queueOffset=7]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088BDDD, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=3], queueOffset=7]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D2177, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=0], queueOffset=7]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D21FD, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=1], queueOffset=7]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D2283, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=2], queueOffset=7]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D259D, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=3], queueOffset=7]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088C7A0, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=0], queueOffset=8]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088C826, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=1], queueOffset=8]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088CAA9, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=2], queueOffset=8]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088CDDE, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=3], queueOffset=8]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D2B66, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=0], queueOffset=8]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D2BEC, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=1], queueOffset=8]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D2F21, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=2], queueOffset=8]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D2FA7, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=3], queueOffset=8]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088DC32, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=0], queueOffset=9]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088DCB8, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=1], queueOffset=9]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088E90F, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=2], queueOffset=9]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128088EE26, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=3], queueOffset=9]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D3570, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=0], queueOffset=9]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D3B38, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=1], queueOffset=9]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D3BBE, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=2], queueOffset=9]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D3ED8, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=3], queueOffset=9]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F0000001280890074, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=0], queueOffset=10]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128089058B, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=1], queueOffset=10]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128089080E, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=2], queueOffset=10]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F0000001280890D25, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=3], queueOffset=10]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D49E4, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=0], queueOffset=10]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D4A6A, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=1], queueOffset=10]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D4AF0, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=2], queueOffset=10]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D4B76, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=3], queueOffset=10]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F0000001280890DAB, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=0], queueOffset=11]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F00000012808912C2, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=1], queueOffset=11]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F0000001280891545, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=2], queueOffset=11]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F00000012808915CB, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=3], queueOffset=11]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D4E8F, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=0], queueOffset=11]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D4F15, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=1], queueOffset=11]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D4F9B, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=2], queueOffset=11]
SendResult [sendStatus=SEND_OK, msgId=0A87111B00002A9F00000003270D5021, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-b, queueId=3], queueOffset=11]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F0000001280891651, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=0], queueOffset=12]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F00000012808916D7, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=1], queueOffset=12]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F000000128089175D, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=2], queueOffset=12]
SendResult [sendStatus=SEND_OK, msgId=0A87111A00002A9F00000012808917E3, messageQueue=MessageQueue [topic=DemoTopic, brokerName=broker-a, queueId=3], queueOffset=12]
可以看出，這100條消息時分別發送到兩個broker上
去看一下
clipboard.png
為什么是4個隊列。可以看這篇。http://www.lxweimin.com/p/ccdf6fc710b0
進去一個隊列，發現有一個00000000000000000000的文件。
順便說一下，消息隊列文件名規則：
commitlog文件的默認存儲大小1G，不是ConsumeQueue的
文件名以已有存儲容量依次遞增，類似如下：
00000000000000000000
00000000001073741824
00000000002147483648
。。。
這是個二進制文件，打開看一下。
[haieradmin@IBMMQ03 0]$ od -Ax -tx1 00000000000000000000
000000 00 00 00 12 80 85 f6 b4 00 00 00 86 00 00 00 00
000010 00 27 a8 07 00 00 00 12 80 86 cb 34 00 00 00 86
000020 00 00 00 00 00 27 a8 07 00 00 00 12 80 88 94 05
000030 00 00 00 86 00 00 00 00 00 27 a8 07 00 00 00 12
000040 80 88 ae 8d 00 00 00 86 00 00 00 00 00 27 a8 07
000050 00 00 00 12 80 88 b3 54 00 00 00 86 00 00 00 00
000060 00 27 a8 07 00 00 00 12 80 88 b8 1b 00 00 00 86
000070 00 00 00 00 00 27 a8 07 00 00 00 12 80 88 ba 33
000080 00 00 00 86 00 00 00 00 00 27 a8 07 00 00 00 12
000090 80 88 bc 4b 00 00 00 86 00 00 00 00 00 27 a8 07
0000a0 00 00 00 12 80 88 c7 a0 00 00 00 86 00 00 00 00
0000b0 00 27 a8 07 00 00 00 12 80 88 dc 32 00 00 00 86
0000c0 00 00 00 00 00 27 a8 07 00 00 00 12 80 89 00 74
0000d0 00 00 00 86 00 00 00 00 00 27 a8 07 00 00 00 12
0000e0 80 89 0d ab 00 00 00 86 00 00 00 00 00 27 a8 07
0000f0 00 00 00 12 80 89 16 51 00 00 00 86 00 00 00 00
000100 00 27 a8 07 00 00 00 00 00 00 00 00 00 00 00 00
000110 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
解釋一下：
最左邊的是這行數據開始的地址。16進制。比如第一行000000開始于第0個字節。第二行000010開始于第16個字節，這只是為了顯示給你看的，方便數字節。
ConsumeQueue中每個消息時20Byte長。結構為
clipboard.png
CommitLog Offset是指這條消息在Commit Log文件中的實際偏移量
Size存儲中消息的大小
Message Tag HashCode存儲消息的Tag的哈希值：主要用于訂閱時消息過濾（訂閱時如果指定了Tag，會根據HashCode來快速查找到訂閱的消息）
所以
clipboard.png
上面藍框是一條ConsumeQueue消息，紅色的00000086是對應真實消息的size。
我之前故意發送了全部相同的消息，所以size都是一樣的。因為tag是一樣的，所以Message Tag HashCode也是一樣的。
可以數一下，這個ConsumeQueue中現在一共13條數據，和前面的日志顯示是一樣的。
測一下消費
package com.yunsheng.simpleExample;
import com.alibaba.rocketmq.client.consumer.DefaultMQPushConsumer;
import com.alibaba.rocketmq.client.consumer.listener.ConsumeConcurrentlyContext;
import com.alibaba.rocketmq.client.consumer.listener.ConsumeConcurrentlyStatus;
import com.alibaba.rocketmq.client.consumer.listener.MessageListenerConcurrently;
import com.alibaba.rocketmq.client.exception.MQClientException;
import com.alibaba.rocketmq.common.consumer.ConsumeFromWhere;
import com.alibaba.rocketmq.common.message.MessageExt;
import java.util.List;
public class SyncConsumer {
public static void main(String[] args) throws InterruptedException, MQClientException {
//聲明并初始化一個consumer
DefaultMQPushConsumer consumer = new DefaultMQPushConsumer("consumer1");
consumer.setNamesrvAddr("10.135.17.26:9876;10.135.17.27:9876");
//這里設置的是一個consumer的消費策略
//CONSUME_FROM_LAST_OFFSET 默認策略，從該隊列最尾開始消費，即跳過歷史消息
//CONSUME_FROM_FIRST_OFFSET 從隊列最開始開始消費，即歷史消息（還儲存在broker的）全部消費一遍
//CONSUME_FROM_TIMESTAMP 從某個時間點開始消費，和setConsumeTimestamp()配合使用，默認是半個小時以前
consumer.setConsumeFromWhere(ConsumeFromWhere.CONSUME_FROM_LAST_OFFSET);
//設置consumer所訂閱的Topic和Tag，*代表全部的Tag
consumer.subscribe("DemoTopic", "*");
//設置一個Listener，主要進行消息的邏輯處理
consumer.registerMessageListener(new MessageListenerConcurrently() {
public ConsumeConcurrentlyStatus consumeMessage(List<MessageExt> msgs,
ConsumeConcurrentlyContext context) {
//
System.out.println(Thread.currentThread().getName() + " Receive New Messages: " + msgs);
// if (msgs.size() > 0){
// System.out.println(new String(msgs.get(0).getBody()));
// }
//返回消費狀態
//CONSUME_SUCCESS 消費成功
//RECONSUME_LATER 消費失敗，需要稍后重新消費
return ConsumeConcurrentlyStatus.CONSUME_SUCCESS;
}
});
//調用start()方法啟動consumer
consumer.start();
System.out.println("Consumer Started.");
}
}
看下輸出：
ConsumeMessageThread_4 Receive New Messages: [MessageExt [queueId=3, storeSize=134, queueOffset=2, sysFlag=0, bornTimestamp=1507713810962, bornHost=/192.168.116.77:51970, storeTimestamp=1507713811012, storeHost=/10.135.17.26:10911, msgId=0A87111A00002A9F000000128088A976, commitLogOffset=79465851254, bodyCRC=1774740973, reconsumeTimes=0, preparedTransactionOffset=0, toString()=Message [topic=DemoTopic, flag=0, properties={MIN_OFFSET=0, MAX_OFFSET=13, WAIT=true, TAGS=TagA}, body=14]]]
現在查下ConsumeQueue，恩，都還在的。
不是消費過了嗎？怎么還有？
但是現在再運行SyncConsumer ，并不會消費消息。
所以，一定是有某種手段保存了消費的進度。這種手段就是，在DefaultMQPushConsumer實例啟動時，會到broker上，拿到本消費組broker已經記錄好的消費進度（consumer offset），按照這個進度發起自己的第一次Pull請求。
所以，現在換一個consumerGroup的名字，會發現又消費了。
RocketMq的刪除機制。RocketMq并不會立即刪除消息，所以消息是可以被重復消費的。 RocketMq的消息時定期清除，默認3天。
