RocksDB Iterator
??RocksDB Iterator提供用戶以有序的方式前向或者后向遍歷DB,也可以seek 到DB的特定key上。為了做到這樣,Iterator必須以有序流的方式訪問DB。RocksDB Iterator的實現類命名為DBIter。
DBIter
- Implementation: db/db_iter.cc
- Interface: Iterator
??DBIter是InternalIterator的封裝。DBIter的任務就是解析由InternalIterator 暴露出來的InternalKeys,最后以user key的形式展現。
Example:
The underlying InternalIterator exposed
InternalKey(user_key="Key1", seqno=10, Type=Put) | Value = "KEY1_VAL2"
InternalKey(user_key="Key1", seqno=9, Type=Put) | Value = "KEY1_VAL1"
InternalKey(user_key="Key2", seqno=16, Type=Put) | Value = "KEY2_VAL2"
InternalKey(user_key="Key2", seqno=15, Type=Delete) | Value = "KEY2_VAL1"
InternalKey(user_key="Key3", seqno=7, Type=Delete) | Value = "KEY3_VAL1"
InternalKey(user_key="Key4", seqno=5, Type=Put) | Value = "KEY4_VAL1"
But what DBIter will expose to the user is
Key="Key1" | Value = "KEY1_VAL2"
Key="Key2" | Value = "KEY2_VAL2"
Key="Key4" | Value = "KEY4_VAL1"
MergingIterator
- implementation: table/merging_iterator.cc
- Interface: InternalIterator
MergingIterator 包括很多 child iterators,MergingIterator 是一個Iterators堆。在MergingIterator 中,我們把所有的child Iterator放進heap中,以一種有序的流形式暴露。
Example:
The underlying child Iterators exposed:
= Child Iterator 1 =
InternalKey(user_key="Key1", seqno=10, Type=Put) | Value = "KEY1_VAL2"
= Child Iterator 2 =
InternalKey(user_key="Key1", seqno=9, Type=Put) | Value = "KEY1_VAL1"
InternalKey(user_key="Key2", seqno=15, Type=Delete) | Value = "KEY2_VAL1"
InternalKey(user_key="Key4", seqno=5, Type=Put) | Value = "KEY4_VAL1"
= Child Iterator 3 =
InternalKey(user_key="Key2", seqno=16, Type=Put) | Value = "KEY2_VAL2"
InternalKey(user_key="Key3", seqno=7, Type=Delete) | Value = "KEY3_VAL1"
MergingIterator 將所有的child Iterators 放到堆中,以有序流的形式暴露。
InternalKey(user_key="Key1", seqno=10, Type=Put) | Value = "KEY1_VAL2"
InternalKey(user_key="Key1", seqno=9, Type=Put) | Value = "KEY1_VAL1"
InternalKey(user_key="Key2", seqno=16, Type=Put) | Value = "KEY2_VAL2"
InternalKey(user_key="Key2", seqno=15, Type=Delete) | Value = "KEY2_VAL1"
InternalKey(user_key="Key3", seqno=7, Type=Delete) | Value = "KEY3_VAL1"
InternalKey(user_key="Key4", seqno=5, Type=Put) | Value = "KEY4_VAL1"
MemtableIterator
- Implementation: db/memtable.cc
- Interface: InternalIterator
MemtableIterator是MemtableRep::Iterator的封裝,內存表實現自己Iterator都是以keys/values的有序流的形式暴露。
BlockIter
- Implementation:table/block.h
- Interface: InternalIterator
這個Iterator用來從SST file中讀取blocks,與blocks是Index blocks或者data block是無關。由于,SST file blocks是有序的且不可更變的,所以我們可以load block數據到memory總,然后為這個有序的數據創建一個BlockIter。
TwoLevelIterator
- Implementation: table/two_level_iterator.cc
- Interface: InternalIterator
一個TwoLevelIterator 包含兩個Iterators
- First level Iterator (first_level_iter_)
- Second level Iterator (second_level_iter_)
first_level_iter_用來計算出接下來要使用的second_level_iter_,second_level_iter_指向我們要讀取的實際數據。
Example:
RocksDB使用TwoLevelIterator 來讀取SST file,first_level_iter_ 是SST File Index block上的BlockIter,second_level_iter_ 是Data Block上的BlockIter。
下面看一個簡單的SST file,有4個Data blocks和1個Index Block
[Data block, offset: 0x0000]
KEY1 | VALUE1
KEY2 | VALUE2
KEY3 | VALUE3
[Data Block, offset: 0x0100]
KEY4 | VALUE4
KEY7 | VALUE7
[Data Block, offset: 0x0250]
KEY8 | VALUE8
KEY9 | VALUE9
[Data Block, offset: 0x0350]
KEY11 | VALUE11
KEY15 | VALUE15
[Index Block, offset: 0x0500]
KEY3 | 0x0000
KEY7 | 0x0100
KEY9 | 0x0250
KEY15 | 0x0500
要想讀這個file,我們要創建一個TwoLevelIterator
- first_level_iter_ => BlockIter over Index block
- second_level_iter_ => BlockIter over Data block that will be determined by first_level_iter_
當通過TwoLevelIterator seek到KEY8時,首先使用first_level_iter_ (BlockIter over Index block)計算出哪個block可能包含這個key,結果是設置second_level_iter_ to be (BlockIter over data block with offset 0x0250)。然后我們使用second_level_iter_ 在data block中讀取key &value