TransportSingleShardAction的主要作用是處理來自客戶端的單個(gè)分片請(qǐng)求。TransportSingleShardAction會(huì)負(fù)責(zé)將請(qǐng)求發(fā)送到正確的節(jié)點(diǎn)，以便執(zhí)行相關(guān)操作。
以下是TransportSingleShardAction的主要作用：
確定請(qǐng)求要訪問的分片：當(dāng)客戶端向集群發(fā)送請(qǐng)求時(shí)，TransportSingleShardAction會(huì)解析請(qǐng)求并確定應(yīng)該訪問哪個(gè)分片。它會(huì)檢查請(qǐng)求中的索引、分片號(hào)等信息，然后將請(qǐng)求路由到正確的節(jié)點(diǎn)上。
處理請(qǐng)求并返回結(jié)果：一旦請(qǐng)求到達(dá)正確的節(jié)點(diǎn)，TransportSingleShardAction將處理請(qǐng)求并返回結(jié)果。它會(huì)從節(jié)點(diǎn)中檢索數(shù)據(jù)、執(zhí)行搜索、更新索引等操作，并將結(jié)果返回給客戶端。
處理故障情況：如果在處理請(qǐng)求時(shí)發(fā)生故障，TransportSingleShardAction會(huì)處理異常并返回錯(cuò)誤消息給客戶端。它還能夠檢測(cè)到節(jié)點(diǎn)故障并嘗試將請(qǐng)求路由到備用節(jié)點(diǎn)上。
總的來說，TransportSingleShardAction負(fù)責(zé)將請(qǐng)求路由到正確的分片上并執(zhí)行相關(guān)操作.
TransportSingleShardAction 能處理的請(qǐng)求, 都是針對(duì)單個(gè)shard的請(qǐng)求SingleShardRequest
TransportSingleShardAction<Request extends SingleShardRequest<Request>, Response extends ActionResponse> 從這里也可以看出: Request extends SingleShardRequest<Request> 這類范型在這里的作用, 其實(shí)想象表達(dá)子類都必須是 SingleShardRequest
- Get
- MultiGet
- TermVectors
- Analyze
- GetField
- Explain
- .....
公共初始化依賴:
- ClusterService
- IndexNameExpressionResolver
- ActionFilters
- ...
提供給子類的抽象方法:
abstract Response shardOperation(Request request, ShardId shardId) throws IOException;abstract boolean resolveIndex(Request request);void resolveRequest(ClusterState state, InternalRequest request)-
abstract ShardsIterator shards(ClusterState state, InternalRequest request);計(jì)算可以執(zhí)行操作的候選shards, 如果返回null, 可能是本地執(zhí)行了 String getExecutor(Request request, ShardId shardId)
整體的執(zhí)行流程封裝在了內(nèi)部類:AsyncSingleAction, 在start方法中:
1. 通過各自子類的 shards 方法, 獲得要執(zhí)行的 shards, 不同的子類實(shí)現(xiàn)不同: 比如, 查某個(gè)邏輯shard的, 這里返回的可能是insync replica 組的所有副本的shard
this.shardIt = shards(clusterState, internalRequest);
...................
...................
2. 判斷是否有需要執(zhí)行的shards, 如果沒有的話, 說明可能是不需要shard就能執(zhí)行的操作
if (shardIt == null) {
executeLocally
}else{
perform // 遞歸操作各個(gè)shard
}
................
.................
3. 獲取shard的node, 發(fā)送內(nèi)部請(qǐng)請(qǐng)求
ShardRouting shardRouting = shardIt.nextOrNull();
if (shardRouting == null) {
listener.onFailure(failure);
return;
}
DiscoveryNode node = nodes.get(shardRouting.currentNodeId());
if (node == null) { // onFailure 會(huì)遞歸調(diào)用 perform , 每個(gè)shard候選都嘗試一遍
onFailure(shardRouting, new NoShardAvailableActionException(shardRouting.shardId()));
} else {
transportService.sendRequest(); 給shard所在的node 發(fā)送內(nèi)部執(zhí)行請(qǐng)求
}
TransportGetAction 分析舉例
重寫方法
1. 定位候選shards
OperationRouting#getShards 會(huì)依據(jù)自適應(yīng)副本策略, 選擇最合適的shards, 把它們放在shards最前面. 因?yàn)樗鼈兊年惞β首罡? 速度可能最快.
@Override
protected ShardIterator shards(ClusterState state, InternalRequest request) {
return clusterService.operationRouting() // operationRouting里面保存了關(guān)于自適應(yīng)選擇副本的設(shè)置
.getShards(
clusterService.state(),
request.concreteIndex(),
request.request().id(),
request.request().routing(),
request.request().preference()
);
}
2. 根據(jù)集群的一些狀態(tài)信息解析Get請(qǐng)求到內(nèi)部請(qǐng)求
3. 選擇線程池
protected String getExecutor(GetRequest request, ShardId shardId) {
final ClusterState clusterState = clusterService.state();
if (clusterState.metadata().getIndexSafe(shardId.getIndex()).isSystem()) {
return ThreadPool.Names.SYSTEM_READ;
} else if (indicesService.indexServiceSafe(shardId.getIndex()).getIndexSettings().isSearchThrottled()) {
return ThreadPool.Names.SEARCH_THROTTLED;
} else {
return super.getExecutor(request, shardId);
}
}
4. 重寫異步shard的執(zhí)行
根據(jù) realtime 參數(shù)的值, 可以看到有兩個(gè)執(zhí)行邏輯:
a. realtime : 直接調(diào)用父類的 shardOperation, 其實(shí)就是會(huì)執(zhí)行子類的shardOperation
b. realtime參數(shù)不為true : 會(huì)等待 shardSearchActive, 即如果有等待刷新落盤的的translog, 那等他刷完之后, 再執(zhí)行shardOperation
這里涉及到一些關(guān)于Get的實(shí)時(shí)性如何實(shí)現(xiàn)的細(xì)節(jié)(從translog讀, 或者refresh下), 在不同版本里Get操作的實(shí)時(shí)性實(shí)現(xiàn)是不同的, 并且有點(diǎn)版本, 還有問題, 比如update的時(shí)候的原子性問題, 具體見下一章節(jié)
asyncShardOperation
@Override
protected void asyncShardOperation(GetRequest request, ShardId shardId, ActionListener<GetResponse> listener) throws IOException {
IndexService indexService = indicesService.indexServiceSafe(shardId.getIndex());
IndexShard indexShard = indexService.getShard(shardId.id());
if (request.realtime()) { // we are not tied to a refresh cycle here anyway
super.asyncShardOperation(request, shardId, listener);
} else {
indexShard.awaitShardSearchActive(b -> {
try {
super.asyncShardOperation(request, shardId, listener);
} catch (Exception ex) {
listener.onFailure(ex);
}
});
}
}
shardOperation
根據(jù) if 語(yǔ)句里的邏輯可知: Get 操作只有在refresh=true&realtime=false的情況下, 會(huì)主動(dòng)觸發(fā)一次refresh
@Override
protected GetResponse shardOperation(GetRequest request, ShardId shardId) {
IndexService indexService = indicesService.indexServiceSafe(shardId.getIndex());
IndexShard indexShard = indexService.getShard(shardId.id());
if (request.refresh() && !request.realtime()) { // 如果設(shè)置了refresh, 先執(zhí)行下. 不影響 realtime
indexShard.refresh("refresh_flag_get");
}
GetResult result = indexShard.getService() // 調(diào)用
.get(
request.type(),
request.id(),
request.storedFields(),
request.realtime(),
request.version(),
request.versionType(),
request.fetchSourceContext()
);
return new GetResponse(result);
}
Get的realtime 表現(xiàn)和背后實(shí)現(xiàn)
可以看一些pr
https://github.com/elastic/elasticsearch/pull/20102
https://github.com/elastic/elasticsearch/pull/48843
https://github.com/elastic/elasticsearch/pull/64504
早期從translog讀, 后來從lucene讀(5.x開始), 或者是searcher讀, 現(xiàn)在7.x 從translog
參考: https://blog.csdn.net/qq_42848795/article/details/109711741
get的實(shí)時(shí)是不受refresh頻率影響的. 只要realtime為true ,一定可以獲取最新的. 哪怕還沒有refresh(并且如果refresh參數(shù)不設(shè)置為true, 不會(huì)主動(dòng)觸發(fā)refresh).
https://www.elastic.co/guide/en/elasticsearch/reference/7.10/docs-get.html
Realtime
By default, the get API is realtime, and is not affected by the refresh rate of the index (when data will become visible for search). In case where stored fields are requested (see stored_fields parameter) and the document has been updated but is not yet refreshed, the get API will have to parse and analyze the source to extract the stored fields. In order to disable realtime GET, the realtime parameter can be set to false.
調(diào)用鏈:
ShardService org.opensearch.index.get.ShardGetService#innerGet 得到org.opensearch.index.get.GetResult
IndexShard#get -> Engine#get 得到Engine.GetResult
可以看到, 在構(gòu)造 Engine.Get 的第二個(gè)參數(shù) readFromTranslog 如果 realtime為true ,則readFromTrasnlog 也為true. 盡管在InternalEngine#get 方法中, 既有從translog讀取, 也有從Saercher讀的邏輯(如果在versionmap中沒有這個(gè)doc, 則說明大概率是已經(jīng)寫入了, 可見了).
get = indexShard.get(
new Engine.Get(realtime, realtime, type, id, uidTerm).version(version)
.versionType(versionType)
.setIfSeqNo(ifSeqNo)
.setIfPrimaryTerm(ifPrimaryTerm)
);
