改寫引擎的職責定位是進行SQL的修改，因為ShardingSphere的核心目標就是屏蔽分庫分表對用戶的影響（當然后來還增加影子表、加解密等功能），使開發(fā)者可以按照像原來傳統(tǒng)單庫單表一樣編寫SQL。表拆分后，表名往往會帶有編號或者日期等標識，但應用中的SQL中表名并不會帶有這些標識，一般稱之為邏輯表（和未拆分前表名完全相同），因此改寫引擎需要用路由引擎計算得到的真正物理表名替換SQL中的邏輯表名，這樣SQL才能正確執(zhí)行。
除了sharding功能中表名替換，目前在ShardingSphere中需要很多種情況會進行SQL改寫，具體有：
- 數(shù)據(jù)分片功能中表名改寫；
- 數(shù)據(jù)分片功能中聚合函數(shù)distinct；
- 數(shù)據(jù)分片功能中avg聚合函數(shù)需添加count、sum；
- 數(shù)據(jù)分片功能中索引重命名；
- 數(shù)據(jù)分片功能中分頁時offset、rowcount改寫；
- 配置分布式自增鍵時自增列、值添加；
- 加解密功能下對列、值得添加修改；
- 影子表功能下對列與值的修改。
代碼調(diào)用分析
回到BasePrepareEngine類中，可以看到在使用改寫功能前注冊了改寫裝飾器。
org.apache.shardingsphere.underlying.pluggble.prepare.BasePrepareEngine
private Collection<ExecutionUnit> executeRewrite(final String sql, final List<Object> parameters, final RouteContext routeContext) {
registerRewriteDecorator();
//創(chuàng)建SQL改寫上下文，主要是生成對應的Token以及參數(shù)
SQLRewriteContext sqlRewriteContext = rewriter.createSQLRewriteContext(sql, parameters, routeContext.getSqlStatementContext(), routeContext);
return routeContext.getRouteResult().getRouteUnits().isEmpty() ? rewrite(sqlRewriteContext) : rewrite(routeContext, sqlRewriteContext);
}
private void registerRewriteDecorator() {
for (Class<? extends SQLRewriteContextDecorator> each : OrderedRegistry.getRegisteredClasses(SQLRewriteContextDecorator.class)) {
SQLRewriteContextDecorator rewriteContextDecorator = createRewriteDecorator(each);
Class<?> ruleClass = (Class<?>) rewriteContextDecorator.getType();
// FIXME rule.getClass().getSuperclass() == ruleClass for orchestration, should decouple extend between orchestration rule and sharding rule
rules.stream().filter(rule -> rule.getClass() == ruleClass || rule.getClass().getSuperclass() == ruleClass).collect(Collectors.toList())
.forEach(rule -> rewriter.registerDecorator(rule, rewriteContextDecorator));
}
}
之后便通過SQL改寫入口類SQLRewriteEntry創(chuàng)建SQL改寫上下文對象
/**
* SQL rewrite entry.
*/
@RequiredArgsConstructor
public final class SQLRewriteEntry {
private final SchemaMetaData schemaMetaData;
private final ConfigurationProperties properties;
private final Map<BaseRule, SQLRewriteContextDecorator> decorators = new LinkedHashMap<>();
/**
* Register route decorator.
*
* @param rule rule
* @param decorator SQL rewrite context decorator
*/
public void registerDecorator(final BaseRule rule, final SQLRewriteContextDecorator decorator) {
decorators.put(rule, decorator);
}
/**
* Create SQL rewrite context.
*
* @param sql SQL
* @param parameters parameters
* @param sqlStatementContext SQL statement context
* @param routeContext route context
* @return SQL rewrite context
*/
public SQLRewriteContext createSQLRewriteContext(final String sql, final List<Object> parameters, final SQLStatementContext sqlStatementContext, final RouteContext routeContext) {
SQLRewriteContext result = new SQLRewriteContext(schemaMetaData, sqlStatementContext, sql, parameters);// 創(chuàng)建一個初始SQL改寫上下文
decorate(decorators, result, routeContext);// 進行裝飾器處理，其實就是根據(jù)Statement上下文，生成一系列的Token生成器
result.generateSQLTokens();// 運行各Token生成器，解構出對應的Token
return result;
}
@SuppressWarnings("unchecked")
private void decorate(final Map<BaseRule, SQLRewriteContextDecorator> decorators, final SQLRewriteContext sqlRewriteContext, final RouteContext routeContext) {
for (Entry<BaseRule, SQLRewriteContextDecorator> entry : decorators.entrySet()) {
BaseRule rule = entry.getKey();
SQLRewriteContextDecorator decorator = entry.getValue();
if (decorator instanceof RouteContextAware) {
((RouteContextAware) decorator).setRouteContext(routeContext);
}
decorator.decorate(rule, properties, sqlRewriteContext);
}
}
}
以最用到的sharding功能實現(xiàn)的SQL改寫上下文裝飾器，看下其實現(xiàn)邏輯：
org.apache.shardingsphere.sharding.rewrite.context.ShardingSQLRewriteContextDecorator
/**
* SQL rewrite context decorator for sharding.
*/
@Setter
public final class ShardingSQLRewriteContextDecorator implements SQLRewriteContextDecorator<ShardingRule>, RouteContextAware {
private RouteContext routeContext;
@SuppressWarnings("unchecked")
@Override
public void decorate(final ShardingRule shardingRule, final ConfigurationProperties properties, final SQLRewriteContext sqlRewriteContext) {
// 獲取參數(shù)改寫器（參數(shù)化SQL才需要），然后依次對SQL改寫上下文中的參數(shù)構造器parameterBuilder進行改寫操作，分片功能下主要是自增鍵以及分頁參數(shù)
for (ParameterRewriter each : new ShardingParameterRewriterBuilder(shardingRule, routeContext).getParameterRewriters(sqlRewriteContext.getSchemaMetaData())) {
if (!sqlRewriteContext.getParameters().isEmpty() && each.isNeedRewrite(sqlRewriteContext.getSqlStatementContext())) {
each.rewrite(sqlRewriteContext.getParameterBuilder(), sqlRewriteContext.getSqlStatementContext(), sqlRewriteContext.getParameters());
}
}
//添加分片功能下對應的Token生成器
sqlRewriteContext.addSQLTokenGenerators(new ShardingTokenGenerateBuilder(shardingRule, routeContext).getSQLTokenGenerators());
}
…
}
可以看到首先會通過ShardingParameterRewriterBuilder創(chuàng)建了數(shù)據(jù)分片功能對應的參數(shù)改寫器，包括了insert自增分布式主鍵參數(shù)和分頁參數(shù)兩個重寫器。
org.apache.shardingsphere.sharding.rewrite.parameter.ShardingParameterRewriterBuilder
/**
* Parameter rewriter builder for sharding.
*/
@RequiredArgsConstructor
public final class ShardingParameterRewriterBuilder implements ParameterRewriterBuilder {
private final ShardingRule shardingRule;
private final RouteContext routeContext;
@Override
public Collection<ParameterRewriter> getParameterRewriters(final SchemaMetaData schemaMetaData) {
Collection<ParameterRewriter> result = getParameterRewriters();
for (ParameterRewriter each : result) {
setUpParameterRewriters(each, schemaMetaData);
}
return result;
}
private static Collection<ParameterRewriter> getParameterRewriters() {
Collection<ParameterRewriter> result = new LinkedList<>();
result.add(new ShardingGeneratedKeyInsertValueParameterRewriter());
result.add(new ShardingPaginationParameterRewriter());
return result;
}
…
}
然后通過ShardingTokenGenerateBuilder生成數(shù)據(jù)分片Token生成器
org.apache.shardingsphere.sharding.rewrite.token.pojo.ShardingTokenGenerateBuilder
/**
* SQL token generator builder for sharding.
*/
@RequiredArgsConstructor
public final class ShardingTokenGenerateBuilder implements SQLTokenGeneratorBuilder {
private final ShardingRule shardingRule;
private final RouteContext routeContext;
@Override
public Collection<SQLTokenGenerator> getSQLTokenGenerators() {
Collection<SQLTokenGenerator> result = buildSQLTokenGenerators();
for (SQLTokenGenerator each : result) {
if (each instanceof ShardingRuleAware) {
((ShardingRuleAware) each).setShardingRule(shardingRule);
}
if (each instanceof RouteContextAware) {
((RouteContextAware) each).setRouteContext(routeContext);
}
}
return result;
}
private Collection<SQLTokenGenerator> buildSQLTokenGenerators() {
Collection<SQLTokenGenerator> result = new LinkedList<>();
addSQLTokenGenerator(result, new TableTokenGenerator());// 表名token處理，用于真實表名替換
addSQLTokenGenerator(result, new DistinctProjectionPrefixTokenGenerator());// select distinct關鍵字處理
addSQLTokenGenerator(result, new ProjectionsTokenGenerator());// select列名處理，主要是衍生列avg處理
addSQLTokenGenerator(result, new OrderByTokenGenerator());// Order by Token處理
addSQLTokenGenerator(result, new AggregationDistinctTokenGenerator());// 聚合函數(shù)的distinct關鍵字處理
addSQLTokenGenerator(result, new IndexTokenGenerator());// 索引重命名
addSQLTokenGenerator(result, new OffsetTokenGenerator());// offset 改寫
addSQLTokenGenerator(result, new RowCountTokenGenerator());// rowCount改寫
addSQLTokenGenerator(result, new GeneratedKeyInsertColumnTokenGenerator());// 分布式主鍵列添加，在insert sql列最后添加
addSQLTokenGenerator(result, new GeneratedKeyForUseDefaultInsertColumnsTokenGenerator());// insert SQL使用默認列名時需要完成補齊真實列名，包括自增列
addSQLTokenGenerator(result, new GeneratedKeyAssignmentTokenGenerator());// SET自增鍵生成
addSQLTokenGenerator(result, new ShardingInsertValuesTokenGenerator());// insert SQL 的values Token解析，為后續(xù)添加自增值做準備
addSQLTokenGenerator(result, new GeneratedKeyInsertValuesTokenGenerator());//為insert values添加自增列值
return result;
}
private void addSQLTokenGenerator(final Collection<SQLTokenGenerator> sqlTokenGenerators, final SQLTokenGenerator toBeAddedSQLTokenGenerator) {
if (toBeAddedSQLTokenGenerator instanceof IgnoreForSingleRoute && routeContext.getRouteResult().isSingleRouting()) {
return;
}
sqlTokenGenerators.add(toBeAddedSQLTokenGenerator);
}
}
可以看到ShardingTokenGenerateBuilder類針對數(shù)據(jù)分片需要改寫SQL的各種情況分別添加了對應的Token生成器，看下最主要的表名Token生成器類TableTokenGenerator
org.apache.shardingsphere.sharding.rewrite.token.generator.impl.TableTokenGenerator
/**
* Table token generator.
*/
@Setter
public final class TableTokenGenerator implements CollectionSQLTokenGenerator, ShardingRuleAware {
private ShardingRule shardingRule;
@Override
public boolean isGenerateSQLToken(final SQLStatementContext sqlStatementContext) {
return true;
}
@Override
public Collection<TableToken> generateSQLTokens(final SQLStatementContext sqlStatementContext) {
return sqlStatementContext instanceof TableAvailable ? generateSQLTokens((TableAvailable) sqlStatementContext) : Collections.emptyList();
}
private Collection<TableToken> generateSQLTokens(final TableAvailable sqlStatementContext) {
Collection<TableToken> result = new LinkedList<>();
for (SimpleTableSegment each : sqlStatementContext.getAllTables()) {
if (shardingRule.findTableRule(each.getTableName().getIdentifier().getValue()).isPresent()) {// 分片功能下，添加TableToken
result.add(new TableToken(each.getStartIndex(), each.getStopIndex(), each.getTableName().getIdentifier(), (SQLStatementContext) sqlStatementContext, shardingRule));
}
}
return result;
}
}
可以看到generateSQLTokens方法中，在判斷時數(shù)據(jù)分片規(guī)則中配置的表后，創(chuàng)建TableToken對象添加到集合中返回。
org.apache.shardingsphere.sharding.rewrite.token.pojo.TableToken
/**
* Table token.
*/
public final class TableToken extends SQLToken implements Substitutable, RouteUnitAware {
…
@Override
public String toString(final RouteUnit routeUnit) {
String actualTableName = getLogicAndActualTables(routeUnit).get(identifier.getValue().toLowerCase());
actualTableName = null == actualTableName ? identifier.getValue().toLowerCase() : actualTableName;
//替換成真實物理表名
return Joiner.on("").join(identifier.getQuoteCharacter().getStartDelimiter(), actualTableName, identifier.getQuoteCharacter().getEndDelimiter());
}
…
}
TableToken的toString方法即根據(jù)RouteUnit對象生成該Token在SQL改寫時需要替換成的字符串。
再次回到BasePrepareEngine類
org.apache.shardingsphere.underlying.pluggble.prepare.BasePrepareEngine# executeRewrite方法，在創(chuàng)建完SQL改寫上下文后，調(diào)用了rewrite方法生成執(zhí)行單元集合
private Collection<ExecutionUnit> executeRewrite(final String sql, final List<Object> parameters, final RouteContext routeContext) {
registerRewriteDecorator();
//創(chuàng)建SQL重寫上下文，主要是生成對應的Token以及參數(shù)
SQLRewriteContext sqlRewriteContext = rewriter.createSQLRewriteContext(sql, parameters, routeContext.getSqlStatementContext(), routeContext);
return routeContext.getRouteResult().getRouteUnits().isEmpty() ? rewrite(sqlRewriteContext) : rewrite(routeContext, sqlRewriteContext);
}
可以看到根據(jù)路由單元數(shù)量，分別對應兩個rewrite私有方法
// 此方法負責將SQL改寫上下文轉化為執(zhí)行單元ExecutionUnit集合
private Collection<ExecutionUnit> rewrite(final SQLRewriteContext sqlRewriteContext) {
SQLRewriteResult sqlRewriteResult = new SQLRewriteEngine().rewrite(sqlRewriteContext);// 將SQL改寫上下文轉化為SQL改寫結果，主要是獲取改寫后的SQL與參數(shù)
String dataSourceName = metaData.getDataSources().getAllInstanceDataSourceNames().iterator().next();
return Collections.singletonList(new ExecutionUnit(dataSourceName, new SQLUnit(sqlRewriteResult.getSql(), sqlRewriteResult.getParameters())));
}
private Collection<ExecutionUnit> rewrite(final RouteContext routeContext, final SQLRewriteContext sqlRewriteContext) {
Collection<ExecutionUnit> result = new LinkedHashSet<>();
for (Entry<RouteUnit, SQLRewriteResult> entry : new SQLRouteRewriteEngine().rewrite(sqlRewriteContext, routeContext.getRouteResult()).entrySet()) {
result.add(new ExecutionUnit(entry.getKey().getDataSourceMapper().getActualName(), new SQLUnit(entry.getValue().getSql(), entry.getValue().getParameters())));
}
return result;
}
第一個rewrite方法中，創(chuàng)建SQLRewriteEngine實例，然后執(zhí)行其rewrite方法生成SQLRewriteResult對象，通過此對象獲取改寫后的SQL以及參數(shù)，之后創(chuàng)建SQLUnit對象、創(chuàng)建ExecutionUnit對象，添加到集合中返回。接下來看下SQLRewriteEngine類：
org.apache.shardingsphere.underlying.rewrite.engine.SQLRewriteEngine
/**
* SQL rewrite engine.
*/
public final class SQLRewriteEngine {
/**
* Rewrite SQL and parameters.
*
* @param sqlRewriteContext SQL rewrite context
* @return SQL rewrite result
*/
public SQLRewriteResult rewrite(final SQLRewriteContext sqlRewriteContext) {
// 將SQL改寫上下文中Token轉化成SQL， 獲取改寫后參數(shù)，然后構造成SQL改寫結果返回
return new SQLRewriteResult(new DefaultSQLBuilder(sqlRewriteContext).toSQL(), sqlRewriteContext.getParameterBuilder().getParameters());
}
}
可以看到該類通過DefaultSQLBuilder類對象的toSQL()生成了改寫后SQL，然后創(chuàng)建了SQLRewriteResult實例，那么接下來繼續(xù)看下DefaultSQLBuilder類
org.apache.shardingsphere.underlying.rewrite.sql.impl.DefaultSQLBuilder
/**
* Default SQL builder.
*/
public final class DefaultSQLBuilder extends AbstractSQLBuilder {
public DefaultSQLBuilder(final SQLRewriteContext context) {
super(context);
}
@Override
protected String getSQLTokenText(final SQLToken sqlToken) {
return sqlToken.toString();// 返回Token對應的文本字符
}
}
DefaultSQLBuilder類只是實現(xiàn)了getSQLTokenText方法，調(diào)用Token.toString方法返回。
看下其父類AbstractSQLBuilder
org.apache.shardingsphere.underlying.rewrite.sql.impl.AbstractSQLBuilder
public abstract class AbstractSQLBuilder implements SQLBuilder {
private final SQLRewriteContext context;
@Override
public final String toSQL() {
if (context.getSqlTokens().isEmpty()) {
return context.getSql();
}
Collections.sort(context.getSqlTokens());// 按照Token的起始位置排序
StringBuilder result = new StringBuilder();
result.append(context.getSql().substring(0, context.getSqlTokens().get(0).getStartIndex()));// 添加第一個Token之前的原始SQL
for (SQLToken each : context.getSqlTokens()) {
result.append(getSQLTokenText(each));// 添加Token對應的SQL片段
result.append(getConjunctionText(each));// 添加Token之間的連接字符
}
return result.toString();
}
…
}
可以看到在AbstractSQLBuilder的toSQL方法中，對Token進行排序，然后通過拼接原始SQL和替換的Token以及連接符，最后形成完整的改寫SQL。
回頭再看下BasePrepareEngine類中的第二個rewrite方法
private Collection<ExecutionUnit> rewrite(final RouteContext routeContext, final SQLRewriteContext sqlRewriteContext) {
Collection<ExecutionUnit> result = new LinkedHashSet<>();
for (Entry<RouteUnit, SQLRewriteResult> entry : new SQLRouteRewriteEngine().rewrite(sqlRewriteContext, routeContext.getRouteResult()).entrySet()) {
result.add(new ExecutionUnit(entry.getKey().getDataSourceMapper().getActualName(), new SQLUnit(entry.getValue().getSql(), entry.getValue().getParameters())));
}
return result;
}
可以看到通過執(zhí)行了SQLRouteRewriteEngine對象的rewrite方法返回了一個Map<RouteUnit, SQLRewriteResult>對象，然后遍歷構建了ExecutionUnit，然后添加到集合中進行返回。那么我們看下SQLRouteRewriteEngine類：
org.apache.shardingsphere.underlying.rewrite.engine.SQLRouteRewriteEngine
/**
* SQL rewrite engine with route.
*/
public final class SQLRouteRewriteEngine {
/**
* Rewrite SQL and parameters.
*
* @param sqlRewriteContext SQL rewrite context
* @param routeResult route result
* @return SQL map of route unit and rewrite result
*/
public Map<RouteUnit, SQLRewriteResult> rewrite(final SQLRewriteContext sqlRewriteContext, final RouteResult routeResult) {
Map<RouteUnit, SQLRewriteResult> result = new LinkedHashMap<>(routeResult.getRouteUnits().size(), 1);
for (RouteUnit each : routeResult.getRouteUnits()) {
result.put(each, new SQLRewriteResult(new RouteSQLBuilder(sqlRewriteContext, each).toSQL(), getParameters(sqlRewriteContext.getParameterBuilder(), routeResult, each)));
}
return result;
}
…
}
可以看到，與SQLRewriteEngine類相似，只不過改為遍歷路由結果中包含的RouteUnit，然后分別創(chuàng)建對應的SQLRewriteResult實例，這里構建改寫SQL的類用了一個RouteSQLBuilder類
org.apache.shardingsphere.underlying.rewrite.sql.impl.RouteSQLBuilder
/**
* SQL builder with route.
*/
public final class RouteSQLBuilder extends AbstractSQLBuilder {
private final RouteUnit routeUnit;
public RouteSQLBuilder(final SQLRewriteContext context, final RouteUnit routeUnit) {
super(context);
this.routeUnit = routeUnit;
}
@Override
protected String getSQLTokenText(final SQLToken sqlToken) {
if (sqlToken instanceof RouteUnitAware) {
return ((RouteUnitAware) sqlToken).toString(routeUnit);
}
return sqlToken.toString();
}
}
…
}
這個類與DefaultSQLBuilder類似，也繼承自AbstractSQLBuilder類，只不過getSQLTokenText方法會判斷是否是RouteUnitAware類型的Token，如果是則調(diào)用RouteUnit參數(shù)的toSQL方法生成SQL。
總結
最后總結下改寫引擎的執(zhí)行流程：
改寫引擎的輸入即為路由上下文RouteContext，輸出為SQL改寫上下文SQLRewriteResult。
- 改寫功能入口SQLRewriteEntry類中創(chuàng)建一個初始化SQLRewriteResult對象。
- 順序執(zhí)行已注冊的SQL改寫上下文裝飾器，添加對應的一系列Token生成器對象。
- 調(diào)用SQLRewriteContext. generateSQLTokens方法，運行各Token生成器，解析出對應的Token。
- 由SQL改寫引擎SQLRewriteEngine或者SQLRouteRewriteEngine類調(diào)用DefaultSQLBuilder或RouteSQLBuilder類將解析出的Token拼接成改寫后的SQL，基于此SQL和參數(shù)，創(chuàng)建SQLRewriteResult實例返回。
- 基于SQLRewriteResult，創(chuàng)建ExecutionUnit，然后再封裝成ExecutionContext。
改寫引擎執(zhí)行流程
