sharding-sphere之SQL解析insert

以mysql為例,官網說明insert語句的寫法如下:

INSERT [LOW_PRIORITY | DELAYED | HIGH_PRIORITY] [IGNORE]
    [INTO] tbl_name
    [PARTITION (partition_name [, partition_name] ...)]
    [(col_name [, col_name] ...)]
    {VALUES | VALUE} (value_list) [, (value_list)] ...
    [ON DUPLICATE KEY UPDATE assignment_list]

INSERT [LOW_PRIORITY | DELAYED | HIGH_PRIORITY] [IGNORE]
    [INTO] tbl_name
    [PARTITION (partition_name [, partition_name] ...)]
    SET assignment_list
    [ON DUPLICATE KEY UPDATE assignment_list]

INSERT [LOW_PRIORITY | HIGH_PRIORITY] [IGNORE]
    [INTO] tbl_name
    [PARTITION (partition_name [, partition_name] ...)]
    [(col_name [, col_name] ...)]
    SELECT ...
    [ON DUPLICATE KEY UPDATE assignment_list]

value:
    {expr | DEFAULT}

value_list:
    value [, value] ...

assignment:
    col_name = value

assignment_list:
    assignment [, assignment] ...

簡單來講,支持一下三種:

--第一種Insert into values
insert into table(column1,column2,column3...) values (data1,data2,data3...)
--第二種Insert into set
insert into table set column1=data1,column2=data2,column3=data3...
--第三種
insert into table select ...

目前來講,sharding-sphere并不支持第三種。原因在這里:

 //如果詞法解析器解析表名后面為select關鍵字,則拋異常。
 if (lexerEngine.equalAny(DefaultKeyword.SELECT, Symbol.LEFT_PAREN)) {
     throw new UnsupportedOperationException("Cannot INSERT SELECT");
 }

接下來,就以第一種為例,介紹一下整個的語法解析過程:
解析sql:

INSERT INTO `table` (`field1`, `field2`) VALUES (10, 1)

語法解析器會將sql最終解析成SqlStatement,sql的解析都會被Sql語法解析引擎SQLParsingEngine去解析,從SQL解析引擎看起,看這條insert sql都經歷了些什么故事?

@Test
public void insertValuesTest() {
    ShardingRule shardingRule = createShardingRuleByLocalHost();
    String insertSQL="INSERT INTO `table` (`field1`, `field2`) VALUES (10, 1)";
    //初始化一個語法解析器
    SQLParsingEngine statementParser = new SQLParsingEngine(DatabaseType.MySQL,insertSQL, shardingRule, null);
    //語法解析器解析
    InsertStatement insertStatement = (InsertStatement) statementParser.parse(false);
    System.out.println(insertStatement);
}

SQLParsingEngine解析過程中,首先判斷是否需要讀緩存,如果需要則從緩存中取,如果不需要,則初始化一個詞法解析器LexerEngine,獲取第一個分詞,根據分詞,數據庫類型獲取真實的SqlParser,這里為MySQLInsertParser,具體過程在鏈接:
詞法解析器

public SQLStatement parse(final boolean useCache) {
    //是否讀緩存
    Optional<SQLStatement> cachedSQLStatement = getSQLStatementFromCache(useCache);
    if (cachedSQLStatement.isPresent()) {
        return cachedSQLStatement.get();
    }
    //詞法解析引擎初始化
    LexerEngine lexerEngine = LexerEngineFactory.newInstance(dbType, sql);
    //分詞,獲取第一個分詞,此處為insert
    lexerEngine.nextToken(); //此處為insert分詞
    //獲取語法解析器,并解析
    SQLStatement result = SQLParserFactory.newInstance(dbType, lexerEngine.getCurrentToken().getType(), shardingRule, lexerEngine, shardingMetaData).parse();
    if (useCache) {
        ParsingResultCache.getInstance().put(sql, result);
    }
    return result;
}

MySQLInsertParser解析過程中,首先獲取insert之后的下一個分詞,然后交由下一個從句解析器去解析,比如:insert之后,會解析into表名,交由InsertIntoClauseParser去解析,然后解析(,)里的字段信息,交由InsertColumnsClauseParser去解析,然后判斷是否是insert...select語句,如果是,則拋異常,表示不支持,否則,當做insert..values語句處理,交由InsertValuesClauseParser處理,若不是insert..values語句,則交由InsertSetClauseParser,當做insert...set語句處理。

@Override
      public final DMLStatement parse() {
        //獲取下一分詞
        lexerEngine.nextToken();
        //初始化返回的結果
        InsertStatement result = new InsertStatement();
        //InsertIntoClauseParser insertInto從句解析器解析結果
        insertClauseParserFacade.getInsertIntoClauseParser().parse(result);
        //解析()及里面的字段信息
        insertClauseParserFacade.getInsertColumnsClauseParser().parse(result, shardingMetaData);
        //如果是insert select 語句,則不支持
        if (lexerEngine.equalAny(DefaultKeyword.SELECT, Symbol.LEFT_PAREN)) {
            throw new UnsupportedOperationException("Cannot INSERT SELECT");
        }
        //InsertValuesClauseParser解析insert value信息
        insertClauseParserFacade.getInsertValuesClauseParser().parse(result, shardingMetaData);
        //InsertSetClauseParser 解析insert...set信息
        insertClauseParserFacade.getInsertSetClauseParser().parse(result);
        //ON DUPLICATE KEY UPDATE 解析
        insertClauseParserFacade.getInsertDuplicateKeyUpdateClauseParser().parse(result);
        //處理需要自動生成值的列
        processGeneratedKey(result);
        return result;
    }

InsertIntoClauseParser的處理流程:

public void parse(final InsertStatement insertStatement) {
    //如果出現不支持的詞在into之前,則終止解析    
    lexerEngine.unsupportedIfEqual(getUnsupportedKeywordsBeforeInto());
    //跳過所有字符,直到出現into
    lexerEngine.skipUntil(DefaultKeyword.INTO);
    //獲取into后的下一分詞,當然這里就是表名了
    lexerEngine.nextToken();
    //解析表引用關系
    tableReferencesClauseParser.parse(insertStatement, true);
    skipBetweenTableAndValues(insertStatement);
}

而在解析表引用關系時,實質是使用MySQLTableReferencesClauseParser,mysql的table從句解析器解析,具體如下:

@Override
protected void parseTableReference(final SQLStatement sqlStatement, final boolean isSingleTableOnly) {
    //解析表
    parseTableFactor(sqlStatement, isSingleTableOnly);
    //PARTITION 解析
    parsePartition();
    parseIndexHint(sqlStatement);
}  

protected final void parseTableFactor(final SQLStatement sqlStatement, final boolean isSingleTableOnly) {
    final int beginPosition = lexerEngine.getCurrentToken().getEndPosition() - lexerEngine.getCurrentToken().getLiterals().length();
    String literals = lexerEngine.getCurrentToken().getLiterals();
    int skippedSchemaNameLength = 0;
    //獲取下一分詞
    lexerEngine.nextToken();
    //跳過如果是. 則在點之前的是schema的名稱,跳過.
    if (lexerEngine.skipIfEqual(Symbol.DOT)) {
         //跳過schema的長度為skippedSchemaNameLength
        skippedSchemaNameLength = literals.length() + Symbol.DOT.getLiterals().length();
        //表名為當前分詞
        literals = lexerEngine.getCurrentToken().getLiterals();
    }
    //獲取表名 如果為`tablename`,則返回tablename
    String tableName = SQLUtil.getExactlyValue(literals);
    if (Strings.isNullOrEmpty(tableName)) {
        return;
    }
    //解析別名 
    Optional<String> alias = aliasExpressionParser.parseTableAlias();
    //如果是單表,或者能根據邏輯實體表名獲取到表規則等其他條件
    if (isSingleTableOnly || shardingRule.tryFindTableRuleByLogicTable(tableName).isPresent() || shardingRule.findBindingTableRule(tableName).isPresent()
            || shardingRule.getShardingDataSourceNames().getDataSourceNames().contains(shardingRule.getShardingDataSourceNames().getDefaultDataSourceName())) {
        //表的信息記錄在返回值里,有數據庫的schema名稱,表名稱,開始位置坐標
        sqlStatement.getSqlTokens().add(new TableToken(beginPosition, skippedSchemaNameLength, literals));
        //表名和別名記錄下來
        sqlStatement.getTables().add(new Table(tableName, alias));
    }
    //解析是否強制索引,insert語句直接跳過,不會使用
    parseForceIndex(tableName, sqlStatement);
    //表關聯解析,直接跳過
    parseJoinTable(sqlStatement);
    //如果參數是單表,且解析結果不是單表,則直接拋異常
    if (isSingleTableOnly && !sqlStatement.getTables().isSingleTable()) {
        throw new UnsupportedOperationException("Cannot support Multiple-Table.");
    }
}

insert into table具體如何解析結束看完了,接下來,到解析表字段了,InsertColumnsClauseParser出場了。在InsertColumnsClauseParser處理邏輯中,從(開始,解析一個又一個的分詞,直到碰到)或者結束分詞。如果沒有(,則獲取全部字段。

public void parse(final InsertStatement insertStatement, final ShardingMetaData shardingMetaData) {
        Collection<Column> result = new LinkedList<>();
        //獲取表名
        String tableName = insertStatement.getTables().getSingleTableName();
        //獲取需要自動生成字段的值
        Optional<Column> generateKeyColumn = shardingRule.getGenerateKeyColumn(tableName);
        int count = 0;
        //如果是`(`
        if (lexerEngine.equalAny(Symbol.LEFT_PAREN)) {
            do {
                //獲取下一個分詞,就是獲取insert into table(field1,filed2...)中`(`后面的fileld1,或者是`,`后面的field2.
                lexerEngine.nextToken();
                //具體解析里面的字段,返回值name為具體屬性
                SQLExpression sqlExpression = basicExpressionParser.parse(insertStatement);
                //根據不同類型,獲取列屬性名稱
                String columnName = null;
                if (sqlExpression instanceof SQLPropertyExpression) {
                    columnName = SQLUtil.getExactlyValue(((SQLPropertyExpression) sqlExpression).getName());
                }
                if (sqlExpression instanceof SQLIdentifierExpression) {
                    columnName = SQLUtil.getExactlyValue(((SQLIdentifierExpression) sqlExpression).getName());
                }
                if (sqlExpression instanceof SQLIgnoreExpression) {
                    columnName = SQLUtil.getExactlyValue(((SQLIgnoreExpression) sqlExpression).getExpression());
                }
                //返回值添加列信息
                result.add(new Column(columnName, tableName));
                if (generateKeyColumn.isPresent() && generateKeyColumn.get().getName().equalsIgnoreCase(columnName)) {
                    //如果有需要自動生成列信息的字段,記錄字段位置
                    insertStatement.setGenerateKeyColumnIndex(count);
                }
                count++;
            } while (!lexerEngine.equalAny(Symbol.RIGHT_PAREN) && !lexerEngine.equalAny(Assist.END));
            insertStatement.setColumnsListLastPosition(lexerEngine.getCurrentToken().getEndPosition() - lexerEngine.getCurrentToken().getLiterals().length());
            lexerEngine.nextToken();
        } else {
          //忽略若干代碼
        }
        insertStatement.getColumns().addAll(result);
}

接下來要解析values后面的值了,慣例是InsertValuesClauseParser,該從句首先判斷是否是values分詞的語句,如果是,則解析,如果不是則直接跳過,等待后面insert ...set處理。

public void parse(final InsertStatement insertStatement, final ShardingMetaData shardingMetaData) {
    Collection<Keyword> valueKeywords = new LinkedList<>();
    valueKeywords.add(DefaultKeyword.VALUES);
    valueKeywords.addAll(Arrays.asList(getSynonymousKeywordsForValues()));
    //是否是value,或者是values的關鍵字,如果是則解析,如果不是,則跳過
    if (lexerEngine.skipIfEqual(valueKeywords.toArray(new Keyword[valueKeywords.size()]))) {
        //解析value值
        parseValues(insertStatement);
    }
}

private void parseValues(final InsertStatement insertStatement) {
    int beginPosition = lexerEngine.getCurrentToken().getEndPosition() - lexerEngine.getCurrentToken().getLiterals().length();
    int endPosition;
    insertStatement.getSqlTokens().add(new InsertValuesToken(beginPosition, insertStatement.getTables().getSingleTableName()));
    do {
        beginPosition = lexerEngine.getCurrentToken().getEndPosition() - lexerEngine.getCurrentToken().getLiterals().length();
        //分詞器解析必須是`(`開頭
        lexerEngine.accept(Symbol.LEFT_PAREN);
        List<SQLExpression> sqlExpressions = new LinkedList<>();
        int columnsCount = 0;
        do {
            //解析每一個分詞sql后面的具體值,對應sql中的INSERT INTO `table` (`field1`, `field2`) VALUES (10, 1)的10和1
            sqlExpressions.add(basicExpressionParser.parse(insertStatement));
            //跳過::
            skipsDoubleColon();
            //列個數+1
            columnsCount++;
            //當解析器遇到的分詞為,號時,繼續解析
        } while (lexerEngine.skipIfEqual(Symbol.COMMA));
        //刪除自動生成的鍵
        removeGenerateKeyColumn(insertStatement, columnsCount);
        columnsCount = 0;
        int parametersCount = 0;
        AndCondition andCondition = new AndCondition();
        for (Column each : insertStatement.getColumns()) {
            SQLExpression sqlExpression = sqlExpressions.get(columnsCount);
            //如果是分片項,則添加Condition
            if (shardingRule.isShardingColumn(each)) {
                andCondition.getConditions().add(new Condition(each, sqlExpression));
            }
            //如果是自動生成的列
            if (insertStatement.getGenerateKeyColumnIndex() == columnsCount) {
                //生成Condition
                insertStatement.getGeneratedKeyConditions().add(createGeneratedKeyCondition(each, sqlExpression));
            }
            //列個數+1
            columnsCount++;
            if (sqlExpression instanceof SQLPlaceholderExpression) {
                parametersCount++;
            }
        }
        endPosition = lexerEngine.getCurrentToken().getEndPosition();
        //分詞解析器必須以)結束
        lexerEngine.accept(Symbol.RIGHT_PAREN);
        //組裝數據
        insertStatement.getInsertValues().getInsertValues().add(new InsertValue(DefaultKeyword.VALUES, lexerEngine.getInput().substring(beginPosition, endPosition), parametersCount));
        insertStatement.getConditions().getOrCondition().getAndConditions().add(andCondition);
    } while (lexerEngine.skipIfEqual(Symbol.COMMA));
    insertStatement.setInsertValuesListLastPosition(endPosition);
}

如果是insert...set語句,則會走到InsertSetClauseParser,去解析set語句,如果發現不是set開頭的語句,直接return。

public void parse(final InsertStatement insertStatement) {
    //判斷是否是set,如不是,直接返回
    if (!lexerEngine.skipIfEqual(getCustomizedInsertKeywords())) {
        return;
    }
    //刪除無用的Token
    removeUnnecessaryToken(insertStatement);
    int beginPosition = lexerEngine.getCurrentToken().getEndPosition() - lexerEngine.getCurrentToken().getLiterals().length();
    insertStatement.getSqlTokens().add(new InsertValuesToken(beginPosition, insertStatement.getTables().getSingleTableName()));
    int parametersCount = 0;
    do {
        //解析set之后的數據庫列名稱
        SQLExpression sqlExpression = basicExpressionParser.parse(insertStatement);
        Column column = null;
        if (sqlExpression instanceof SQLPropertyExpression) {
            column = new Column(SQLUtil.getExactlyValue(((SQLPropertyExpression) sqlExpression).getName()), insertStatement.getTables().getSingleTableName());
        }
        if (sqlExpression instanceof SQLIdentifierExpression) {
            column = new Column(SQLUtil.getExactlyValue(((SQLIdentifierExpression) sqlExpression).getName()), insertStatement.getTables().getSingleTableName());
        }
        if (sqlExpression instanceof SQLIgnoreExpression) {
            column = new Column(SQLUtil.getExactlyValue(((SQLIgnoreExpression) sqlExpression).getExpression()), insertStatement.getTables().getSingleTableName());
        }
        //列屬性后面必須緊跟=號
        lexerEngine.accept(Symbol.EQ);
        //獲取=號下一分詞類型,根據類型組裝
        if (lexerEngine.equalAny(Literals.INT)) {
            sqlExpression = new SQLNumberExpression(Integer.parseInt(lexerEngine.getCurrentToken().getLiterals()));
        } else if (lexerEngine.equalAny(Literals.FLOAT)) {
            sqlExpression = new SQLNumberExpression(Double.parseDouble(lexerEngine.getCurrentToken().getLiterals()));
        } else if (lexerEngine.equalAny(Literals.CHARS)) {
            sqlExpression = new SQLTextExpression(lexerEngine.getCurrentToken().getLiterals());
        } else if (lexerEngine.equalAny(DefaultKeyword.NULL)) {
            sqlExpression = new SQLIgnoreExpression(DefaultKeyword.NULL.name());
        } else if (lexerEngine.equalAny(Symbol.QUESTION)) {
            sqlExpression = new SQLPlaceholderExpression(insertStatement.getParametersIndex());
            insertStatement.increaseParametersIndex();
            parametersCount++;
        } else {
            throw new UnsupportedOperationException("");
        }
        //獲取下一分詞
        lexerEngine.nextToken();
        if (lexerEngine.equalAny(Symbol.COMMA, DefaultKeyword.ON, Assist.END)) {
            //組裝Condition
            insertStatement.getConditions().add(new Condition(column, sqlExpression), shardingRule);
        } else {
            //跳過,直到出現,on
            lexerEngine.skipUntil(Symbol.COMMA, DefaultKeyword.ON);
        }
    //如果是,結尾,繼續循環,解析其他的字段
    } while (lexerEngine.skipIfEqual(Symbol.COMMA));
    //組裝返回數據
    int endPosition = lexerEngine.getCurrentToken().getEndPosition() - lexerEngine.getCurrentToken().getLiterals().length();
    insertStatement.getInsertValues().getInsertValues().add(new InsertValue(DefaultKeyword.VALUES, lexerEngine.getInput().substring(beginPosition, endPosition), parametersCount));
    insertStatement.setInsertValuesListLastPosition(endPosition);
}

insert語句解析過程結束,解析過程復雜,能用如此簡潔的代碼寫完,代碼功底深厚令人折服。大寫的服。但其實還是有一些能優化的點。
最后,再看一下InsertStatement的信息,解析之后,該對象組裝了些啥。

InsertStatement.png

對應sql

INSERT INTO `table` (`field1`, `field2`) VALUES (10, 1)
public final class InsertStatement extends DMLStatement {
    //列屬性信息
    private final Collection<Column> columns = new LinkedList<>();
    //自動生成的表字段信息
    private List<GeneratedKeyCondition> generatedKeyConditions = new LinkedList<>();
    //插入的值
    private final InsertValues insertValues = new InsertValues();
    //數據庫字段屬性最后結束的位置
    private int columnsListLastPosition;
    //自動生成鍵的位置
    private int generateKeyColumnIndex = -1;
    //insert values最后結束的位置
    private int insertValuesListLastPosition;
}

public abstract class AbstractSQLStatement implements SQLStatement {
    //sql類型
    private final SQLType type;
    //表信息
    private final Tables tables = new Tables();
    //分片信息
    private final Conditions conditions = new Conditions();
    //sql分詞信息
    private final List<SQLToken> sqlTokens = new LinkedList<>();
    
    private int parametersIndex;

}
InsertStatement.png
?著作權歸作者所有,轉載或內容合作請聯系作者
平臺聲明:文章內容(如有圖片或視頻亦包括在內)由作者上傳并發布,文章內容僅代表作者本人觀點,簡書系信息發布平臺,僅提供信息存儲服務。

推薦閱讀更多精彩內容