JSON的語(yǔ)法簡(jiǎn)單，很適合上完編譯原理課程之后或者學(xué)習(xí)完龍書(shū)語(yǔ)法分析之后的練手。
**JSON**
JSON格式
{"a" : 123 }
[1,2,3]
{"a": [1,2,3] }
{"a" : {"b" : 123 }}
基礎(chǔ)為以上幾種。有了基本的格式，可以考慮先寫(xiě)詞法分析部分。JSON格式類(lèi)似于key/value存儲(chǔ)，其中key為string類(lèi)型，value復(fù)雜一些，可包括，string，int，double，jsonobject，jsonarray，true，false，null標(biāo)記。所以對(duì)每個(gè)token都要逐一分析，構(gòu)造出DFA。
- 字符串
字符串以 " 開(kāi)頭，以 " 結(jié)束。其中需要考慮的是轉(zhuǎn)義字符的忽略以及unicode的格式（\u hex hex hex hex）
private Token readString() throws IOException, JsonLexException {
StringBuilder builder = new StringBuilder();
while ( (this.at_ = read()) != '\"') {
if (this.at_ == '\r' || this.at_ == '\n') error("string", this.at_);
if (this.at_ == '\\') {
//check escape char
this.at_ = read();
if (isEscapeChar(this.at_)) {
builder.append('\\');
builder.append((char) this.at_);
} else if (this.at_ == 'u'){
//unicode
builder.append('\\');
builder.append('u');
builder.append(unicode());
} else {
error("string", this.at_);
}
} else {
builder.append((char) this.at_);
}
}
if ( this.at_ != '\"') {
//string is not closed
error("string[not closed]", this.at_);
}
return this.token_ = new Token(JsonToken.STRING, new Value(builder.toString()));
}
private String unicode() throws IOException, JsonLexException {
StringBuilder builder = new StringBuilder();
int i=0;
for (;i<4 && isHexChar(this.at_ = read()); builder.append((char) this.at_),++i);
if (i < 4) error("unicode", this.at_);
return builder.toString();
}
- 數(shù)字
數(shù)字包含有符號(hào)int double以及科學(xué)計(jì)數(shù)法表示稍微麻煩，但是畫(huà)出DFA分析就會(huì)明了很多。
DFA
據(jù)此就可很快寫(xiě)出來(lái)。
private Token readNumber(int at) throws IOException,JsonLexException {
StringBuilder builder = new StringBuilder();
int status = 0;
while (true) {
switch (status) {
case 0:
this.at_ = at;
if (this.at_ == '+' || this.at_ == '-') status = 1;
else if (Character.isDigit(this.at_)) status = 2;
else error("number", this.at_);
builder.append((char) this.at_);
break;
case 1:
this.at_ = read();
if (Character.isDigit(this.at_)) status = 2;
else error("number", this.at_);
builder.append((char) this.at_);
break;
case 2:
this.at_ = read();
if (this.at_ == '.') status = 3;
else if (Character.isDigit(this.at_)) status = 2;
else if (this.at_ == 'E' || this.at_ == 'e') status = 5;
else status = 8;
if (status != 8) {
builder.append((char) this.at_);
}
break;
case 3:
this.at_ = read();
if (Character.isDigit(this.at_)) status = 4;
else error("number", this.at_);
builder.append((char) this.at_);
break;
case 4:
this.at_ = read();
if (Character.isDigit(this.at_)) status = 4;
else if (this.at_ == 'E' || this.at_ == 'e') status = 5;
else status = 9;
if (status != 9) {
builder.append((char) this.at_);
}
break;
case 5:
this.at_ = read();
if (this.at_ == '+' || this.at_ == '-') status = 6;
else if (Character.isDigit(this.at_)) status = 7;
else error("number", this.at_);
builder.append((char) this.at_);
break;
case 6:
this.at_ = read();
if (Character.isDigit(this.at_)) status = 7;
else error("number", this.at_);
builder.append((char) this.at_);
break;
case 7:
this.at_ = read();
if (Character.isDigit(this.at_)) status = 7;
else status = 10;
if (status != 10) {
builder.append((char) this.at_);
}
break;
case 8: // int
unread(this.at_); // not a digit
return this.token_ = new Token(JsonToken.INT, new Value(Integer.valueOf(builder.toString())));
case 9: //double without 'E' or 'e'
unread(this.at_); // not a digit
return this.token_ = new Token(JsonToken.DOUBLE, new Value(Double.valueOf(builder.toString())));
case 10://double with 'E' or 'e'
unread(this.at_);// not a digit
return this.token_ = new Token(JsonToken.DOUBLE,new Value(Double.valueOf(builder.toString())));
default:
error("number", this.at_);
}
}
}
- 其他字符
對(duì)于其他終結(jié)符，true false null則只要匹配首字符來(lái)判斷就可以了。
綜上就可寫(xiě)出來(lái)lexer了
public Token scan() throws IOException, JsonLexException {
//this.at_ = '\0';
while (isWhiteBlack( this.at_ = read() ));
switch ((int)this.at_) {
case '{':
return this.token_ = new Token(JsonToken.BEGIN_OBJ, new Value("{"));
case '}':
return this.token_ = new Token(JsonToken.END_OBJ, new Value("}"));
case '[':
return this.token_ = new Token(JsonToken.BEGIN_ARR, new Value("["));
case ']':
return this.token_ = new Token(JsonToken.END_ARR, new Value("]"));
case ':':
return this.token_ = new Token(JsonToken.COLON, new Value(":"));
case ',':
return this.token_ = new Token(JsonToken.COMMA, new Value(","));
case '1': case '2': case '3': case '4': case '5':
case '6': case '7': case '8': case '9': case '0':
case '-': case '+': case '.':
return this.token_ = readNumber(this.at_);
case '\"':
return this.token_ = readString();
case 'n':
return this.token_ = readNull();
case 't':
return this.token_ = readTrue();
case 'f':
return this.token_ = readFalse();
case -1:
return this.token_ = new Token(JsonToken.EOF, new Value("eof"));
default:
this.token_ = null;
error("scan->default",this.at_);
return null;
}
}
詞法分析就完成了。
語(yǔ)法分析
語(yǔ)法分析有LL， LR等方法這里采取LL(1)分析。
首先要構(gòu)造出JSON文法。根據(jù)JSON基礎(chǔ)格式。
obj -> { members }
members -> pair members' | eps
members' -> , pair members' | eps
pair -> string : value
array -> [ elem ]
elem -> value elem' | eps
elem' -> , value elem' | eps
value -> obj | array | number | string | true | false | null
求出FIRST集和FOLLOW集，驗(yàn)證可知為L(zhǎng)L(1)文法。這樣就可以用遞歸下降來(lái)做語(yǔ)法分析。
之后寫(xiě)出SDT就可以根據(jù)SDT來(lái)編寫(xiě)代碼了。
public Json parse() throws IOException, JsonParseException, JsonLexException {
Token token = lexer_.scan();
if (token.getToken() == JsonToken.BEGIN_OBJ) {
return object();
} else if (token.getToken() == JsonToken.BEGIN_ARR) {
return array();
} else {
error("parse", token.toString());
return null;
}
}
//文法 { member }
public JsonObject object() throws IOException,
JsonParseException, JsonLexException {
Map<String, Value> map = new HashMap<>();
return new JsonObject(member(map));
}
//對(duì)應(yīng)文法 pair mem' | eps
public Map<String ,Value> member(Map<String, Value> map) throws IOException,
JsonLexException, JsonParseException {
Token token = lexer_.scan();
if (token.getToken() == JsonToken.END_OBJ) { //eps
return map;
}
return member_1( pair(map) );
}
//對(duì)應(yīng)文法 , pair mem' | eps
public Map<String, Value> member_1(Map<String, Value> map) throws IOException, JsonLexException, JsonParseException {
Token token = lexer_.scan();
if (token.getToken() == JsonToken.COMMA) { //,
lexer_.scan();
return member_1( pair(map) );
} else if (token.getToken() == JsonToken.END_OBJ) { //eps
return map;
} else {
error("member_1", token.toString());
return null;
}
}
//文法 string : value
private Map<String, Value> pair(Map<String, Value> map) throws IOException, JsonLexException, JsonParseException {
Token token = lexer_.peek();
if (token.getToken() == JsonToken.STRING) {
String key = (String) token.getValue().get();
token = lexer_.scan();
if (token.getToken() == JsonToken.COLON) {
lexer_.scan();
map.put(key, value());
return map;
} else {
error("pair", token.toString());
return null;
}
} else {
error("pair", token.toString());
return null;
}
}
//文法 [ elem ]
public JsonArray array() throws IOException,JsonParseException, JsonLexException {
List<Value> list = new LinkedList<>();
return new JsonArray(elem(list));
}
//文法 value elem' | eps
public List<Value> elem(List<Value> list) throws IOException,
JsonLexException,JsonParseException {
Token token = lexer_.scan();
if (token.getToken() == JsonToken.END_ARR) { // eps
return list;
} else {
list.add(value());
return elem_1(list);
}
}
// 文法 , value elem' | eps
public List<Value> elem_1(List<Value> list) throws IOException,
JsonLexException, JsonParseException {
Token token = lexer_.scan();
if (token.getToken() == JsonToken.COMMA) { // ,
lexer_.scan();
list.add(value());
return elem_1(list);
} else if (token.getToken() == JsonToken.END_ARR) { // eps
return list;
} else {
error("elem_1",token.toString());
return null;
}
}
public Value value() throws IOException, JsonLexException,
JsonParseException {
Token token = lexer_.peek();
if (isCommonValue(token.getToken())) {
return new Value(token.getValue());
} else if (token.getToken() == JsonToken.BEGIN_OBJ) {
return new Value(object());
} else if (token.getToken() == JsonToken.BEGIN_ARR) {
return new Value(array());
} else {
error("value",token.toString());
return null;
}
}
private boolean isCommonValue(JsonToken token) {
return (token == JsonToken.STRING || token == JsonToken.FALSE ||
token == JsonToken.INT || token == JsonToken.DOUBLE ||
token == JsonToken.TRUE || token == JsonToken.NULL) ? true : false;
}
public void error(String position, String value) throws JsonParseException {
throw new JsonParseException(String.format("an error occurred on Parser [%s %s]",position,value));
}
至此一個(gè)簡(jiǎn)單的解析器已經(jīng)寫(xiě)完了，只是實(shí)現(xiàn)了簡(jiǎn)單的解析功能，一個(gè)完善的JAVA JSON解析器至少可以對(duì)對(duì)象序列化和反序列化。后續(xù)將會(huì)添加上這部分功能，再更新一遍 :-|
源代碼：ToyJSON
