query context & filter context
image.png
- 高級搜索的功能：支持多想文本輸入，針對多個字段進行搜索
- 搜索引擎一般也提供基于時間，價格等條件的guolv
- elasticsearch中，有query和filter兩種不同的context
- query contest：相關性算分
- filter context：不需要算分，可以利用cache獲得更好的性能
條件組合
- 假設要搜索一本電影，包含了以下一些條件
- 評論中包含了guitar，用戶打分高于3分，同時上映日期要在1993到2000年之間
- 這個搜索其實包含了3段邏輯，針對不同的字段
- 評論字段中要包含guitar/用戶評分大于3/上映日期需要在給定的范圍
- 同時包含這三個邏輯，并且有比較好的性能？
- 符合查詢：bool query
bool 查詢
- 一個bool查詢，是一個或者多個查詢子句的組合
- 總共包括4種子句，其中2種會影響算分，2種不影響算分
- 相關性并不只是全文本檢索的專利，也適用于yes | no的子句，匹配的子句越多，相關性評分越高。如果多條查詢子句被合并為一條符合查詢語句，比如bool查詢，則每個查詢子句計算得出的評分會被合并到總的相關性評分中
| 英文 | 描述 |
|---|---|
| must | 必須匹配，貢獻算分 |
| should | 選擇性匹配，貢獻算分 |
| must_not | filter context，查詢子句，必須不能匹配 |
| filter | filter context，必須匹配，但是不貢獻算分 |
bool查詢語法
- 子查詢可以以任意順序出席那
- 可以嵌套多個查詢
- 如果你的bool查詢中 ，沒有must條件，should中必須至少滿足一條查詢
image.png
如何解決結構化查詢-“包含而不是相等”的問題
-
解決方案：增加一個genre count字段進行計數
image.png
image.png -
從業務角度，按需修改elasticsearch的數據模型
image.png
filter context - 不影響算分
image.png
bool嵌套
-
實現了should not的邏輯
image.png
查詢語句的結構，會對相關度算分產生影響
- 同一層級下的競爭字段，具有相同的權重
-
通過嵌套bool查詢，可以改變對算分的影響
image.png
控制字段的boosting
- boosting是控制相關度的一種手段
- 索引，字段或者查詢子條件
- 參數boost的含義
- 當boost>1時候，打分的相關度相對性提升
- 當0<boost<1時候，打分的相關度相對性降低
-
當boost<0時候，貢獻負分
image.png
not quite not
-
要求蘋果公司的產品信息優先
image.png
回顧
- query context vs filter context
- bool query - 更多的條件組合
- 查詢結構與相關性算分
- 如何控制查詢的精確度
- boosting & boosting query
示例
POST /products/_bulk
{ "index": { "_id": 1 }}
{ "price" : 10,"avaliable":true,"date":"2018-01-01", "productID" : "XHDK-A-1293-#fJ3" }
{ "index": { "_id": 2 }}
{ "price" : 20,"avaliable":true,"date":"2019-01-01", "productID" : "KDKE-B-9947-#kL5" }
{ "index": { "_id": 3 }}
{ "price" : 30,"avaliable":true, "productID" : "JODL-X-1937-#pV7" }
{ "index": { "_id": 4 }}
{ "price" : 30,"avaliable":false, "productID" : "QQPX-R-3956-#aD8" }
#基本語法
POST /products/_search
{
"query": {
"bool" : {
"must" : {
"term" : { "price" : "30" }
},
"filter": {
"term" : { "avaliable" : "true" }
},
"must_not" : {
"range" : {
"price" : { "lte" : 10 }
}
},
"should" : [
{ "term" : { "productID.keyword" : "JODL-X-1937-#pV7" } },
{ "term" : { "productID.keyword" : "XHDK-A-1293-#fJ3" } }
],
"minimum_should_match" :1
}
}
}
#改變數據模型，增加字段。解決數組包含而不是精確匹配的問題
POST /newmovies/_bulk
{ "index": { "_id": 1 }}
{ "title" : "Father of the Bridge Part II","year":1995, "genre":"Comedy","genre_count":1 }
{ "index": { "_id": 2 }}
{ "title" : "Dave","year":1993,"genre":["Comedy","Romance"],"genre_count":2 }
#must，有算分
POST /newmovies/_search
{
"query": {
"bool": {
"must": [
{"term": {"genre.keyword": {"value": "Comedy"}}},
{"term": {"genre_count": {"value": 1}}}
]
}
}
}
#Filter。不參與算分，結果的score是0
POST /newmovies/_search
{
"query": {
"bool": {
"filter": [
{"term": {"genre.keyword": {"value": "Comedy"}}},
{"term": {"genre_count": {"value": 1}}}
]
}
}
}
#Filtering Context
POST _search
{
"query": {
"bool" : {
"filter": {
"term" : { "avaliable" : "true" }
},
"must_not" : {
"range" : {
"price" : { "lte" : 10 }
}
}
}
}
}
#Query Context
POST /products/_bulk
{ "index": { "_id": 1 }}
{ "price" : 10,"avaliable":true,"date":"2018-01-01", "productID" : "XHDK-A-1293-#fJ3" }
{ "index": { "_id": 2 }}
{ "price" : 20,"avaliable":true,"date":"2019-01-01", "productID" : "KDKE-B-9947-#kL5" }
{ "index": { "_id": 3 }}
{ "price" : 30,"avaliable":true, "productID" : "JODL-X-1937-#pV7" }
{ "index": { "_id": 4 }}
{ "price" : 30,"avaliable":false, "productID" : "QQPX-R-3956-#aD8" }
POST /products/_search
{
"query": {
"bool": {
"should": [
{
"term": {
"productID.keyword": {
"value": "JODL-X-1937-#pV7"}}
},
{"term": {"avaliable": {"value": true}}
}
]
}
}
}
#嵌套，實現了 should not 邏輯
POST /products/_search
{
"query": {
"bool": {
"must": {
"term": {
"price": "30"
}
},
"should": [
{
"bool": {
"must_not": {
"term": {
"avaliable": "false"
}
}
}
}
],
"minimum_should_match": 1
}
}
}
#Controll the Precision
POST _search
{
"query": {
"bool" : {
"must" : {
"term" : { "price" : "30" }
},
"filter": {
"term" : { "avaliable" : "true" }
},
"must_not" : {
"range" : {
"price" : { "lte" : 10 }
}
},
"should" : [
{ "term" : { "productID.keyword" : "JODL-X-1937-#pV7" } },
{ "term" : { "productID.keyword" : "XHDK-A-1293-#fJ3" } }
],
"minimum_should_match" :2
}
}
}
POST /animals/_search
{
"query": {
"bool": {
"should": [
{ "term": { "text": "brown" }},
{ "term": { "text": "red" }},
{ "term": { "text": "quick" }},
{ "term": { "text": "dog" }}
]
}
}
}
POST /animals/_search
{
"query": {
"bool": {
"should": [
{ "term": { "text": "quick" }},
{ "term": { "text": "dog" }},
{
"bool":{
"should":[
{ "term": { "text": "brown" }},
{ "term": { "text": "brown" }},
]
}
}
]
}
}
}
DELETE blogs
POST /blogs/_bulk
{ "index": { "_id": 1 }}
{"title":"Apple iPad", "content":"Apple iPad,Apple iPad" }
{ "index": { "_id": 2 }}
{"title":"Apple iPad,Apple iPad", "content":"Apple iPad" }
POST blogs/_search
{
"query": {
"bool": {
"should": [
{"match": {
"title": {
"query": "apple,ipad",
"boost": 1.1
}
}},
{"match": {
"content": {
"query": "apple,ipad",
"boost":
}
}}
]
}
}
}
DELETE news
POST /news/_bulk
{ "index": { "_id": 1 }}
{ "content":"Apple Mac" }
{ "index": { "_id": 2 }}
{ "content":"Apple iPad" }
{ "index": { "_id": 3 }}
{ "content":"Apple employee like Apple Pie and Apple Juice" }
POST news/_search
{
"query": {
"bool": {
"must": {
"match":{"content":"apple"}
}
}
}
}
POST news/_search
{
"query": {
"bool": {
"must": {
"match":{"content":"apple"}
},
"must_not": {
"match":{"content":"pie"}
}
}
}
}
POST news/_search
{
"query": {
"boosting": {
"positive": {
"match": {
"content": "apple"
}
},
"negative": {
"match": {
"content": "pie"
}
},
"negative_boost": 0.5
}
}
}
