快速入門
以下是基于官方關于hawkular alert的示例在本地的實踐,更改了官方示例中不正確的部分。
添加警報
警報觸發器的組成:
- 一組觸發警報/事件的條件;
- 滿足條件時要執行一個或多個操作(發送電子郵件,調用webhook等);
- 一些額外的元數據(如嚴重性);
下面的示例觸發條件為:當氣溫低于0攝氏度時,示例會發送一封電子郵件至“admin@example.org”。
此處為了演示方便將所有的內容放在了一個文件中,如下:
trigger_definition.json
{
"triggers": [
{
"trigger": {
"id": "temperature-trigger", //指定trigger id
"name": "Trigger for the temperature sensor", //指定trigger name
"severity": "HIGH", //級別
"enabled": true, //啟用
"actions": [
{
"actionPlugin": "email", //觸發后將會以郵件的形式提醒
"actionId": "notify-admin"
}
]
},
"conditions": [
{
"triggerMode": "FIRING",
"type": "threshold",
"dataId": "hm_g_temperature", //指標數據(此處指標名稱為:temperature,由于是gauge類型,因此加“hm_g_前綴”)
"operator": "LT",
"threshold": 0
}
]
}
],
"actions": [
{
"actionPlugin": "email",
"actionId": "notify-admin",
"properties": {
"to": "admin@example.org" //郵箱地址,可改成你可用的郵箱
}
}
]
}
導入觸發器定義:
curl -u myUsername:myPassword -X POST http://localhost:8080/hawkular/alerts/import/all -d @trigger_definition.json \
-H "Content-Type: application/json" -H "Hawkular-Tenant: myTenant"
測試
基于之前的配置,導入滿足觸發條件的指標數據,將會發送電子郵件。
補充:hawkular默認使用localhost:25發送郵件。
metrics_day_2.json
[
{"timestamp": 1468620000000, "value": 8},
{"timestamp": 1468621800000, "value": 6},
{"timestamp": 1468623600000, "value": 3},
{"timestamp": 1468625400000, "value": 0},
{"timestamp": 1468627200000, "value": -2},
{"timestamp": 1468629000000, "value": -3},
{"timestamp": 1468630800000, "value": -1},
{"timestamp": 1468632600000, "value": 2},
{"timestamp": 1468634400000, "value": 4},
{"timestamp": 1468636200000, "value": 5},
{"timestamp": 1468638000000, "value": 8},
{"timestamp": 1468639800000, "value": 12},
{"timestamp": 1468641600000, "value": 13},
{"timestamp": 1468643400000, "value": 12},
{"timestamp": 1468645200000, "value": 13.4},
{"timestamp": 1468647000000, "value": 14},
{"timestamp": 1468648800000, "value": 14.3},
{"timestamp": 1468650600000, "value": 14.6},
{"timestamp": 1468652400000, "value": 17},
{"timestamp": 1468654200000, "value": 17.3},
{"timestamp": 1468656000000, "value": 17.5},
{"timestamp": 1468657800000, "value": 17.9},
{"timestamp": 1468659600000, "value": 18},
{"timestamp": 1468661400000, "value": 18.7},
{"timestamp": 1468663200000, "value": 19.2}
]
存入指標數據:
curl -u myUsername:myPassword -X POST http://localhost:8080/hawkular/metrics/gauges/temperature/raw -d @metrics_day_2.json \
-H "Content-Type: application/json" -H "Hawkular-Tenant: myTenant"
完成之后可以在日志文件中看到如下的輸出:
15:10:02,110 INFO [org.hawkular.alerts.actions.api] (standalone-action-3) HAWKALERT240001: Plugin [email] has received an action message: [StandaloneActionMessage[action=Action[eventId='temperature-trigger-1501053001894-ee945e8a-7c26-4a5a-8236-de476d21f70c', ctime=1501053001903, event=Alert [tenantId=myTenant, triggerId=temperature-trigger, severity=HIGH, status=OPEN, ctime=1501053001894, lifecycle=[LifeCycle{user='system', status=OPEN, stime=1501053001894}], resolvedEvalSets=null], result='WAITING']]]
15:10:02,113 INFO [org.hawkular.alerts.actions.api] (standalone-action-4) HAWKALERT240001: Plugin [email] has received an action message: [StandaloneActionMessage[action=Action[eventId='temperature-trigger-1501053001895-d8046519-6e4e-430d-85e9-059f6333f549', ctime=1501053001903, event=Alert [tenantId=myTenant, triggerId=temperature-trigger, severity=HIGH, status=OPEN, ctime=1501053001895, lifecycle=[LifeCycle{user='system', status=OPEN, stime=1501053001895}], resolvedEvalSets=null], result='WAITING']]]
15:10:02,121 INFO [org.hawkular.alerts.actions.api] (standalone-action-5) HAWKALERT240001: Plugin [email] has received an action message: [StandaloneActionMessage[action=Action[eventId='temperature-trigger-1501053001892-55aa4be1-e955-4eaf-8d14-b79e8d4ec24f', ctime=1501053001897, event=Alert [tenantId=myTenant, triggerId=temperature-trigger, severity=HIGH, status=OPEN, ctime=1501053001892, lifecycle=[LifeCycle{user='system', status=OPEN, stime=1501053001892}], resolvedEvalSets=null], result='WAITING']]]
此時,我們可以在Cassandra數據庫中查看一下我們的警報配置。
-
進入當前集群hawkular中的node1節點,啟動cqlsh:
[root@localhost node1]# ./bin/cqlsh Connected to hawkular at 127.0.0.1:9042. [cqlsh 5.0.1 | Cassandra 3.0.12 | CQL spec 3.4.0 | Native protocol v4] Use HELP for help. cqlsh>
-
查看所有的keyspace:
cqlsh> describe keyspaces; system_schema hawkular_metrics hawkular_alerts system_traces system_auth system system_distributed
-
進入hawkular_alerts鍵空間:
cqlsh> use hawkular_alerts; cqlsh:hawkular_alerts>
-
查看該空間中的所有表:
cqlsh:hawkular_alerts> describe tables; alerts_statuses events cassalog actions_history_ctimes dampenings actions_definitions events_triggers tags alerts_triggers alerts_severities actions_history_alerts action_plugins alerts_lifecycle actions_history triggers_actions alerts_stimes actions_history_results sys_config alerts_ctimes events_categories actions_history_actions alerts triggers conditions events_ctimes
-
在表actions_definitions數據:
cqlsh:hawkular_alerts> select * from actions_definitions; tenantid | actionplugin | actionid | payload ----------+--------------+--------------+----------------------------------------------- ---------------------------------------------------------------------------------- myTenant | email | notify-admin | {"tenantId":"myTenant","actionPlugin":"email","actionId":"notify-admin","global":false,"properties":{"to":"admin@example.org"}} (1 rows)
上述結果可以看到我們之前的觸發定義信息。
登錄郵箱可看到如下圖的郵件:
補充:
在Hawkular Alerting Details中有介紹關于alert中condition部分的dataId名稱的問題,不同類型metric的dataId要添加不同的前綴,具體如下:
- hm_a: availability
- hm_c: counter
- hm_cr: counter rate
- hm_g: gauge
- hm_gr: gauge rate
- hm_s: string
因此,當我們有一個gauge類型的指標temperature時,dataId就要寫成:hm_g_temperature,否則就不會發生告警(所以,按照官方文檔實踐的同學就要注意啦,記得要改過來喲!)。
常用API命令示例
如果使用的是openshift中的hawkular metrics,則需要將-u "username: userpassword"
替換為-H "Authorization: Bearer XXXXXX"
,并使用https
。
查看alert狀態
curl -u "username:userpassword" -X GET http://localhost:8080/hawkular/alerts/status -H "Content-Type: application/json" -H "Hawkular-Tenant: myTenant"
響應結果:
{"distributed":"false","Implementation-Version":"1.6.0.Final","Built-From-Git-SHA1":"82b3bb25a3b5a6d3e0e793110c6c544d6d0ce1cf","status":"STARTED"}
導入triggers和actions定義
curl -u "username:userpassword" -X POST http://localhost:8080/hawkular/alerts/import/all -d @trigger_definition.json -H "Content-Type: application/json" -H "Hawkular-Tenant: myTenant"
響應結果:
{"triggers":[{"trigger":{"tenantId":"myTenant","id":"memory_usage-trigger","name":"Trigger for the memory sensor","type":"STANDARD","eventType":"ALERT","eventCategory":null,"eventText":null,"severity":"HIGH","actions":[{"tenantId":"myTenant","actionPlugin":"email","actionId":"notify-admin"}],"autoDisable":false,"autoEnable":false,"autoResolve":false,"autoResolveAlerts":true,"autoResolveMatch":"ALL","enabled":true,"firingMatch":"ALL","source":"_none_"},"conditions":[{"tenantId":"myTenant","triggerId":"memory_usage-trigger","triggerMode":"FIRING","type":"THRESHOLD","conditionSetSize":1,"conditionSetIndex":1,"conditionId":"myTenant-memory_usage-trigger-FIRING-1-1","dataId":"hm_g_elasticsearch/9f5dae41-71d0-11e7-97e5-fa163eaf6ed5/memory/usage","operator":"GT","threshold":1.8E9}]}],"actions":[{"tenantId":"myTenant","actionPlugin":"email","actionId":"notify-admin","global":false,"properties":{"to":"email@example.com"}}]}
導出triggers和actions定義
curl -u "username:userpassword" -X GET http://localhost:8080/hawkular/alerts/export -H "Content-Type: application/json" -H "Hawkular-Tenant: myTenant"
響應結果:
{"triggers":[{"trigger":{"tenantId":"myTenant","id":"temperature-trigger","name":"Trigger for the temperature sensor","type":"STANDARD","eventType":"ALERT","eventCategory":null,"eventText":null,"severity":"HIGH","actions":[{"tenantId":"myTenant","actionPlugin":"email","actionId":"notify-admin"}],"autoDisable":false,"autoEnable":false,"autoResolve":false,"autoResolveAlerts":true,"autoResolveMatch":"ALL","enabled":true,"firingMatch":"ALL","source":"_none_"},"conditions":[{"tenantId":"myTenant","triggerId":"temperature-trigger","triggerMode":"FIRING","type":"THRESHOLD","conditionSetSize":1,"conditionSetIndex":1,"conditionId":"myTenant-temperature-trigger-FIRING-1-1","dataId":"hm_g_temperature","operator":"LT","threshold":0.0}]}],"actions":[{"tenantId":"myTenant","actionPlugin":"email","actionId":"notify-admin","global":false,"properties":{"to":"email@example.com"}}]}
查看可使用的action插件
curl -u "username:userpassword" -X GET http://localhost:8080/hawkular/alerts/plugins -H "Content-Type: application/json" -H "Hawkular-Tenant: myTenant"
響應結果:
["elasticsearch","email","webhook"]
獲取已經定義的trigger
curl -u "username:userpassword" -X GET http://localhost:8080/hawkular/alerts/triggers/your_trigger_id -H "Content-Type: application/json" -H "Hawkular-Tenant: myTenant"
響應結果:
{"tenantId":"myTenant","id":"temperature-trigger","name":"Trigger for the temperature sensor","type":"STANDARD","eventType":"ALERT","eventCategory":null,"eventText":null,"severity":"HIGH","actions":[{"tenantId":"myTenant","actionPlugin":"email","actionId":"notify-admin"}],"autoDisable":false,"autoEnable":false,"autoResolve":false,"autoResolveAlerts":true,"autoResolveMatch":"ALL","enabled":true,"firingMatch":"ALL","source":"_none_"}
獲取特定trigger的condition
curl -u "username:userpassword" -X GET http://localhost:8080/hawkular/alerts/triggers/temperature-trigger/conditions -H "Content-Type: application/json" -H "Hawkular-Tenant: myTenant"
響應結果:
[{"tenantId":"myTenant","triggerId":"temperature-trigger","triggerMode":"FIRING","type":"THRESHOLD","conditionSetSize":1,"conditionSetIndex":1,"conditionId":"myTenant-temperature-trigger-FIRING-1-1","dataId":"hm_g_temperature","operator":"LT","threshold":0.0}]