前言
開(kāi)源版本的EMQ服務(wù)器不支持消息的持久化，但是支持通過(guò)開(kāi)發(fā)擴(kuò)展插件的方式實(shí)現(xiàn)消息的持久化。本文記錄一下EMQ通過(guò)插件將消息發(fā)往kafka的實(shí)現(xiàn)過(guò)程。內(nèi)容寫(xiě)的比較詳細(xì)，其實(shí)不復(fù)雜，一步步來(lái)就可以實(shí)現(xiàn)的。
參考文檔：
1、物聯(lián)網(wǎng)架構(gòu)成長(zhǎng)之路(3)-EMQ消息服務(wù)器了解
2、物聯(lián)網(wǎng)架構(gòu)成長(zhǎng)之路(4)-EMQ插件創(chuàng)建
3、EMQ集成Kafka插件編寫(xiě)過(guò)程 emq_plugin_kafka
4、物聯(lián)網(wǎng)架構(gòu)成長(zhǎng)之路(5)-EMQ插件配置
1和2幫助了解EMQ服務(wù)器，對(duì)源代碼進(jìn)行編譯，新增擴(kuò)展插件，并編譯；
3和4進(jìn)行實(shí)際的擴(kuò)展插件開(kāi)發(fā)及編譯，啟動(dòng)測(cè)試等。
一、源代碼編譯
EMQ服務(wù)器是基于Erlang/OTP語(yǔ)言平臺(tái)開(kāi)發(fā)的，需要先準(zhǔn)備OTP環(huán)境
1.1 Erlang/OTP環(huán)境安裝
我自己到官網(wǎng)下載的opt_src_21安裝的過(guò)程中出現(xiàn)了錯(cuò)誤，好像是openssl的加密庫(kù)有變化，后面使用opt_src_19還比較順利，可以通過(guò)我的分享下載該版本，opt_src_19下載。
具體的安裝過(guò)程不再贅述，請(qǐng)參考CentOS 部署EMQ服務(wù)的2.2章節(jié) 準(zhǔn)備Erlang環(huán)境
1.2 EMQ源代碼下載及編譯
命令行執(zhí)行：git clone https://github.com/emqtt/emq-relx下載EMQ源碼
執(zhí)行：cd emq-relx，進(jìn)入源碼文件路徑
執(zhí)行：make，進(jìn)行編譯，這個(gè)過(guò)程會(huì)比較耗時(shí)，需要下載較多的依賴(lài)。
如果沒(méi)有報(bào)錯(cuò)，會(huì)在路徑下看到_rel文件夾，里面是編譯好的emqttd相關(guān)文件，進(jìn)入到bin目錄，執(zhí)行sh emqttd console，正常的話(huà)是可以啟動(dòng)EMQ服務(wù)的。
二、新增一個(gè)擴(kuò)展插件
emq-rex的文件路徑如圖：
1、執(zhí)行：cd deps，進(jìn)入deps目錄
2、執(zhí)行：cp -r emp_plugin_template emp_plugin_kafka，template是自帶的插件模板，我們復(fù)制一份作為kafka插件的基礎(chǔ)
3、執(zhí)行：cd emq_plugin_kafka，進(jìn)入kafka目錄，執(zhí)行make clean，接著把全部文件名中的template替換為kafka，注意是全部文件及子目錄下的文件，包括文件的內(nèi)容，都要替換。具體如下：
a、etc路徑下.config后綴改為.conf后綴，文件名中template替換為kafka，內(nèi)容清空
b、Makefile中增加：
內(nèi)容中的
template替換為kafka c、src路徑下文件名及內(nèi)容的
template替換為kafka，三個(gè)_demo.erl文件隨便加個(gè)后綴，如_demo_gx.erl，同時(shí)修改內(nèi)容中的template替換為kafka，內(nèi)容中module后面()里面的內(nèi)容修改為文件名。emq_plugin_kafka_app.erl文件中引用到_demo的地方也增加剛剛添加的后綴src文件目錄：
emq_acl_demo_gx.erl文件中
emq_plugin_kafka_app.erl文件中
d、test路徑下文件名及內(nèi)容的
template替換為kafka
4、在Makefile文件的當(dāng)前路徑下，執(zhí)行make，編譯插件emq_plugin_kafka，直到編譯成功。 如果沒(méi)有報(bào)錯(cuò)，就是成功了，如下圖：
5、回到emq-relx目錄下，執(zhí)行vi Makefile，增加我們新增的自定義插件emq_plugin_kafka，如下圖：
執(zhí)行vi relx.config，增加如下圖：
6、執(zhí)行rm -rf _rel，先刪除上次編譯生成的文件，執(zhí)行make clean && make，直到編譯成功
7、執(zhí)行cd _rel/emqttd/bin，進(jìn)入編譯好的文件路徑，執(zhí)行sh emqttd console，啟動(dòng)EMQ服務(wù)，啟動(dòng)成功后如下圖：
瀏覽器訪問(wèn)本機(jī)ip:18083，登錄賬號(hào)密碼為（admin、public），啟動(dòng)emq_plugin_kafka插件
到此我們新增的一個(gè)插件就配置好了，只是該插件還未實(shí)現(xiàn)具體功能
三、實(shí)現(xiàn)emq_plugin_kafka的具體功能
1、回到emq-relx/deps/emq_plugin_kafka，插件目錄，執(zhí)行mkdir priv，創(chuàng)建一個(gè)文件夾，進(jìn)入priv路徑，新增一個(gè)文件emq_plugin_kafka.schema，這個(gè)文件的作用是給最后編譯出來(lái)的emqttd的配置屬性中增加兩個(gè)配置參數(shù)，kafka中消息的topic和server地址。文件內(nèi)容如下：
%% emq.plugin.kafka.server, %是Erlang語(yǔ)言的注釋標(biāo)記，表示給emqttd增加一個(gè)emq.plugin.kafka.server的可配置屬性
{
mapping,
"emq.plugin.kafka.server",
"emq_plugin_kafka.kafka",
[
{default, {"127.0.0.1", 9092}},
{datatype, [integer, ip, string]}
]
}.
%% emq.plugin.kafka.topic 表示給emqttd增加一個(gè)emq.plugin.kafka.topic的可配置屬性
{
mapping,
"emq.plugin.kafka.topic",
"emq_plugin_kafka.kafka",
[
{default, "test"},
{datatype, string},
hidden
]
}.
%% translation
{
translation,
"emq_plugin_kafka.kafka",
fun(Conf) ->
{RHost, RPort} = case cuttlefish:conf_get("emq.plugin.kafka.server", Conf) of
{Ip, Port} -> {Ip, Port};
S -> case string:tokens(S, ":") of
[Domain] -> {Domain, 9092};
[Domain, Port] -> {Domain, list_to_integer(Port)}
end
end,
Topic = cuttlefish:conf_get("emq.plugin.kafka.topic", Conf),
[
{host, RHost},
{port, RPort},
{topic, Topic}
]
end
}.
2、執(zhí)行vi etc/emq_plugin_kafka.conf，編輯配置文件，增加下面兩行：
emq.plugin.kafka.server = 127.0.0.1:9092
emq.plugin.kafka.topic = test_emq
server是插件要連接的kafka服務(wù)器地址(kafka還不太熟悉，我是本地起的kafka服務(wù)，可以正常連接)，topic是發(fā)往kafka的消息的topic
3、執(zhí)行vi src/emq_plugin_kafka.erl，編輯插件的實(shí)現(xiàn)代碼，內(nèi)容如下：
%%--------------------------------------------------------------------
%% Copyright (c) 2013-2018 EMQ Enterprise, Inc. (http://emqtt.io)
%%
%% Licensed under the Apache License, Version 2.0 (the "License");
%% you may not use this file except in compliance with the License.
%% You may obtain a copy of the License at
%%
%% http://www.apache.org/licenses/LICENSE-2.0
%%
%% Unless required by applicable law or agreed to in writing, software
%% distributed under the License is distributed on an "AS IS" BASIS,
%% WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
%% See the License for the specific language governing permissions and
%% limitations under the License.
%%--------------------------------------------------------------------
-module(emq_plugin_kafka).
-include_lib("emqttd/include/emqttd.hrl").
-define(APP, emq_plugin_kafka).
-export([load/1, unload/0]).
%% Hooks functions
-export([on_client_connected/3, on_client_disconnected/3]).
-export([on_client_subscribe/4, on_client_unsubscribe/4]).
-export([on_session_created/3, on_session_subscribed/4, on_session_unsubscribed/4, on_session_terminated/4]).
-export([on_message_publish/2, on_message_delivered/4, on_message_acked/4]).
%% Called when the plugin application start
load(Env) ->
ekaf_init([Env]),
emqttd:hook('client.connected', fun ?MODULE:on_client_connected/3, [Env]),
emqttd:hook('client.disconnected', fun ?MODULE:on_client_disconnected/3, [Env]),
emqttd:hook('client.subscribe', fun ?MODULE:on_client_subscribe/4, [Env]),
emqttd:hook('client.unsubscribe', fun ?MODULE:on_client_unsubscribe/4, [Env]),
emqttd:hook('session.created', fun ?MODULE:on_session_created/3, [Env]),
emqttd:hook('session.subscribed', fun ?MODULE:on_session_subscribed/4, [Env]),
emqttd:hook('session.unsubscribed', fun ?MODULE:on_session_unsubscribed/4, [Env]),
emqttd:hook('session.terminated', fun ?MODULE:on_session_terminated/4, [Env]),
emqttd:hook('message.publish', fun ?MODULE:on_message_publish/2, [Env]),
emqttd:hook('message.delivered', fun ?MODULE:on_message_delivered/4, [Env]),
emqttd:hook('message.acked', fun ?MODULE:on_message_acked/4, [Env]),
io:format("load completed~n", []).
on_client_connected(ConnAck, Client = #mqtt_client{client_id = ClientId}, _Env) ->
io:format("client ~s connected, connack: ~w~n", [ClientId, ConnAck]),
ekaf_send(<<"connected">>, ClientId, {}, _Env),
{ok, Client}.
on_client_disconnected(Reason, _Client = #mqtt_client{client_id = ClientId}, _Env) ->
io:format("client ~s disconnected, reason: ~w~n", [ClientId, Reason]),
ekaf_send(<<"disconnected">>, ClientId, {}, _Env),
ok.
on_client_subscribe(ClientId, Username, TopicTable, _Env) ->
io:format("client(~s/~s) will subscribe: ~p~n", [Username, ClientId, TopicTable]),
{ok, TopicTable}.
on_client_unsubscribe(ClientId, Username, TopicTable, _Env) ->
io:format("client(~s/~s) unsubscribe ~p~n", [ClientId, Username, TopicTable]),
{ok, TopicTable}.
on_session_created(ClientId, Username, _Env) ->
io:format("session(~s/~s) created.", [ClientId, Username]).
on_session_subscribed(ClientId, Username, {Topic, Opts}, _Env) ->
io:format("session(~s/~s) subscribed: ~p~n", [Username, ClientId, {Topic, Opts}]),
ekaf_send(<<"subscribed">>, ClientId, {Topic, Opts}, _Env),
{ok, {Topic, Opts}}.
on_session_unsubscribed(ClientId, Username, {Topic, Opts}, _Env) ->
io:format("session(~s/~s) unsubscribed: ~p~n", [Username, ClientId, {Topic, Opts}]),
ekaf_send(<<"unsubscribed">>, ClientId, {Topic, Opts}, _Env),
ok.
on_session_terminated(ClientId, Username, Reason, _Env) ->
io:format("session(~s/~s) terminated: ~p.", [ClientId, Username, Reason]).
%% transform message and return
on_message_publish(Message = #mqtt_message{topic = <<"$SYS/", _/binary>>}, _Env) ->
{ok, Message};
on_message_publish(Message, _Env) ->
io:format("publish ~s~n", [emqttd_message:format(Message)]),
% ekaf_send(Message, _Env),
ekaf_send(<<"public">>, {}, Message, _Env),
{ok, Message}.
on_message_delivered(ClientId, Username, Message, _Env) ->
io:format("delivered to client(~s/~s): ~s~n", [Username, ClientId, emqttd_message:format(Message)]),
{ok, Message}.
on_message_acked(ClientId, Username, Message, _Env) ->
io:format("client(~s/~s) acked: ~s~n", [Username, ClientId, emqttd_message:format(Message)]),
{ok, Message}.
%% Called when the plugin application stop
unload() ->
emqttd:unhook('client.connected', fun ?MODULE:on_client_connected/3),
emqttd:unhook('client.disconnected', fun ?MODULE:on_client_disconnected/3),
emqttd:unhook('client.subscribe', fun ?MODULE:on_client_subscribe/4),
emqttd:unhook('client.unsubscribe', fun ?MODULE:on_client_unsubscribe/4),
emqttd:unhook('session.created', fun ?MODULE:on_session_created/3),
emqttd:unhook('session.subscribed', fun ?MODULE:on_session_subscribed/4),
emqttd:unhook('session.unsubscribed', fun ?MODULE:on_session_unsubscribed/4),
emqttd:unhook('session.terminated', fun ?MODULE:on_session_terminated/4),
emqttd:unhook('message.publish', fun ?MODULE:on_message_publish/2),
emqttd:unhook('message.delivered', fun ?MODULE:on_message_delivered/4),
emqttd:unhook('message.acked', fun ?MODULE:on_message_acked/4).
%% ==================== ekaf_init STA.===============================%%
ekaf_init(_Env) ->
% clique 方式讀取配置文件
Env = application:get_env(?APP, kafka),
{ok, Kafka} = Env,
Host = proplists:get_value(host, Kafka),
Port = proplists:get_value(port, Kafka),
Broker = {Host, Port},
Topic = proplists:get_value(topic, Kafka),
io:format("~w ~w ~w ~n", [Host, Port, Topic]),
% init kafka
application:set_env(ekaf, ekaf_partition_strategy, strict_round_robin),
application:set_env(ekaf, ekaf_bootstrap_broker, Broker),
application:set_env(ekaf, ekaf_bootstrap_topics, list_to_binary(Topic)),
%application:set_env(ekaf, ekaf_bootstrap_broker, {"127.0.0.1", 9092}),
%application:set_env(ekaf, ekaf_bootstrap_topics, <<"test">>),
io:format("Init ekaf with ~s:~b~n", [Host, Port]),
%%ekaf:produce_async_batched(<<"test">>, list_to_binary(Json)),
ok.
%% ==================== ekaf_init END.===============================%%
%% ==================== ekaf_send STA.===============================%%
ekaf_send(Type, ClientId, {}, _Env) ->
Json = mochijson2:encode([
{type, Type},
{client_id, ClientId},
{message, {}},
{cluster_node, node()},
{ts, emqttd_time:now_ms()}
]),
ekaf_send_sync(Json);
ekaf_send(Type, ClientId, {Reason}, _Env) ->
Json = mochijson2:encode([
{type, Type},
{client_id, ClientId},
{cluster_node, node()},
{message, Reason},
{ts, emqttd_time:now_ms()}
]),
ekaf_send_sync(Json);
ekaf_send(Type, ClientId, {Topic, Opts}, _Env) ->
Json = mochijson2:encode([
{type, Type},
{client_id, ClientId},
{cluster_node, node()},
{message, [
{topic, Topic},
{opts, Opts}
]},
{ts, emqttd_time:now_ms()}
]),
ekaf_send_sync(Json);
ekaf_send(Type, _, Message, _Env) ->
Id = Message#mqtt_message.id,
From = Message#mqtt_message.from, %需要登錄和不需要登錄這里的返回值是不一樣的
Topic = Message#mqtt_message.topic,
Payload = Message#mqtt_message.payload,
Qos = Message#mqtt_message.qos,
Dup = Message#mqtt_message.dup,
Retain = Message#mqtt_message.retain,
Timestamp = Message#mqtt_message.timestamp,
ClientId = c(From),
Username = u(From),
Json = mochijson2:encode([
{type, Type},
{client_id, ClientId},
{message, [
{username, Username},
{topic, Topic},
{payload, Payload},
{qos, i(Qos)},
{dup, i(Dup)},
{retain, i(Retain)}
]},
{cluster_node, node()},
{ts, emqttd_time:now_ms()}
]),
ekaf_send_sync(Json).
ekaf_send_sync(Msg) ->
Topic = ekaf_get_topic(),
ekaf_send_sync(Topic, Msg).
ekaf_send_sync(Topic, Msg) ->
ekaf:produce_sync_batched(list_to_binary(Topic), list_to_binary(Msg)).
i(true) -> 1;
i(false) -> 0;
i(I) when is_integer(I) -> I.
c({ClientId, Username}) -> ClientId;
c(From) -> From.
u({ClientId, Username}) -> Username;
u(From) -> From.
%% ==================== ekaf_send END.===============================%%
%% ==================== ekaf_set_topic STA.===============================%%
ekaf_set_topic(Topic) ->
application:set_env(ekaf, ekaf_bootstrap_topics, list_to_binary(Topic)),
ok.
ekaf_get_topic() ->
Env = application:get_env(?APP, kafka),
{ok, Kafka} = Env,
Topic = proplists:get_value(topic, Kafka),
Topic.
%% ==================== ekaf_set_topic END.===============================%%
插件啟動(dòng)后，在load(Env)方法中調(diào)用ekaf_init([Env])，初始化ekaf，ekaf是Erlang語(yǔ)言編寫(xiě)的kafka生產(chǎn)者工具，用于發(fā)送消息到kafka。
在EMQ的各種方法鉤子，如on_client_connected、on_message_publish中調(diào)用ekaf_send()方法即可。
4、執(zhí)行 make clean && make，重新編譯emq_plugin_kafka插件，編譯成功入下圖：
5、回到emq-relx目錄下，執(zhí)行vi relx.config，增加如圖：
6、執(zhí)行vi data/loaded_plugins，這個(gè)文件是emq啟動(dòng)時(shí)自動(dòng)加載并啟動(dòng)的插件，添加一行emq_plugin_kafka.，這樣emq啟動(dòng)時(shí)，會(huì)自動(dòng)加載并啟動(dòng)我們的emq_plugin_kafka插件
7、執(zhí)行rm -rf _rel，執(zhí)行make clean && make，執(zhí)行cd _rel/emqttd/bin，進(jìn)入編譯好的emqttd的bin目錄，執(zhí)行sh emqttd console，啟動(dòng)emqttd服務(wù)。
8、下載kafka壓縮包，解壓縮后進(jìn)入bin目錄，分別啟動(dòng)自帶的zookeeper、啟動(dòng)kafka服務(wù)、創(chuàng)建一個(gè)上文定義的topic、啟動(dòng)一個(gè)kafka消費(fèi)者，正常的話(huà)kafka消費(fèi)者會(huì)接收到插件發(fā)來(lái)的消息。參見(jiàn)kafka quick start
下面是kafka消費(fèi)者接收到的消息：
四、總結(jié)
上文簡(jiǎn)單實(shí)現(xiàn)了EMQ服務(wù)器端監(jiān)聽(tīng)客戶(hù)端連接/斷開(kāi)、話(huà)題訂閱/取消訂閱、消息發(fā)布過(guò)程中發(fā)消息到kafka中，采用的是同步發(fā)消息的方式(保證消息順序)，可自行調(diào)整消息格式或采用異步發(fā)消息等。
