官方說明
source runs a given Unix command on start-up and expects that process to continuously produce data on standard out (stderr is simply discarded, unless property logStdErr is set to true). If the process exits for any reason, the source also exits and will produce no further data. This means configurations such as cat [named pipe] or tail -F [file] are going to produce the desired results where as date will probably not - the former two commands produce streams of data where as the latter produces a single event and exits.
??Exec源在啟動時運行給定的Unix命令,并期望該進程在標準輸出上連續生成數據(除非將屬性logStdErr設置為true,否則將丟棄stderr)。
如果進程因任何原因退出,則源也會退出并且不會產生更多數據。
這意味著諸如cat [named pipe]或tail -F [file]之類的配置將產生所需的結果,而日期可能不會 - 前兩個命令產生數據流,而后者產生單個事件并退出。
整體思路:
exec source + memory channel + logger sink
編寫conf文件
??需要配置的有source的type、command、shell屬性,其他的可以直接使用默認值。
exec-memory-logger.conf
a1.sources = r1
a1.sinks = k1
a1.channels = c1
a1.sources.r1.type = exec
a1.sources.r1.command = tail -f /Users/david/Cores/apache-flume-1.6.0-cdh5.7.0-bin/conf/data.log
a1.sources.r1.shell = /bin/sh -c
a1.sources.r1.bind = localhost
#logger 控制臺
a1.sinks.k1.type = logger
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
啟動flume
??啟動命令
flume-ng agent \
--name a1 \
--conf $FLUME_HOME/conf \
--conf-file $FLUME_HOME/conf/exec-memory-logger.conf \
-Dflume.root.logger=INFO,console
測試
??配置文件指定了監聽的文件目錄,向文件中插入數據,查看flume控制臺的輸出。如果正常輸出,即為配置成功。