1.刪除flume1.6.0/lib里面的guava-.jar and jackson-core-.jar
2.拷貝es2.3.5/lib里es2.3.5.jar包，到flume1.6.0/lib
3.拷貝elasticsearch-sink2-1.0.jar，到flume1.6.0/lib
4.編輯flume1.6.0/conf/es.conf
5.啟動es
6.創建測試文件，并啟動flume
3.拷貝elasticsearch-sink2-1.0.jar，到flume1.6.0/lib
也可以build出來jar包
https://github.com/lucidfrontier45/ElasticsearchSink2
ElasticsearchSink2
This is Flume-NG Sink for Elasticsearch >= 2.0. I developed this because the official version does not support Elasticsearch >= 2.0 due to API changes. Hope Flume dev team will fix this soon.
Requirements
Flume-NG >= 1.6
Elasticsearch >= 2.0
Build
Build standard jar by the following command
$ ./gradlew build
Build fat jar which contains elasticsearch dependencies
$ ./gradlew assembly
Jar will be generated in build/libs
Usage
Append the built jar to Flume's classpath
remove guava-*.jar and jackson-core-*.jar in flume's default libs dir. They are outdated and newer version are included in Elasticsearch.
set sink type to com.frontier45.flume.sink.elasticsearch2.ElasticSearchSink in flume.conf
start flume agent
4.編輯flume1.6.0/conf/es.conf
agent.sources = tail
agent.channels = memoryChannel
agent.channels.memoryChannel.type = memory
agent.sources.tail.channels = memoryChannel
agent.sources.tail.type = exec
agent.sources.tail.command = tail -F /tmp/ttt
# Describe the sink ES
agent.sinks = elasticsearch
agent.sinks.elasticsearch.type = com.frontier45.flume.sink.elasticsearch2.ElasticSearchSink
agent.sinks.elasticsearch.hostNames = 127.0.0.1:9300
agent.sinks.elasticsearch.indexType = logs
agent.sinks.elasticsearch.indexName = flume
agent.sinks.elasticsearch.clusterName = elas
agent.sinks.elasticsearch.batchSize = 500
agent.sinks.elasticsearch.ttl = 5d
agent.sinks.elasticsearch.serializer = com.frontier45.flume.sink.elasticsearch2.ElasticSearchDynamicSerializer
agent.sinks.elasticsearch.indexNameBuilder = com.frontier45.flume.sink.elasticsearch2.TimeBasedIndexNameBuilder
agent.sinks.elasticsearch.channel = memoryChannel
5.啟動es
bin/elasticsearch
6.創建測試文件，并啟動flume
touch /tmp/ttt
echo "test" >> /tmp/ttt
bin/flume-ng agent -c . -f conf/es.conf -n agent -Dflume.root.logger=INFO,console
