如何将kafka的数据导入到Elastic

想到有三种方法:

1. logstash

2. kafka-connect-elasticsearch

3. elasticsearch-river-kafka-1.2.1-plugin

方法一:简单,只需启动一个代理程序

方法二:与confluent绑定紧,有些复杂

方法三:代码很久没更新,后续支持比较差

 

logstash使用如下:

input {
kafka {
zk_connect => “kafka:2181”
group_id => “logstash”
topic_id => “apache_logs”
consumer_threads => 16
}
}
output {
elasticsearch {
document_id => “%{my_uuid}”
}
}

https://www.elastic.co/blog/just-enough-kafka-for-the-elastic-stack-part2