想到有三种方法:
1. logstash
2. kafka-connect-elasticsearch
3. elasticsearch-river-kafka-1.2.1-plugin
方法一:简单,只需启动一个代理程序
方法二:与confluent绑定紧,有些复杂
方法三:代码很久没更新,后续支持比较差
logstash使用如下:
input {
kafka {
zk_connect => “kafka:2181”
group_id => “logstash”
topic_id => “apache_logs”
consumer_threads => 16
}
}
output {
elasticsearch {
document_id => “%{my_uuid}”
}
}
https://www.elastic.co/blog/just-enough-kafka-for-the-elastic-stack-part2