[TOC] Logstash 是一個具有實時流水線功能的開源數據收集引擎。Logstash 可以動態地統一來自不同來源的數據，并將數據規范化到您選擇的目的地。為各種高級下游分析和可視化用例清理和普及所有數據。 雖然 Logstash 最初推動了日志收集的創新，但它的功能遠遠超出了該用例。任何類型的事件都可以通過廣泛的輸入、過濾器和輸出插件進行豐富和轉換，許多本地編解碼器進一步簡化了攝取過程。Logstash 通過利用更大量和更多樣化的數據來加速您的洞察力。 ## 下載 ```shell [elk@elk02 ~]$ curl -O https://artifacts.elastic.co/downloads/logstash/logstash-7.17.2-linux-x86_64.tar.gz [elk@elk02 ~]$ sudo tar xf logstash-7.17.2-linux-x86_64.tar.gz -C /opt/ ``` ## 修改權限 ```shell [elk@elk02 ~]$ sudo chown -R elk.elk /opt/logstash-7.17.2/ [elk@elk02 ~]$ mkdir /opt/logstash-7.17.2/{logs,pid} ``` ## 配置項 ### 數據源filebeat ```shell [elk@elk02 logstash-7.17.2]$ cd /opt/logstash-7.17.2/ [elk@elk02 logstash-7.17.2]$ cp config/logstash-sample.conf config/logstash.conf [elk@elk02 logstash-7.17.2]$ cat config/logstash.conf input { beats { port => 5044 } } output { elasticsearch { hosts => ["192.168.31.29:9200", "192.168.31.193:9200", "192.168.31.120:9200"] index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" #user => "elastic" #password => "changeme" } } ``` ### 數據源kafka ```shell [elk@elk02 logstash-7.17.2]$ cat config/logstash.conf input { kafka { bootstrap_servers => "10.0.0.127:9092,10.0.0.128:9092,10.0.0.129:9092" group_id => "logstash" auto_offset_reset => "latest" consumer_threads => "5" topics => ["messages"] type => "logs" } } output { elasticsearch { hosts => ["http://10.0.0.127:9200", "http://10.0.0.128:9200", "http://10.0.0.129:9200"] index => "logstash-%{+YYYY.MM}" #user => "elastic" #password => "changeme" } } ``` ## 啟動 ```shell cd /opt/logstash-7.17.2 nohup ./bin/logstash -f config/logstash.conf &>> logs/logstash-server-`date "+%Y%m%d"`.log & echo $! > pid/logstash.pid ``` ## 停止 ```shell cat /opt/logstash-7.17.2/pid/logstash.pid | xargs -I {} kill {} ``` ## 參考文章 logstash官方文檔：https://www.elastic.co/guide/en/logstash/7.17/index.html