## logstash工作原理  ## example 將apache的日志發送到elasticsearch ~~~ vim apache.conf ~~~ ~~~ input { file { path => "/etc/httpd/logs/*log" start_position => beginning ignore_older => 0 } } filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}"} } geoip { source => "clientip" } } output { elasticsearch { hosts => [ "localhost:9200" ] } } ~~~ #### 測試文件語法 ~~~ /opt/logstash/bin/logstash -f apache.conf --configtest ~~~ #### 執行文件 ~~~ /opt/logstash/bin/logstash -f apache.conf ~~~ #### 查看結果 ~~~ curl -XGET 'localhost:9200/logstash-2017.05.22/_search?pretty&q=response=200' ~~~ ## input插件 ### stdin插件 ~~~ input { stdin { } } output { stdout { codec => rubydebug } } ~~~ ### udp插件 collectd收集服務器信息，使用udp協議發送給logstash ~~~ input { udp { port => 1991 codec => collectd {} type => "collectd" } } output { stdout { codec => rubydebug } } ~~~ ### file插件 將指定的文件作為logstash的輸入 ~~~ input { file { path => "/path/to/file/*.log" start_position => beginning ignore_older => 0 } } ~~~ * 默認的file插件像tail -f一樣會從日志的最后讀取新的日志， `start_position`設置為`beginning`讓其從文件頭讀取。 * 忽略文件的時間 ``` Setting ignore_older to 0 disables file age checking so that the tutorial file is processed even though it’s older than a day ``` ### redis插件 ~~~ input { redis { port => "6379" host => "192.168.101.201" data_type => "list" type => "apachelog" key => "logstash-apachelog" } } ~~~ ## filter插件 對input的內容進行過濾，切割詞語 ### grok插件 ~~~ filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}"} } } ~~~ ### geoip插件 從input中獲得IP ~~~ filter { grok { match => { "message" => "%{COMBINEDAPACHELOG}"} overwrite => [ "message" ] } geoip { source => "clientip" } } ~~~ ## output插件 ### elasticsearch插件 將logstash加工過的內容發送到elasticsearch ~~~ output { elasticsearch { hosts => [ "localhost:9200" ] } } ~~~ ### redis插件 ~~~ output { redis { port => "6379" host => ["192.168.101.201"] data_type => "list" key => "logstash-%{type}" } } ~~~