[TOC] # **日志中心** ## 日志處理流程   ## log-spring-boot-starter基礎日志模塊封裝 log-spring-boot-starter的logback-spring.xml  此處修改 ``` <springProperty name="LOG_FILE" scope="context" source="logging.file" defaultValue="/logs/${APP_NAME}"/> ``` 將所有日志放在/logs下面,在部署微服務的機器中執行 ``` mkdir /logs chmod -R 777 /logs ``` 還有一種方式，不需要修改上訴logback.xml配置，將微服務的日志都建立軟連接，參考下面ln -s 建立軟連接方式， ``` mkdir /logs ln -s /app/ocp/user-center/logs/user-center/ /logs ln -s /app/ocp/eureka-server/logs/eureka-server/ /logs ln -s /app/ocp/api-gateway/logs/auth-gateway/ /logs ln -s /app/ocp/api-gateway/logs/api-gateway/ /logs ln -s /app/ocp/auth-server/logs/auth-server/ /logs ln -s /app/ocp/file-center/logs/file-center/ /logs ```  以上兩種方式目的是將所有的微服務日志都放在/logs中，方便filebeat抽取改目錄的日志 ## log-center 對應es索引 * 對應es的索引字段  * java對象對應es的mapping信息  * ServiceLogDao讀取Es數據到ServiceLogDocument中   ## 核心原理  ### Greenwich.SR6 版本es6 方式 此方式需要注意修改application.yml的配置，默認是es7配置 es6 application.yml需要按以下修改配置文件 ``` spring: #elasticsearch服務配置 data: elasticsearch: cluster-name: elasticsearch cluster-nodes: 47.99.88.28:9300 repositories: enabled: true properties: transport: tcp: connect_timeout: 120s ``` es6方式 type= doc  ### Hoxton.SR8 版本es7方式 ([es7搭建方式](http://www.hmoore.net/owenwangwen/open-capacity-platform/1656401)) 默認是es7配置，high levl rest 高并發集成spring data es方式 配置文件 ``` spring: #elasticsearch服務配置 elasticsearch: rest: uris: - http://192.168.11.130:9200 data: elasticsearch: client: reactive: endpoints : 192.168.11.130:9200 socket-timeout: 3000 connection-timeout: 3000 ``` es7方式 type= _doc  ## spring data es  * AbstractElasticsearchConfiguration：創建 * ElasticsearchRestTemplate。 * AbstractReactiveElasticsearchConfiguration：創建 * ReactiveElasticsearchTemplate * ElasticsearchRepositoryConfigExtension * ReactiveElasticsearchRepositoryConfigurationExtension * ElasticsearchCrudRepository：支持crud的抽象接口。 * ReactiveElasticsearchRepository：支持crud的reactive抽象接口。  ReactiveElasticsearchClient使用Elasticsearch core項目提供的請求/響應對象，調用直接在響應堆棧上操作，而不是使用異步線程池的方式進行響應。 這是高版本的Spring Data Elasticsearch默認支持的方式，使用方式如下： ``` elasticsearch: rest: uris: - http://192.168.11.130:9200 data: elasticsearch: client: reactive: endpoints : 192.168.11.130:9200 socket-timeout: 3000 connection-timeout: 3000 = ``` ## swagger訪問接口  啟動 log-center ，之前需要部署 Filebeat logstash elasticearch | 軟件 | 版本 | 備注 | | --- | --- |--- | | centos| 7.5 | | | JDK | 1.8 |on 47.99.88.28 | | elasticsearch| 6.5.4 |on 47.99.88.28| |elasticsearch-head|6.x|windows| | filebeat| 6.5.4|on 47.99.88.28 | | logstash|6.5.4 |on 47.99.88.28 | # elasticearch安裝 ## 創建目錄 ``` mkdir /app cd /app wget https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.5.4.tar.gz tar -zxvf elasticsearch-6.5.4.tar.gz useradd es cd elasticsearch-6.5.4 修改config/jvm.options為內存的一半大小 vi config/jvm.options -Xms512m -Xmx512m 修改 max file 和 max virtual memory 參數 用root 或 sudo 用戶 vi /etc/sysctl.conf 添加下面配置： vm.max_map_count=655360 并執行命令： sysctl -p ``` ## 修改/etc/security/limits.conf ``` grep -q "* - nofile" /etc/security/limits.conf || cat >> /etc/security/limits.conf << EOF ######################################## * soft nofile 65536 * hard nofile 65536 * soft nproc 4096 * hard nproc 4096 EOF ``` ## 修改elasticsearch.yml ``` vi /app/elasticsearch-6.5.4/config/elasticsearch.yml cluster.name: elasticsearch node.name: node-1 network.host: 0.0.0.0 http.port: 9200 node.max_local_storage_nodes: 2 http.cors.enabled: true http.cors.allow-origin: "*" ``` ## 賦權啟動 ``` chown -R es:es /app/elasticsearch-6.5.4/ su - es -c '/app/elasticsearch-6.5.4/bin/elasticsearch -d' ``` ## 查看進程 ``` jinfo -flags 2114 VM Flags: -XX:+AlwaysPreTouch -XX:CICompilerCount=2 -XX:CMSInitiatingOccupancyFraction=75 -XX:ErrorFile=logs/hs_err_pid%p.log -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:InitialHeapSize=536870912 -XX:MaxDirectMemorySize=268435456 -XX:MaxHeapSize=536870912 -XX:MaxNewSize=87228416 -XX:MaxTenuringThreshold=6 -XX:MinHeapDeltaBytes=196608 -XX:NewSize=87228416 -XX:NonNMethodCodeHeapSize=5825164 -XX:NonProfiledCodeHeapSize=122916538 -XX:OldSize=449642496 -XX:-OmitStackTraceInFastThrow -XX:ProfiledCodeHeapSize=122916538 -XX:-RequireSharedSpaces -XX:ReservedCodeCacheSize=251658240 -XX:+SegmentedCodeCache -XX:ThreadStackSize=1024 -XX:+UseCMSInitiatingOccupancyOnly -XX:+UseCompressedClassPointers -XX:+UseCompressedOops -XX:+UseConcMarkSweepGC -XX:-UseSharedSpaces ``` ## 查看jvm參數 ``` jinfo -flag MaxHeapSize 2114 -XX:MaxHeapSize=536870912 jinfo -flag NewSize 2114 -XX:NewSize=87228416 jinfo -flag ThreadStackSize 2114 -XX:ThreadStackSize=1024 jinfo -flag OldSize 2114 -XX:OldSize=449642496 ``` ## 動態修改jvm參數 ``` jinfo -flag +HeapDumpOnOutOfMemoryError 2114 jinfo -flag HeapDumpPath=/app/elasticsearch-6.5.4/dump 2114 ``` # logstash ## logstash 安裝配置 ``` cd /app wget https://artifacts.elastic.co/downloads/logstash/logstash-6.5.4.tar.gz tar -zxvf logstash-6.5.4.tar.gz cd logstash-6.5.4/ ls cd bin ``` ## logstash.conf如下 vi logstash.conf ``` input { beats { port => 5044 } } filter { if [fields][docType] == "sys-log" { grok { patterns_dir => ["/app/logstash-6.5.4/patterns"] match => { "message" => "\[%{NOTSPACE:appName}\:%{NOTSPACE:serverIp}\:%{NOTSPACE:serverPort}\] \[%{MYAPPNAME:contextTraceId},%{MYAPPNAME:currentTraceId}\] %{TIMESTAMP_ISO8601:logTime} %{LOGLEVEL:logLevel} %{WORD:pid} \[%{MYTHREADNAME:threadName}\] %{NOTSPACE:classname} %{GREEDYDATA:message}" } overwrite => ["message"] } date { match => ["logTime","yyyy-MM-dd HH:mm:ss.SSS"] } date { match => ["logTime","yyyy-MM-dd HH:mm:ss.SSS"] target => "timestamp" } mutate { remove_field => "logTime" remove_field => "@version" remove_field => "host" remove_field => "offset" } } } output { if [fields][docType] == "sys-log" { elasticsearch { hosts => ["127.0.0.1:9200"] manage_template => false index => "ocp-log-%{+YYYY.MM.dd}" document_type => "%{[@metadata][type]}" } } if [fields][docType] == "biz-log" { elasticsearch { hosts => ["127.0.0.1:9200"] manage_template => false index => "biz-log-%{+YYYY.MM.dd}" document_type => "%{[@metadata][type]}" } } } ``` ## 在Logstash中使用grok ~~~ mkdir -p /app/logstash-6.5.4/patterns cd /app/logstash-6.5.4/patterns vi java # user-center MYAPPNAME ([0-9a-zA-Z_-]*) MYTHREADNAME ([0-9a-zA-Z._-]|\(|\)|\s)* ~~~ ## 權限 ``` chmod -R 777 /app/logstash-6.5.4 ``` ## 啟動 ``` cd /app/logstash-6.5.4/bin nohup ./logstash -f logstash.conf >&/dev/null & ``` # filebeat ## 注意filebeat與logstash在同一臺服務器中 filebeat(收集、聚合) ->logstash(過濾結構化) -> ES  filebeat 抽取的是/logs/*/*.log的日志，可以建立軟連接，將不同模塊的日志都放在/logs下面 * 準備工作 ``` mkdir /logs ln -s /app/ocp/user-center/logs/user-center/ /logs ln -s /app/ocp/eureka-server/logs/eureka-server/ /logs ln -s /app/ocp/api-gateway/logs/auth-gateway/ /logs ln -s /app/ocp/auth-server/logs/auth-server/ /logs ln -s /app/ocp/file-center/logs/file-center/ /logs ```  * 下載filebeat ``` cd /app wget https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.5.4-linux-x86_64.tar.gz tar -zxvf filebeat-6.5.4-linux-x86_64.tar.gz cd /app/filebeat-6.5.4-linux-x86_64 ``` * 配置filebeat.yml配置 vi filebeat.yml ``` ###################### Filebeat Configuration Example ######################### # This file is an example configuration file highlighting only the most common # options. The filebeat.reference.yml file from the same directory contains all the # supported options with more comments. You can use it as a reference. # # You can find the full configuration reference here: # https://www.elastic.co/guide/en/beats/filebeat/index.html # For more available modules and options, please see the filebeat.reference.yml sample # configuration file. #=========================== Filebeat inputs ============================= filebeat.inputs: # Each - is an input. Most options can be set at the input level, so # you can use different inputs for various configurations. # Below are the input specific configurations. - type: log enabled: true paths: #- /var/log/*.log - /logs/*/*.log ##控制臺日志 exclude_lines: ['^DEBUG'] ##增加字段 fields: docType: sys-log project: open-capacity-platform #聚合日志 multiline: pattern: ^\[ negate: true match: after - type: log enabled: true paths: #- /var/log/*.log - /app/ocp/user-center/logs/biz/*.log ##業務日志 exclude_lines: ['^DEBUG'] ##增加字段 fields: docType: biz-log project: open-capacity-platform #聚合日志 #keys_under_root可以讓字段位于根節點，默認為false json.keys_under_root: true #對于同名的key，覆蓋原有key值 json.overwrite_keys: true #message_key是用來合并多行json日志使用的，如果配置該項還需要配置multiline的設置，后面會講 json.message_key: message #將解析錯誤的消息記錄儲存在error.message字段中 json.add_error_key: true # Exclude lines. A list of regular expressions to match. It drops the lines that are # matching any regular expression from the list. #exclude_lines: ['^DBG'] # Include lines. A list of regular expressions to match. It exports the lines that are # matching any regular expression from the list. #include_lines: ['^ERR', '^WARN'] # Exclude files. A list of regular expressions to match. Filebeat drops the files that # are matching any regular expression from the list. By default, no files are dropped. #exclude_files: ['.gz$'] # Optional additional fields. These fields can be freely picked # to add additional information to the crawled log files for filtering #fields: # level: debug # review: 1 ### Multiline options # Multiline can be used for log messages spanning multiple lines. This is common # for Java Stack Traces or C-Line Continuation # The regexp Pattern that has to be matched. The example pattern matches all lines starting with [ #multiline.pattern: ^\[ # Defines if the pattern set under pattern should be negated or not. Default is false. #multiline.negate: false # Match can be set to "after" or "before". It is used to define if lines should be append to a pattern # that was (not) matched before or after or as long as a pattern is not matched based on negate. # Note: After is the equivalent to previous and before is the equivalent to to next in Logstash #multiline.match: after #============================= Filebeat modules =============================== filebeat.config.modules: # Glob pattern for configuration loading path: ${path.config}/modules.d/*.yml # Set to true to enable config reloading reload.enabled: false # Period on which files under path should be checked for changes #reload.period: 10s #==================== Elasticsearch template setting ========================== setup.template.settings: index.number_of_shards: 3 setup.template.name: "filebeat" setup.template.pattern: "filebeat-*" #index.codec: best_compression #_source.enabled: false #================================ General ===================================== # The name of the shipper that publishes the network data. It can be used to group # all the transactions sent by a single shipper in the web interface. #name: # The tags of the shipper are included in their own field with each # transaction published. #tags: ["service-X", "web-tier"] # Optional fields that you can specify to add additional information to the # output. #fields: # env: staging #============================== Dashboards ===================================== # These settings control loading the sample dashboards to the Kibana index. Loading # the dashboards is disabled by default and can be enabled either by setting the # options here, or by using the `-setup` CLI flag or the `setup` command. #setup.dashboards.enabled: false # The URL from where to download the dashboards archive. By default this URL # has a value which is computed based on the Beat name and version. For released # versions, this URL points to the dashboard archive on the artifacts.elastic.co # website. #setup.dashboards.url: #============================== Kibana ===================================== # Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API. # This requires a Kibana endpoint configuration. setup.kibana: # Kibana Host # Scheme and port can be left out and will be set to the default (http and 5601) # In case you specify and additional path, the scheme is required: http://localhost:5601/path # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601 #host: "localhost:5601" # Kibana Space ID # ID of the Kibana Space into which the dashboards should be loaded. By default, # the Default Space will be used. #space.id: #============================= Elastic Cloud ================================== # These settings simplify using filebeat with the Elastic Cloud (https://cloud.elastic.co/). # The cloud.id setting overwrites the `output.elasticsearch.hosts` and # `setup.kibana.host` options. # You can find the `cloud.id` in the Elastic Cloud web UI. #cloud.id: # The cloud.auth setting overwrites the `output.elasticsearch.username` and # `output.elasticsearch.password` settings. The format is `<user>:<pass>`. #cloud.auth: #================================ Outputs ===================================== # Configure what output to use when sending the data collected by the beat. #-------------------------- Elasticsearch output ------------------------------ #output.elasticsearch: # Array of hosts to connect to. # hosts: ["192.168.28.130:9200"] # index: "filebeat-log" # Optional protocol and basic auth credentials. #protocol: "https" #username: "elastic" #password: "changeme" #----------------------------- Logstash output -------------------------------- output.logstash: # The Logstash hosts hosts: ["127.0.0.1:5044"] bulk_max_size: 2048 # Optional SSL. By default is off. # List of root certificates for HTTPS server verifications #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"] # Certificate for SSL client authentication #ssl.certificate: "/etc/pki/client/cert.pem" # Client Certificate Key #ssl.key: "/etc/pki/client/cert.key" #================================ Procesors ===================================== # Configure processors to enhance or manipulate events generated by the beat. processors: - add_host_metadata: ~ - add_cloud_metadata: ~ #================================ Logging ===================================== # Sets log level. The default log level is info. # Available log levels are: error, warning, info, debug #logging.level: debug # At debug level, you can selectively enable logging only for some components. # To enable all selectors use ["*"]. Examples of other selectors are "beat", # "publish", "service". #logging.selectors: ["*"] #============================== Xpack Monitoring =============================== # filebeat can export internal metrics to a central Elasticsearch monitoring # cluster. This requires xpack monitoring to be enabled in Elasticsearch. The # reporting is disabled by default. # Set to true to enable the monitoring reporter. #xpack.monitoring.enabled: false # Uncomment to send the metrics to Elasticsearch. Most settings from the # Elasticsearch output are accepted here as well. Any setting that is not set is # automatically inherited from the Elasticsearch output configuration, so if you # have the Elasticsearch output configured, you can simply uncomment the # following line. #xpack.monitoring.elasticsearch: ``` ## 注意filebeat與logstash如果不在一臺服務器中，上面的配置文件需要修改 ``` output.logstash: # The Logstash hosts ，假設在47.99.88.66部署了logstash hosts: ["47.99.88.66:5044"] bulk_max_size: 2048 ``` * 權限 ``` chmod -R 777 /app/filebeat-6.5.4 chmod go-w /app/filebeat-6.5.4/filebeat.yml ``` * 啟動 ``` nohup ./filebeat -e -c filebeat.yml >&/dev/null & ``` * lsof -p filebeat進程號查看啟動情況 ``` [root@iZbp178t3hp8rt4k9u953rZ filebeat-6.5.4]# lsof -p `ps | grep "filebeat" | grep -v "grep" |awk '{print $1}'` COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME filebeat 15802 root cwd DIR 253,1 4096 2359362 /app/filebeat-6.5.4 filebeat 15802 root rtd DIR 253,1 4096 2 / filebeat 15802 root txt REG 253,1 35930715 2359779 /app/filebeat-6.5.4/filebeat filebeat 15802 root mem REG 253,1 61624 1052387 /usr/lib64/libnss_files-2.17.so filebeat 15802 root mem REG 253,1 2156160 1052369 /usr/lib64/libc-2.17.so filebeat 15802 root mem REG 253,1 19288 1052375 /usr/lib64/libdl-2.17.so filebeat 15802 root mem REG 253,1 142232 1052395 /usr/lib64/libpthread-2.17.so filebeat 15802 root mem REG 253,1 163400 1052362 /usr/lib64/ld-2.17.so filebeat 15802 root 0w CHR 1,3 0t0 18 /dev/null filebeat 15802 root 1w CHR 1,3 0t0 18 /dev/null filebeat 15802 root 2w CHR 1,3 0t0 18 /dev/null filebeat 15802 root 3u IPv4 583959162 0t0 TCP iZbp178t3hp8rt4k9u953rZ:47346->izbp1jc2amxbl3xjw02s2xz:XmlIpcRegSvc (ESTABLISHED) filebeat 15802 root 4u a_inode 0,10 0 6091 [eventpoll] filebeat 15802 root 5r REG 253,1 627836 1710139 /app/openresty/nginx/logs/access.log filebeat 15802 root 6r REG 253,1 298509 1442290 /app/ocp/user-center/logs/user-center/user-center-info.log filebeat 15802 root 7r REG 253,1 956962 1442293 /app/ocp/api-gateway/logs/api-gateway/api-gateway-info.log filebeat 15802 root 8r REG 253,1 1347580 1442289 /app/ocp/auth-server/logs/auth-server/auth-server-info.log filebeat 15802 root 10u IPv4 583959848 0t0 TCP iZbp178t3hp8rt4k9u953rZ:47348->izbp1jc2amxbl3xjw02s2xz:XmlIpcRegSvc (ESTABLISHED) filebeat 15802 root 11r REG 253,1 50445 1442291 /app/ocp/file-center/logs/file-center/file-center-info.log ``` * 結構化日志數據為以下格式存在ES中 ``` { "contextTraceId":"上下文traceId", "currentTraceId":"當前traceId", "timestamp": "時間", "message": "具體日志信息", "threadName": "線程名", "serverPort": "服務端口", "serverIp": "服務ip", "logLevel": "日志級別", "appName": "工程名稱", "classname": "類名" } ``` * linux統計調用次數 ``` awk '{print $7} ' user-center-info.log | sort | uniq -c | sort -fr ```  # elasticseach-head ## 安裝 1. 安裝 ElasticSearch 6.x，訪問 http://47.99.88.28:9200/ 查看是否安裝成功。 2. 安裝 Node，使用?node -v?查看是否安裝成功。 3. 在 Node 中執行?npm install -g grunt-cli?安裝grunt，使用?grunt -version?查看是否安裝成功。 4. 安裝?elasticsearch-head。 * 訪問 https://github.com/mobz/elasticsearch-head 下載 head 插件（選擇 zip 壓縮包下載方式）。  * 修改?~\\elasticsearch-6.6.2\\elasticsearch-head-master\\Gruntfile.js，在對應的位置加上 hostname:'\*' 配置項。  * 在?~\\elasticsearch-6.6.2\\elasticsearch-head-master?下執行?npm install?開始安裝，完成后可執行?grunt server?或者?npm run start?運行 head 插件。  * 安裝成功，訪問 http://localhost:9100/。  5. 答疑 Issue - 在 head 中連接 ES 失敗。?  對于?Access-Control-Allow-Origin?的問題，可以在?ElasticSearch 6.x 的?~\\config\\elasticsearch.yml?文件的末尾加入以下代碼： ``` http.cors.enabled: true http.cors.allow-origin: "\*" node.master: true node.data: true ``` 配置更新后，重啟 ES 即可連接成功。 ## 使用  # 安裝grokdebug ## 安裝docker ``` [root@localhost ~]# systemctl stop firewalld [root@localhost ~]# systemctl disable firewalld [root@localhost ~]# sed -i 's/enforcing/disabled/' /etc/selinux/config [root@localhost ~]# cat /etc/selinux/config [root@localhost ~]# getenforce [root@localhost ~]# setenforce 0 [root@localhost ~]# getenforce [root@localhost ~]# sed -i 's/enforcing/disabled/' /etc/selinux/config [root@localhost ~]# swapoff -a [root@localhost ~]# yum install -y yum-utils device-mapper-persistent-data lvm2 [root@localhost ~]# wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo [root@localhost ~]# yum -y install docker-ce-18.06.1.ce-3.el7 [root@localhost ~]# systemctl enable docker && systemctl start docker Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service. [root@localhost ~]# docker --version Docker version 18.06.1-ce, build e68fc7a ``` ## 安裝docker compose ``` curl -L https://github.com/docker/compose/releases/download/1.24.1/docker-compose-`uname -s`-`uname -m` > /usr/local/bin/docker-compose chmod +x /usr/local/bin/docker-compose ``` ## 配置docker-compose.yml ``` [root@JD app]# cat docker-compose.yml version: "3" services: grok: image: epurs/grokdebugger ports: - "80:80" ``` ## 啟動 ``` [root@JD app]# docker-compose up -d ``` ## 查看 * [user-center:172.16.26.117:7000] [869f32593b6bbf6b,5aa8fbe5ba17b0d8] 2019-02-25 00:40:58.749 INFO 3417 [http-nio-7000-exec-197] com.open.capacity.log.aop.LogAnnotationAOP 開始請求，transid=869f32593b6bbf6b, url=com.open.capacity.user.controller.SysUserController/findByUsername , httpMethod=null, reqData=["admin"] * \[%{NOTSPACE:appName}\:%{NOTSPACE:serverIp}\:%{NOTSPACE:serverPort}\] \[%{MYAPPNAME:contextTraceId},%{MYAPPNAME:currentTraceId}\] %{TIMESTAMP_ISO8601:logTime} %{LOGLEVEL:logLevel} %{WORD:pid} \[%{MYTHREADNAME:threadName}\] %{NOTSPACE:classname} %{GREEDYDATA:message} * MYAPPNAME ([0-9a-zA-Z_-]*) * MYTHREADNAME ([0-9a-zA-Z._-]|\(|\)|\s)*  ## 解析后文件 ``` ~~~ { "appName": [ [ "user-center" ] ], "serverIp": [ [ "172.16.26.117" ] ], "serverPort": [ [ "7000" ] ], "contextTraceId": [ [ "869f32593b6bbf6b" ] ], "currentTraceId": [ [ "5aa8fbe5ba17b0d8" ] ], "logTime": [ [ "2019-02-25 00:40:58.749" ] ], "YEAR": [ [ "2019" ] ], "MONTHNUM": [ [ "02" ] ], "MONTHDAY": [ [ "25" ] ], "HOUR": [ [ "00", null ] ], "MINUTE": [ [ "40", null ] ], "SECOND": [ [ "58.749" ] ], "ISO8601_TIMEZONE": [ [ null ] ], "logLevel": [ [ "INFO" ] ], "pid": [ [ "3417" ] ], "threadName": [ [ "http-nio-7000-exec-197" ] ], "classname": [ [ "com.open.capacity.log.aop.LogAnnotationAOP" ] ], "message": [ [ "開始請求，transid=869f32593b6bbf6b, url=com.open.capacity.user.controller.SysUserController/findByUsername , httpMethod=null, reqData=["admin"] " ] ] } ~~~ ``` # 業務日志 業務日志采用結構化輸出 {"message":"tttt","transId":"46d803fd318f1dd3","token":"6d8ea03f-c82d-4e39-bdf2-11347d8f6be5","username":"admin","msg":"hello","error":null,"host":"130.75.131.208","appName":"user-center"} 這樣可以不再使用logstash的gork格式化日志。 * 代碼使用 BizLog.info("角色列表", LogEntry.builder().clazz(this.getClass().getName()).method("findRoles").msg("hello").path("/roles").build()); * 效果  ## filebeat 結構化日志es ``` #=========================== Filebeat inputs ============================= filebeat.inputs: # Each - is an input. Most options can be set at the input level, so # you can use different inputs for various configurations. # Below are the input specific configurations. - type: log enabled: true paths: #- /var/log/*.log - /app/ocp/user-center/logs/biz/*.log exclude_lines: ['^DEBUG'] ##增加字段 fields: docType: biz-log project: open-capacity-platform #聚合日志 #keys_under_root可以讓字段位于根節點，默認為false json.keys_under_root: true #對于同名的key，覆蓋原有key值 json.overwrite_keys: true #message_key是用來合并多行json日志使用的，如果配置該項還需要配置multiline的設置，后面會講 json.message_key: message #將解析錯誤的消息記錄儲存在error.message字段中 json.add_error_key: true #============================= Filebeat modules =============================== filebeat.config.modules: # Glob pattern for configuration loading path: ${path.config}/modules.d/*.yml # Set to true to enable config reloading reload.enabled: false # Period on which files under path should be checked for changes #reload.period: 10s #==================== Elasticsearch template setting ========================== setup.template.settings: index.number_of_shards: 3 setup.template.name: "filebeat" setup.template.pattern: "filebeat-*" #index.codec: best_compression #_source.enabled: false setup.kibana: # Kibana Host # Scheme and port can be left out and will be set to the default (http and 5601) # In case you specify and additional path, the scheme is required: http://localhost:5601/path # IPv6 addresses should always be defined as: https://[2001:db8::1]:5601 #host: "localhost:5601" # Kibana Space ID # ID of the Kibana Space into which the dashboards should be loaded. By default, # the Default Space will be used. #space.id: output.elasticsearch: enabled: true hosts: ["127.0.0.1:9200"] index: "biz-log-%{+yyyy-MM-dd}" #================================ Processors ===================================== # Configure processors to enhance or manipulate events generated by the beat. processors: #- add_host_metadata: ~ #- add_cloud_metadata: ~ - drop_fields: fields: ["beat.name", "beat.version", "host.architecture","host.architecture","host.name","beat.hostname","log.file.path"] ```  生產建議參考 [13.統一日志中心](11.%E7%BB%9F%E4%B8%80%E6%97%A5%E5%BF%97%E4%B8%AD%E5%BF%83.md)