003、filebeat项目实战案例
|
Elk
|
23

1129 字
|
1.2 小时
本文最后更新于 66 天前，其中的信息可能已经过时，如有错误请发送邮件到wuxianglongblog@163.com

一.filebeat收集系统日志

1.系统日志有哪些?

    系统日志其实很宽泛，通常我们说的是messages,secure,cron,dmesg,ssh,boot等日志。

2.系统日志收集思路

    系统中有很多日志，逐个配置收集变得非常麻烦了。所以我们需要对这些日志进行统一，集中的管理。

    可以通过rsyslog将本地所有类型的日志都写入"/var/log/oldboy.log"文件中，然后使用filebeat对该文件进行收集即可。

    综上所述，系统收集日志架构图如下所示。

image-20210530090741140

3.安装rsyslog

[root@elk101.oldboyedu.com ~]# yum -y install rsyslog

4.配置rsyslog

[root@elk101.oldboyedu.com ~]# egrep -v "^#|^$" /etc/rsyslog.conf 

...

# 开启TCP协议的监听端口
$ModLoad imtcp
$InputTCPServerRun 514

...

# 配置收集日志的方式
# *.*       @IP:514  # 将本地所有日志通过网络发送给指定IP的远程服务器。
*.*         /var/log/oldboyedu.log  # 将本地所有日志聚合在同一个文件内，以便于我们使用filebeat进行日志收集。
[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# systemctl restart rsyslog  # 重启服务使得配置生效!

5.测试rsyslog服务是否生效

[root@elk101.oldboyedu.com ~]# logger "My name is Jason Yin."
[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# grep -i "jason" /var/log/oldboyedu.log 
May 30 09:19:32 elk101 root: My name is Jason Yin.
[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# su -l oldboy
上一次登录：六 5月 29 17:48:51 CST 2021pts/0 上
[oldboy@elk101.oldboyedu.com ~]$ 
[oldboy@elk101.oldboyedu.com ~]$ logger "my name is oldboy"
[oldboy@elk101.oldboyedu.com ~]$ 
[oldboy@elk101.oldboyedu.com ~]$ logout
[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# grep -i "name" /var/log/oldboyedu.log 
May 30 09:18:44 elk101 polkitd[561]: Unregistered Authentication Agent for unix-process:3238:216176 (system bus name :1.22, object path /org/freedesktop/PolicyKit1/AuthenticationAgent, locale zh_CN.UTF-8) (disconnected from bus)
May 30 09:19:32 elk101 root: My name is Jason Yin.
May 30 09:20:26 elk101 root: my name is oldboy
[root@elk101.oldboyedu.com ~]#

6.配置filebeat收集日志（该版本中包含所有日志级别信息）

[root@elk101.oldboyedu.com ~/conf/project]# vim rsyslog-to-es.yaml
[root@elk101.oldboyedu.com ~/conf/project]# 
[root@elk101.oldboyedu.com ~/conf/project]# cat rsyslog-to-es.yaml
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/oldboyedu.log

output.elasticsearch:
  hosts: ["elk101.oldboyedu.com:9200","elk102.oldboyedu.com:9200","elk103.oldboyedu.com:9200"]
[root@elk101.oldboyedu.com ~/conf/project]# 
[root@elk101.oldboyedu.com ~/conf/project]# 
[root@elk101.oldboyedu.com ~/conf/project]# filebeat -e -c rsyslog-to-es.yaml

7.再次配置filebeat收集日志（该版本中可过滤日志级别，记得删除上一个索引信息的配置哟）

[root@elk101.oldboyedu.com ~/conf/project]# vim rsyslog-to-es2.yaml 
[root@elk101.oldboyedu.com ~/conf/project]# 
[root@elk101.oldboyedu.com ~/conf/project]# cat rsyslog-to-es2.yaml 
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/oldboyedu.log
  # 仅包含，错误信息，警告信息和sshd的相关配置，其他的都会过滤掉
  include_lines: ["^ERR","^WARN","sshd"]

output.elasticsearch:
  hosts: ["elk101.oldboyedu.com:9200","elk102.oldboyedu.com:9200","elk103.oldboyedu.com:9200"]
[root@elk101.oldboyedu.com ~/conf/project]# 
[root@elk101.oldboyedu.com ~/conf/project]# filebeat -e -c rsyslog-to-es2.yaml 

温馨提示：
    进行本实验时，建议先删除filebeat的索引，避免对实验的影响。

8.参考以上步骤，在将另一个节点的日志收集到ES

    几乎不需要改动任何参数，只需重复上述步骤即可。

    温馨提示：
        由于filebeat会自带有标签，我们可用在kibana的WebUI中通过agent.hostname:"elk102.oldboyedu.com"来实现数据的过滤。当然，前提是要创建索引哈。

二.filebeat收集nginx日志

1.为什么要收集nginx日志

    我们需要获取用户的信息，比如:来源的IP是哪个地域，网站的PV,UV,状态码，访问时间等等。

    使用filebeat收集nginx日志的架构图如下所示。

1634091478570

2.安装nginx服务并创建测试数据

(1)安装nginx服务
    [root@elk101.oldboyedu.com ~]# yum -y install  epel-release
    [root@elk101.oldboyedu.com ~]#
    [root@elk101.oldboyedu.com ~]# yum -y install nginx

(2)配置nginx服务
    [root@elk101.oldboyedu.com ~]# vim /etc/nginx/conf.d/elk.oldboyedu.conf
    [root@elk101.oldboyedu.com ~]# 
    [root@elk101.oldboyedu.com ~]# cat /etc/nginx/conf.d/elk.oldboyedu.conf
    server {
        listen 80;

        server_name elk101.oldboyedu.com;

        root /oldboy/data/nginx/code;

        location / {
            index index.html;
        }
    }
    [root@elk101.oldboyedu.com ~]# 

(3)创建测试数据
    [root@elk101.oldboyedu.com ~]# mkdir -pv /oldboy/data/nginx/code
    mkdir: 已创建目录 "/oldboy/data/nginx"
    mkdir: 已创建目录 "/oldboy/data/nginx/code"
    [root@elk101.oldboyedu.com ~]# 
    [root@elk101.oldboyedu.com ~]# echo "<h1>https://www.cnblogs.com/yinzhengjie/</h1>" > /oldboy/data/nginx/code/index.html
    [root@elk101.oldboyedu.com ~]# 
    [root@elk101.oldboyedu.com ~]# cat /oldboy/data/nginx/code/index.html
    <h1>https://www.cnblogs.com/yinzhengjie/</h1>
    [root@elk101.oldboyedu.com ~]# 

(4)重启nginx服务
    [root@elk101.oldboyedu.com ~]# nginx -t
    nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
    nginx: configuration file /etc/nginx/nginx.conf test is successful
    [root@elk101.oldboyedu.com ~]# 

(5)浏览器访问nginx并查看Nginx日志信息
    [root@elk101.oldboyedu.com ~]# >/var/log/nginx/access.log 
    [root@elk101.oldboyedu.com ~]# 
    [root@elk101.oldboyedu.com ~]# tail -10f /var/log/nginx/access.log

image-20210531125859823

3.配置filebeat收集nginx日志

[root@elk101.oldboyedu.com ~/conf/project]# vim nginx_access-to-es.yaml 
[root@elk101.oldboyedu.com ~/conf/project]# 
[root@elk101.oldboyedu.com ~/conf/project]# cat nginx_access-to-es.yaml 
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/nginx/access.log

output.elasticsearch:
  index: "nginx-access-%{[agent.version]}-%{+yyyy.MM.dd}"
  hosts: ["elk101.oldboyedu.com:9200","elk102.oldboyedu.com:9200","elk103.oldboyedu.com:9200"]

setup.ilm.enabled: false
# 定义模板名称.
setup.template.name: "nginx"
# 定义模板的匹配索引名称.
setup.template.pattern: "nginx-*"
[root@elk101.oldboyedu.com ~/conf/project]# 
[root@elk101.oldboyedu.com ~/conf/project]# filebeat -e -c nginx_access-to-es.yaml &>/dev/null &

温馨提示:
    (1)我们可以使用"&>/dev/null &"选项将服务放在后台启动，但这样看不了日志了。生产环境中测试好了就可以这样用哟~
    (2)如果想要查看filebeat的日志，我们也可以使用"nohup"命令进行后台启动;

4.nginx的json格式配置

[root@elk101.oldboyedu.com ~]# vim /etc/nginx/nginx.conf

    ...

    # 将默认的日志格式注释
    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    # 自定义nginx的日志格式为json格式
    log_format oldboyedu_nginx_json '{"@timestamp":"$time_iso8601",'
                              '"host":"$server_addr",'
                              '"clientip":"$remote_addr",'
                              '"size":$body_bytes_sent,'
                              '"responsetime":$request_time,'
                              '"upstreamtime":"$upstream_response_time",'
                              '"upstreamhost":"$upstream_addr",'
                              '"http_host":"$host",'
                              '"uri":"$uri",'
                              '"domain":"$host",'
                              '"xff":"$http_x_forwarded_for",'
                              '"referer":"$http_referer",'
                              '"tcp_xff":"$proxy_protocol_addr",'
                              '"http_user_agent":"$http_user_agent",'
                              '"status":"$status"}';

    #access_log  /var/log/nginx/access.log  main;
    access_log  /var/log/nginx/access.log  oldboyedu_nginx_json;

    ...

[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# systemctl restart nginx
[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# tail -100f /var/log/nginx/access.log 

...

{"@timestamp":"2021-05-31T14:34:26+08:00","host":"172.200.3.101","clientip":"172.200.1.19","size":0,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"","http_host":"elk101.oldboyedu.com","uri":"/index.html","domain":"elk101.oldboyedu.com","xff":"-","referer":"-","tcp_xff":"","http_user_agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.128 Safari/537.36","status":"304"}

...

5.基于filebeat收集nginx的json格式日志

[root@elk101.oldboyedu.com ~/conf/project]# vim nginx_access_json-to-es.yaml 
[root@elk101.oldboyedu.com ~/conf/project]# 
[root@elk101.oldboyedu.com ~/conf/project]# cat nginx_access_json-to-es.yaml 
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/nginx/access.log
  # false会将json解析的格式存储至message，改为true则不存储至message
  json.keys_under_root: true
  # 覆盖默认的message字段，使用自定义json格式的key
  json.overwrite_keys: true

output.elasticsearch:
  index: "nginx-access-%{[agent.version]}-%{+yyyy.MM.dd}"
  hosts: ["elk101.oldboyedu.com:9200","elk102.oldboyedu.com:9200","elk103.oldboyedu.com:9200"]

setup.ilm.enabled: false
# 定义模板名称.
setup.template.name: "nginx"
# 定义模板的匹配索引名称.
setup.template.pattern: "nginx-*"
[root@elk101.oldboyedu.com ~/conf/project]# 
[root@elk101.oldboyedu.com ~/conf/project]# filebeat -e -c nginx_access_json-to-es.yaml

6.查看数据是否成功写入到ES集群

(1)基于postman查询日志:
    curl -X GET http://elk101.oldboyedu.com:9200/nginx-access-7.12.1-2021.05.31/_search
        {
            "_source": [
                "clientip",
                "http_user_agent",
                "status"
            ]
        }

(2)基于kibana出图展示
    1)X轴选择client.ip;
    2)Y轴选择count(计数);

7.filebeat收集nginx的错误日志

[root@elk101.oldboyedu.com ~/conf/project]# vim nginx_json.yaml 
[root@elk101.oldboyedu.com ~/conf/project]# 
[root@elk101.oldboyedu.com ~/conf/project]# cat nginx_json.yaml 
filebeat.inputs:
- type: log
  paths:
    - /var/log/nginx/access.log*
  # 如果设置为true则将message字段的JSON信息拆解,放入根路径下.如果设置为false,则放在根下的"json"对象中
  json.keys_under_root: true
  # 给输入的类型打标签
  tags: ["nginx-access-10.0.0.106"]

- type: log
  paths:
    - /var/log/nginx/error.log*
  # 给输入的类型打标签
  tags: ["nginx-error-10.0.0.106"]

output.elasticsearch:
  hosts: 
    - "http://10.0.0.106:9200"
    - "http://10.0.0.107:9200"
    - "http://10.0.0.108:9200"

  # index: "oldboyedu-linux77-nginx-%{+yyyy.MM.dd}"
  indices:
    - index: "oldboyedu-linux77-nginx-access-%{+yyyy.MM.dd}"
      when.contains:
        tags: "nginx-access-10.0.0.106"
    - index: "oldboyedu-linux77-nginx-error-%{+yyyy.MM.dd}"
      when.contains:
        tags: "nginx-error-10.0.0.106"

# 禁用索引生命周期并设置索引的模板!
setup.ilm.enabled: false
setup.template.name: "oldboyedu-linux77-nginx"
setup.template.pattern: "oldboyedu-linux77-nginx*"
setup.template.overwrite: true
setup.template.settings:
  index.number_of_shards: 3
  index.number_of_replicas: 2
[root@elk101.oldboyedu.com ~/conf/project]# 
[root@elk101.oldboyedu.com ~/conf/project]# 
[root@elk101.oldboyedu.com ~/conf/project]# filebeat -e -c nginx_json.yaml 

基于kibana查看错误日志:
    依次点击"管理",“索引模式”，"创建索引"，"Discover"，查看nginx索引的message错误日志数据。

三.filebeat收集nginx多虚拟主机日志

1.配置nginx的多虚拟主机

(1)配置blog.oldboyedu.com虚拟主机创建配置文件
cat > /etc/nginx/conf.d/blog-oldboyedu.conf <<EOF
    server {
        listen 80;

        server_name blog.oldboyedu.com;

        root /oldboy/data/nginx/code/blog;

        # 注意，一定要自定义访问日志的路径，否则都会写入到access.log中哟~
        access_log /var/log/nginx/blog.log oldboyedu_nginx_json;

        location / {
            index index.html;
        }
    }
EOF
nginx -t

(2)配置demo.oldboyedu.com虚拟主机创建配置文件
cat > /etc/nginx/conf.d/linux-oldboyedu.conf <<EOF
    server {
        listen 80;

        server_name linux.oldboyedu.com;

        root /oldboy/data/nginx/code/linux;

        # 注意，一定要自定义访问日志的路径，否则都会写入到access.log中哟~
        access_log /var/log/nginx/linux.log oldboyedu_nginx_json;

        location / {
            index index.html;
        }
    }
EOF
nginx -t

温馨提示:
    如果selinux和firewalld未关闭,可能会导致服务无法访问到,可能会一直报错"403".

2.创建测试数据

(1)创建测试数据
mkdir -pv /oldboy/data/nginx/code/{linux,blog}
echo "<h1>oldboyedu linux77 <font color="red" size="7">linux</font> </h1>" >  /oldboy/data/nginx/code/linux/index.html
echo "<h1>oldboyedu linux77 <font color="green" size="7">blog</font></h1>" >  /oldboy/data/nginx/code/blog/index.html

(2)重启nginx服务
systemctl restart nginx

(3)访问WebUI页面
    如下图所示.

1634097867547

3.配置主机名解析，重启nginx服务并访问虚拟主机观察日志是否生成

(1)修改主机解析
    [root@elk101.oldboyedu.com ~]# vim /etc/hosts
    [root@elk101.oldboyedu.com ~]# 
    [root@elk101.oldboyedu.com ~]# cat /etc/hosts
    127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
    ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6

    172.200.3.101 elk101.oldboyedu.com
    172.200.3.102 elk102.oldboyedu.com
    172.200.3.103 elk103.oldboyedu.com

    # 用于解析多虚拟主机测试
    172.200.3.101 demo.oldboyedu.com
    172.200.3.101 blog.oldboyedu.com
    [root@elk101.oldboyedu.com ~]# 

(2)重启nginx服务并严重服务
    [root@elk101.oldboyedu.com ~]# systemctl restart nginx
    [root@elk101.oldboyedu.com ~]# 
    [root@elk101.oldboyedu.com ~]# curl demo.oldboyedu.com
    <h1>demo</h1>
    [root@elk101.oldboyedu.com ~]# 
    [root@elk101.oldboyedu.com ~]# 
    [root@elk101.oldboyedu.com ~]# curl blog.oldboyedu.com
    <h1>blog</h1>
    [root@elk101.oldboyedu.com ~]# 

(3)严重JSON格式的日志文件是否生成。
[root@elk101.oldboyedu.com ~]# ll /var/log/nginx/
总用量 16
-rw-r--r-- 1 root root 1630 5月  31 22:25 access.log
-rw-r--r-- 1 root root  329 6月   1 20:37 blog.log
-rw-r--r-- 1 root root  329 6月   1 20:37 demo.log
-rw-r--r-- 1 root root  805 6月   1 20:12 error.log
[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# cat /var/log/nginx/blog.log 
{"@timestamp":"2021-06-01T20:37:25+08:00","host":"172.200.3.101","clientip":"172.200.3.101","size":14,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"blog.oldboyedu.com","uri":"/index.html","domain":"blog.oldboyedu.com","xff":"-","referer":"-","tcp_xff":"","http_user_agent":"curl/7.29.0","status":"200"}[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# cat /var/log/nginx/demo.log 
{"@timestamp":"2021-06-01T20:37:18+08:00","host":"172.200.3.101","clientip":"172.200.3.101","size":14,"responsetime":0.000,"upstreamtime":"-","upstreamhost":"-","http_host":"demo.oldboyedu.com","uri":"/index.html","domain":"demo.oldboyedu.com","xff":"-","referer":"-","tcp_xff":"","http_user_agent":"curl/7.29.0","status":"200"}[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]#

4.修改filebeat收集多虚拟主机的日志

(1)创建配置文件
cat > conf/output/es/09-nginx-to-es.yaml << EOF
filebeat.inputs:
- type: log
  paths:
    - /var/log/nginx/access.log*
  json.keys_under_root: true
  tags: ["nginx-access-10.0.0.106"]

- type: log
  paths:
    - /var/log/nginx/blog.log*
  json.keys_under_root: true
  tags: ["nginx-access-blog-10.0.0.106"]

- type: log
  paths:
    - /var/log/nginx/linux.log*
  json.keys_under_root: true
  tags: ["nginx-access-linux-10.0.0.106"]
- type: log
  paths:
    - /var/log/nginx/error.log*
  tags: ["nginx-error-10.0.0.106"]

output.elasticsearch:
  hosts: 
    - "http://10.0.0.106:9200"
    - "http://10.0.0.107:9200"
    - "http://10.0.0.108:9200"

  indices:
    - index: "oldboyedu-linux77-nginx-access-%{+yyyy.MM.dd}"
      when.contains:
        tags: "nginx-access-10.0.0.106"

    - index: "oldboyedu-linux77-nginx-access-blog-%{+yyyy.MM.dd}"
      when.contains:
        tags: "nginx-access-blog-10.0.0.106"

    - index: "oldboyedu-linux77-nginx-access-linux-%{+yyyy.MM.dd}"
      when.contains:
        tags: "nginx-access-linux-10.0.0.106"

    - index: "oldboyedu-linux77-nginx-error-%{+yyyy.MM.dd}"
      when.contains:
        tags: "nginx-error-10.0.0.106"

# 禁用索引生命周期并设置索引的模板!
setup.ilm.enabled: false
setup.template.name: "oldboyedu-linux77-nginx"
setup.template.pattern: "oldboyedu-linux77-nginx*"
setup.template.overwrite: true
setup.template.settings:
  index.number_of_shards: 3
  index.number_of_replicas: 2
EOF

(2)启动配置文件
./filebeat run -e -c  conf/output/es/09-nginx-to-es.yaml

四.filebeat收集tomcat访问日志

1.安装tomcat服务

[root@elk101.oldboyedu.com ~]# tar zxf apache-tomcat-10.0.6.tar.gz -C /oldboy/softwares/
[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# cd /oldboy/softwares/
[root@elk101.oldboyedu.com /oldboy/softwares]#
[root@elk101.oldboyedu.com /oldboy/softwares]# ln -sv apache-tomcat-10.0.6 tomcat
"tomcat" -> "apache-tomcat-10.0.6"
[root@elk101.oldboyedu.com /oldboy/softwares]# 
[root@elk101.oldboyedu.com ~]# vim /etc/profile.d/tomcat.sh
[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# cat /etc/profile.d/tomcat.sh
#!/bin/bash

TOMCAT_HOME=/oldboy/softwares/tomcat
PATH=$PATH:$TOMCAT_HOME/bin
[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# source /etc/profile.d/tomcat.sh
[root@elk101.oldboyedu.com ~]# 
[root@elk101.oldboyedu.com ~]# catalina.sh version
Using CATALINA_BASE:   /oldboy/softwares/tomcat
Using CATALINA_HOME:   /oldboy/softwares/tomcat
Using CATALINA_TMPDIR: /oldboy/softwares/tomcat/temp
Using JRE_HOME:        /oldboy/softwares/jdk
Using CLASSPATH:       /oldboy/softwares/tomcat/bin/bootstrap.jar:/oldboy/softwares/tomcat/bin/tomcat-juli.jar
Using CATALINA_OPTS:   
Server version: Apache Tomcat/10.0.6
Server built:   May 8 2021 15:24:15 UTC
Server number:  10.0.6.0
OS Name:        Linux
OS Version:     3.10.0-1160.el7.x86_64
Architecture:   amd64
JVM Version:    1.8.0_291-b10
JVM Vendor:     Oracle Corporation
[root@elk101.oldboyedu.com ~]# 

温馨提示:
    tomcat的下载地址: 
        https://tomcat.apache.org/

2.配置tomcat服务

[root@elk101.oldboyedu.com ~]# vim /oldboy/softwares/tomcat/conf/server.xml 

    ...

          <Host name="tomcat.oldboyedu.com"  appBase="webapps"
                unpackWARs="true" autoDeploy="true">

        <Valve className="org.apache.catalina.valves.AccessLogValve" directory="logs"
            prefix="tomcat.oldboyedu.com_access_log" suffix=".txt"
pattern="{"clientip":"%h","ClientUser":"%l","authenticated":"%u","AccessTime":"%t","request":"%r","status":"%s","SendBytes":"%b","Query?string":"%q","partner":"%{Referer}i","AgentVersion":"%{User-Agent}i"}"/>

          </Host>

    ...

温馨提示:
    仅需要修改主机名和对应的日志文件和JSON格式即可。

3.启动tomcat并测试访问

[root@elk101.oldboyedu.com ~]# catalina.sh start
Using CATALINA_BASE:   /oldboy/softwares/tomcat
Using CATALINA_HOME:   /oldboy/softwares/tomcat
Using CATALINA_TMPDIR: /oldboy/softwares/tomcat/temp
Using JRE_HOME:        /oldboy/softwares/jdk
Using CLASSPATH:       /oldboy/softwares/tomcat/bin/bootstrap.jar:/oldboy/softwares/tomcat/bin/tomcat-juli.jar
Using CATALINA_OPTS:   
Tomcat started.
[root@elk101.oldboyedu.com ~]# 

温馨提示:
    启动服务后，直接执行"curl -I tomcat.oldboyedu.com:8080"进行数据的查看。

image-20210602011918532

4.配置filebeat收集tomcat访问日志

cat > conf/output/es/11-tomcat-to-es.yaml << EOF
filebeat.inputs:
- type: log
  paths:
    - /oldboyedu/softwares/tomcat/logs/*.txt
  json.keys_under_root: true
  tags: ["tomcat-access-10.0.0.106"]

output.elasticsearch:
  hosts: 
    - "http://10.0.0.106:9200"
    - "http://10.0.0.107:9200"
    - "http://10.0.0.108:9200"

  indices:
    - index: "oldboyedu-linux77-tomcat-access-%{+yyyy.MM.dd}"
      when.contains:
        tags: "tomcat-access-10.0.0.106"

# 禁用索引生命周期并设置索引的模板!
setup.ilm.enabled: false
setup.template.name: "oldboyedu-linux77-tomcat"
setup.template.pattern: "oldboyedu-linux77-tomcat*"
setup.template.overwrite: true
setup.template.settings:
  index.number_of_shards: 3
  index.number_of_replicas: 2

EOF
./filebeat run -e -c conf/output/es/11-tomcat-to-es.yaml

温馨提示:
    查看数据是否写入成功。注意访问tomcat服务以生成日志。

五.收集tomcat的错误日志

1.制造tomcat错误信息

方法很多，比如将配置文件修改错误即可。

2.收集错误日志（故意抛出问题）

cat > conf/output/es/12-tomcat-to-es.yaml << EOF
filebeat.inputs:
- type: log
  paths:
    - /oldboyedu/softwares/tomcat/logs/*.txt
  json.keys_under_root: true
  tags: ["tomcat-access-10.0.0.106"]

- type: log
  paths:
    - /oldboyedu/softwares/tomcat/logs/catalina*
  tags: ["tomcat-error-10.0.0.106"]

output.elasticsearch:
  hosts: 
    - "http://10.0.0.106:9200"
    - "http://10.0.0.107:9200"
    - "http://10.0.0.108:9200"

  indices:
    - index: "oldboyedu-linux77-tomcat-access-%{+yyyy.MM.dd}"
      when.contains:
        tags: "tomcat-access-10.0.0.106"

    - index: "oldboyedu-linux77-tomcat-error-%{+yyyy.MM.dd}"
      when.contains:
        tags: "tomcat-error-10.0.0.106"

# 禁用索引生命周期并设置索引的模板!
setup.ilm.enabled: false
setup.template.name: "oldboyedu-linux77-tomcat"
setup.template.pattern: "oldboyedu-linux77-tomcat*"
setup.template.overwrite: true
setup.template.settings:
  index.number_of_shards: 3
  index.number_of_replicas: 2

EOF
./filebeat run -e -c conf/output/es/12-tomcat-to-es.yaml

温馨提示:
    如下图所示，按照本案例的方式进行收集，很明显是有问题的，因为Java的报错日志是多行哟~

image-20210602201806977

3.JAVA错误日志收集思路

(1)JAVA错误日志的特点:
    1)报错信息比较多;
    2)报错信息分多行，如下图所示，ES的报错和tomcat的报错还是有所区别的;

(2)收集JAVA日志的通用思路:
    1)tomcat正常日志以日期开头，而报错日志的错误都不是以日期开头的。所以我们匹配以"日期"开头的到下一个日期出现则为一个事件日志;
    2)ES正常日志是以[]开头，而报错日志中间的错误信息不是以[]开头，送一可以匹配以"["开头的行，一直到下一个"["开头的出现则为一个事件日志。

推荐阅读:
    https://www.elastic.co/guide/en/beats/filebeat/7.15/multiline-examples.html

image-20210602202728108

image-20210602203037387

4.filebeat收集错误日志实战案例

cat > conf/output/es/13-tomcat-to-es.yaml << EOF
filebeat.inputs:
- type: log
  paths:
    - /oldboyedu/softwares/tomcat/logs/*.txt
  json.keys_under_root: true
  tags: ["tomcat-access-10.0.0.106"]

- type: log
  paths:
    - /oldboyedu/softwares/tomcat/logs/catalina*
  tags: ["tomcat-error-10.0.0.106"]
  # 匹配JAVA的多行模式
  multiline.type: pattern
  # 匹配以2个数字开头的行
  multiline.pattern: '^\d{2}'
  # 以下2个参数可以参考官方的图解说明.
  multiline.negate: true
  multiline.match: after

output.elasticsearch:
  hosts: 
    - "http://10.0.0.106:9200"
    - "http://10.0.0.107:9200"
    - "http://10.0.0.108:9200"

  indices:
    - index: "oldboyedu-linux77-tomcat-access-%{+yyyy.MM.dd}"
      when.contains:
        tags: "tomcat-access-10.0.0.106"

    - index: "oldboyedu-linux77-tomcat-error-%{+yyyy.MM.dd}"
      when.contains:
        tags: "tomcat-error-10.0.0.106"

# 禁用索引生命周期并设置索引的模板!
setup.ilm.enabled: false
setup.template.name: "oldboyedu-linux77-tomcat"
setup.template.pattern: "oldboyedu-linux77-tomcat*"
setup.template.overwrite: true
setup.template.settings:
  index.number_of_shards: 3
  index.number_of_replicas: 2
EOF
./filebeat run -e -c conf/output/es/13-tomcat-to-es.yaml

image-20210602210125473

5.扩展知识-收集ES的错误日志

cat >  conf/output/es/14-tomcat-to-es.yaml << EOF
filebeat.inputs:
- type: log
  paths:
    - /oldboyedu/softwares/elasticsearch/logs/oldboyedu-linux77.log
  tags: ["es-10.0.0.106"]
  multiline.type: pattern
  multiline.pattern: '^\[\d{4}-\d{2}-\d{2}'
  multiline.negate: true
  multiline.match: after

output.elasticsearch:
  hosts: 
    - "http://10.0.0.106:9200"
    - "http://10.0.0.107:9200"
    - "http://10.0.0.108:9200"

  indices:
    - index: "oldboyedu-linux77-es-%{+yyyy.MM.dd}"
      when.contains:
        tags: "es-10.0.0.106"

# 禁用索引生命周期并设置索引的模板!
setup.ilm.enabled: false
setup.template.name: "oldboyedu-linux77-es"
setup.template.pattern: "oldboyedu-linux77-es*"
setup.template.overwrite: true
setup.template.settings:
  index.number_of_shards: 3
  index.number_of_replicas: 1
EOF
./filebeat run -e -c conf/output/es/14-tomcat-to-es.yaml
谨此笔记，记录过往。凭君阅览，如能收益，莫大奢望。
ELKfilebeat
暂无评论

发送评论 编辑评论 


				

			

						

				

					

						

							

								
							

							
						

					

				

				

					

						

							

								
							

							
						

					

				

				

					

						

							

								
							

							
							
													

					

				

			

			
			

				

					
				

			

				

											

							
							
						

																

							
							
						

																

							
							
						

										
					
											
						

	

		
|´・ω・)ノ
ヾ(≧∇≦*)ゝ
(☆ω☆)
（╯‵□′）╯︵┴─┴
￣﹃￣
(/ω＼)
∠( ᐛ 」∠)＿
(๑•̀ㅁ•́ฅ)
→_→
୧(๑•̀⌄•́๑)૭
٩(ˊᗜˋ*)و
(ノ°ο°)ノ
(´இ皿இ｀)
⌇●﹏●⌇
(ฅ´ω`ฅ)
(╯°A°)╯︵○○○
φ(￣∇￣o)
ヾ(´･ ･｀｡)ノ"
( ง ᵒ̌皿ᵒ̌)ง⁼³₌₃
(ó﹏ò｡)
Σ(っ °Д °;)っ
( ,,´･ω･)ﾉ"(´っω･｀｡)
╮(╯▽╰)╭
o(*////▽////*)q
＞﹏＜
( ๑´•ω•) "(ㆆᴗㆆ)
😂
😀
😅
😊
🙂
🙃
😌
😍
😘
😜
😝
😏
😒
🙄
😳
😡
😔
😫
😱
😭
💩
👻
🙌
🖕
👍
👫
👬
👭
🌚
🌝
🙈
💊
😶
🙏
🍦
🍉
😣
Source: github.com/k4yt3x/flowerhd
	

	

		
颜文字
Emoji
小恐龙
花!
	


									

			

			
			
		

	






上一篇
下一篇 

                    

                    
			        推荐文章
		            

		            

							
002、logstash的基本使用

							
							

							
001、部署logstash

							
							

							
002、配置filebeat实战案例

							
							

							
004、kafka集群的压力测试

							
							

							
004、kafka常用的脚本使用案例