Fluentd apache format [warn]: pattern not match:

前端 未结 2 1383
忘掉有多难
忘掉有多难 2021-01-23 01:50

In my /etc/fluent/fluent.conf


  @type tail
  format apache2
  path /var/log/apache2/other_vhosts_access.log
  tag apache2.access

<         


        
相关标签:
2条回答
  • 2021-01-23 02:13

    I have been facing warn: pattern not match in fluentd, and because of this my filter section was not working. And then I took this warning seriously and resolved this by creating a regex. So, my td-agent.conf is as follows:

     <source>
        @type tail
        format **/^([^ ]*) (?<host>[^ ]*) [^ ]* "(?<method>\S+) (?<path>[^ ]* +\S*)? (?<code>[^ ]*) (?<size>[^ ]*) (?: "(?<referer>[^\"]*)" "(?<agent>[^\"]*)")?/**
        path /var/log/apache2/access.log
        pos_file /var/log/td-agent/httpd.access.pos
        tag s3.apache.access
      </source>
      <filter **>
        @type grep
        <regexp>
          key path
          pattern \/aws\/project_launch\/view\/[\w\W]*
        </regexp>
      </filter>
      <match **>
        @type s3
        aws_key_id xxxxxx
        aws_sec_key xxxxxx
        s3_bucket bucketalifluentd
        s3_region eu-west-1
        path logs_viewshare/
        buffer_path /var/log/td-agent/buffer/s3
        time_slice_format %Y-%m-%d/%H
        time_slice_wait 2m
      </match>
    

    Please note that, the apache2 logs may be different for you, because you have made a different configuration in apache2.conf. You can use rubular

    for creating regex in ruby, because fluentd/td-agent has been written in ruby. After that you can view buffer that has collected your logs in directory /var/log/td-agent/buffer/s3.xxx

    0 讨论(0)
  • 2021-01-23 02:19

    It seems that tail plugin does not support the format for apache log format "vhost_combined" but "combined".
    How about changing the apache configuration file as follows:

    /etc/apache2/conf-available/other-vhosts-access-log.conf
    Before:CustomLog ${APACHE_LOG_DIR}/other_vhosts_access.log vhost_combined
    (change vhost_combined to combined)
    After:CustomLog ${APACHE_LOG_DIR}/other_vhosts_access.log combined

    0 讨论(0)
提交回复
热议问题