Fluentd apache format [warn]: pattern not match:

Question

In my /etc/fluent/fluent.conf

  @type tail
  format apache2
  path /var/log/apache2/other_vhosts_access.log
  tag apache2.access

<         


        

Answer 1

I have been facing warn: pattern not match in fluentd, and because of this my filter section was not working. And then I took this warning seriously and resolved this by creating a regex. So, my td-agent.conf is as follows:

 
    @type tail
    format **/^([^ ]*) (?[^ ]*) [^ ]* "(?\S+) (?[^ ]* +\S*)? (?[^ ]*) (?[^ ]*) (?: "(?[^\"]*)" "(?[^\"]*)")?/**
    path /var/log/apache2/access.log
    pos_file /var/log/td-agent/httpd.access.pos
    tag s3.apache.access
  
  
    @type grep
    
      key path
      pattern \/aws\/project_launch\/view\/[\w\W]*
    
  
  
    @type s3
    aws_key_id xxxxxx
    aws_sec_key xxxxxx
    s3_bucket bucketalifluentd
    s3_region eu-west-1
    path logs_viewshare/
    buffer_path /var/log/td-agent/buffer/s3
    time_slice_format %Y-%m-%d/%H
    time_slice_wait 2m
  

Please note that, the apache2 logs may be different for you, because you have made a different configuration in apache2.conf. You can use rubular

for creating regex in ruby, because fluentd/td-agent has been written in ruby. After that you can view buffer that has collected your logs in directory /var/log/td-agent/buffer/s3.xxx