Subset of tag to a second output

问题

I've got a service I'm trying to record logs from and send them to certain locations depending on what they refer to. Fluentd makes sense. Currently, I just have everything being chucked off to s3, and that works fine. However I now want a small subset (any line beginning with the phrase "ACTION:") to also be sent to a mongo database. I still want everything sent to s3.

I have this config file here, which of course isn't going to work

<source>
  @type  forward
  @id    input
  @label mainstream
  port   24224
</source>

<label @mainstream>

  <match foo.**>
    @type copy

    <store>
      @type s3

      ...
    </store>
    <store>
      @type rewrite_tag_filter
      rewriterule1 log "^ACTION:" foo.bar
    </store>

  </match>

  <match foo.bar>
    @type mongo

    ...
  </matc>
</label>

It's all going to be collected and stopped once we've got to the first match tag, the normal behaviour of the rewrite_tag_filter doesn't seem to pass on past the <store>.

Other things I've tried.

Copying the s3 store so it's used twice. However I don't want my s3 outputs to be split.
Forward to itself at a different port, run the source through a different label, however it wouldn't connect.

I'm sure what I'm asking isn't too insane. Has anyone done something like this before?

回答1:

Just need to be careful with what to retag. If you use something which will be matched by the above match, it won't move on, so it needs to be different. In this case, even though the tag was different, because I was starting it with foo, it wouldn't move on past the <match foo.**>.

<source>
  @type  forward
  @id    input
  @label mainstream
  port   24224
</source>

<label @mainstream>

  <match foo.**>
    @type copy

    <store>
      @type s3

      ...
    </store>
    <store>
      @type rewrite_tag_filter
      # Changed tag from foo.bar to this
      rewriterule1 log "^ACTION:" totally.different.tag.name
    </store>

  </match>


  # the new totally.different.tag.name won't get caught by the 
  # first <match foo.**> so it will actually reach this one
  <match totally.different.tag.name>
    @type mongo

    ...
  </match>
</label>

来源：https://stackoverflow.com/questions/38448233/subset-of-tag-to-a-second-output

标签

logging

fluentd