Use Prometheus “target relabeling” to extract cAdvisor's Docker image name without tag

Question

I use Prometheus, together with cAdvisor to monitor my environment.

Now, I tried to use Prometheus' "target relabeling", and create a label that its value is the Docker container's image name, without a tag. It is based on the originally scraped image label.

It doesn't work, for some reason, showing no errors when running on debug log level. I can see metrics scraped from cAdvisor (for example container_last_seen) but my newly created label isn't there.

My job configuration:

  - job_name: "cadvisor"
    scrape_interval: "5s"
    dns_sd_configs:
      - names: ['cadvisor.marathon.mesos']
    relabel_configs:
      - source_labels: ['image']
        # [REGISTRYHOST/][USERNAME/]NAME[:TAG]
        regex: '([^/]+/)?([^/]+/)?([^:]+)(:.+)?'
        target_label: 'image_tagless'
        replacement: '${1}${2}${3}'

My label - image_tagless - is missing from the scraped metrics.

Any help would be much appreciated.

Answer 1

The image label is not a target label, it's on the metrics themselves. Thus you should use metric_relabel_configs rather than relabel_configs

My blog on Life of a Label explains how this works.