How to start the cloudwatch agent in container?

Question

From the docker hub there is an image which is maintained by amazon.

Any one know how to configure and start the container as I cannot find any documen

Answer 1

try this dockerfile:

FROM amazonlinux:2.0.20190508
RUN yum -y install https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm
COPY agent.json  /opt/aws/amazon-cloudwatch-agent/bin/default_linux_config.json
ENV RUN_IN_CONTAINER=True
ENTRYPOINT ["/opt/aws/amazon-cloudwatch-agent/bin/start-amazon-cloudwatch-agent"]

json example:

  {
      "agent": {
        "metrics_collection_interval": 10,
        "logfile": "/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log"
      },
      "metrics": {
        "metrics_collected": {
          "cpu": {
            "resources": [
              "*"
            ],
            "measurement": [
              {"name": "cpu_usage_idle", "rename": "CPU_USAGE_IDLE", "unit": "Percent"},
              {"name": "cpu_usage_nice", "unit": "Percent"},
              "cpu_usage_guest"
            ],
            "totalcpu": false,
            "metrics_collection_interval": 10,
            "append_dimensions": {
              "customized_dimension_key_1": "customized_dimension_value_1",
              "customized_dimension_key_2": "customized_dimension_value_2"
            }
          },
          "disk": {
            "resources": [
              "/",
   .
   .
   .
   .
}

Answer 2

Here is how I got it working in our Docker containers without systemctl or System V init.

Answer 3

This is from official Documentation:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -m ec2 -c file:configuration-file-path -s

here the Docs:

https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/install-CloudWatch-Agent-commandline-fleet.html#start-CloudWatch-Agent-EC2-commands-fleet

Installation path may be different, but that is how the agent is started as per docs.

Answer 4

I got this working! I was having the same issue with you when you see Reading json config file path: /opt/aws/amazon-cloudwatch-agent/bin/default_linux_config.json ... Cannot access /etc/cwagentconfig: lstat /etc/cwagentconfig: no such file or directoryValid Json input schema.

What you need to do is put your config file in /etc/cwagentconfig. A functioning dockerfile:

FROM amazon/cloudwatch-agent:1.230621.0
COPY config.json /etc/cwagentconfig

Where config.json is some cloudwatch agent configuration, such as given by LinPy's answer.

You can ignore the warning about /opt/aws/amazon-cloudwatch-agent/bin/default_linux_config.json, or you can also COPY the config.json file to that location in the dockerfile as well.

I will also share how I found this answer:

I needed this run in ECS as a sidecar, and I could only find docs on how to run it in kubernetes. Following this documentation: https://docs.aws.amazon.com/en_pv/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-StatsD.html I decided to download all the example k8s manifests, when I saw this one:

apiVersion: v1
kind: Pod
metadata:
  namespace: default
  name: amazonlinux
spec:
  containers:
    - name: amazonlinux
      image: amazonlinux
      command: ["/bin/sh"]
      args: ["-c", "sleep 300"]
    - name: cloudwatch-agent
      image: amazon/cloudwatch-agent
      imagePullPolicy: Always
      resources:
        limits:
          cpu:  200m
          memory: 100Mi
        requests:
          cpu: 200m
          memory: 100Mi
      volumeMounts:
        - name: cwagentconfig
          mountPath: /etc/cwagentconfig
  volumes:
    - name: cwagentconfig
      configMap:
        name: cwagentstatsdconfig
  terminationGracePeriodSeconds: 60

So I saw that the volume mount cwagentconfig mounts to /etc/cwagentconfig and that's from the cwagentstatsdconfig configmap, and that's just the json file.

Answer 5

You just to run the container with log-opt, as the log agent is the main process of the container.

docker run --log-driver=awslogs --log-opt awslogs-region=us-west-2 --log-opt awslogs-group=myLogGroup amazon/cloudwatch-agent

You can find more details here and here.

I do not know why you need an agent in a container, but the best practice is to send each container log directly to cloud watch using aws log driver.

Btw this is entrypoint of the container.

  "Entrypoint": [
         "/opt/aws/amazon-cloudwatch-agent/bin/start-amazon-cloudwatch-agent"
 ],

All you need to call

/opt/aws/amazon-cloudwatch-agent/bin/start-amazon-cloudwatch-agent