基于Grafana实现的Loki日志聚合系统部署

相比较ELK或EFK更轻便，高效，类似于k3s之于k8s。

Loki操作简单，且资源效率高，节省成本，Grafana原生支持。

使用与 Prometheus 相同的标签记录流对日志进行索引和分组，这使得日志的扩展和操作效率更高。

Loki由3个部分组成：

1.  loki是主服务器，负责存储日志和处理查询。

2.  promtail是代理，负责收集日志并将其发送给 loki 。

3.  Grafana用于 UI 展示。

从官网看，最简单的部署方式是：docker-compose部署

wget https://raw.githubusercontent.com/grafana/loki/v1.5.0/production/docker-compose.yaml -O docker-compose.yaml
docker-compose -f docker-compose.yaml up -d

#通过docker-compose ps查看docker容器运行情况

但是要部署至k3s集群就需要转化成yaml文件，以支持k8s的kubectl命令部署，这里需要用到工具：kompose

#下载包
curl -L https://github.com/kubernetes/kompose/releases/download/v1.16.0/kompose-linux-amd64 -o kompose
chmod +x kompose

#在docker-compose.yaml文件目录下执行
./kompose convert
#根据yaml文件自动生成3个组件的deployment，service的yaml文件，可直接kubectl apply/create -f ..部署

这里记录生成的3个服务yaml：

1. loki-deployment.yaml  loki-service.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: ./kompose convert
    kompose.version: 1.16.0 (0c01309)
  creationTimestamp: null
  labels:
    io.kompose.service: loki
  name: loki
spec:
  replicas: 1
  selector:
    matchLabels:
      io.kompose.service: loki
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        io.kompose.service: loki
    spec:
      containers:
      - args:
        - -config.file=/etc/loki/local-config.yaml
        image: grafana/loki:1.5.0
        name: loki
        ports:
        - containerPort: 3100
        resources: {}
      restartPolicy: Always
status: {}
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    kompose.cmd: ./kompose convert
    kompose.version: 1.16.0 (0c01309)
  creationTimestamp: null
  labels:
    io.kompose.service: loki
  name: loki
spec:
  ports:
  - name: "3100"
    port: 3100
    targetPort: 3100
  selector:
    io.kompose.service: loki
status:
  loadBalancer: {}

2. promtail服务：promtail-deployment.yaml，promtail-claim0-persistentvolumeclaim.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: ./kompose convert
    kompose.version: 1.16.0 (0c01309)
  creationTimestamp: null
  labels:
    io.kompose.service: promtail
  name: promtail
spec:
  replicas: 1
  selector:
    matchLabels:
      io.kompose.service: promtail
  strategy:
    type: Recreate
  template:
    metadata:
      creationTimestamp: null
      labels:
        io.kompose.service: promtail
    spec:
      containers:
      - args:
        - -config.file=/etc/promtail/docker-config.yaml
        image: grafana/promtail:1.5.0
        name: promtail
        resources: {}
        volumeMounts:
        - mountPath: /var/log
          name: promtail-claim0
      restartPolicy: Always
      volumes:
      - name: promtail-claim0
        persistentVolumeClaim:
          claimName: promtail-claim0
status: {}
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  creationTimestamp: null
  labels:
    io.kompose.service: promtail-claim0
  name: promtail-claim0
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 100Mi
status: {}

3. grafana服务：grafana-deployment.yaml，grafana-service.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: ./kompose convert
    kompose.version: 1.16.0 (0c01309)
  creationTimestamp: null
  labels:
    io.kompose.service: grafana
  name: grafana
spec:
  replicas: 1
  selector:
    matchLabels:
      io.kompose.service: grafana
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        io.kompose.service: grafana
    spec:
      containers:
      - image: grafana/grafana:master
        name: grafana
        ports:
        - containerPort: 3000
        resources: {}
      restartPolicy: Always
status: {}
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    kompose.cmd: ./kompose convert
    kompose.version: 1.16.0 (0c01309)
  creationTimestamp: null
  labels:
    io.kompose.service: grafana
  name: grafana
spec:
  ports:
  - name: "3000"
    port: 3000
    targetPort: 3000
    nodePort: 30009
  selector:
    io.kompose.service: grafana
  type: NodePort
status:
  loadBalancer: {}

在k3s集群中，通过kubectl apply -f ..运行以上3个服务。

由于grafana-service配置的nodePort:30009，则通过http://masterip:30009即可访问grafana web，左侧的expore中添加loki存储库，配置url: http://loki:3100，即可。

经过实践，发现添加loki的sourcedata之后，lables not found。猜测主要原因是promtail采集的目录不对，因为他的原理是在每台node上监听：/var/log/containers目录下各个容器日志文件的输出。

而且，官方没有类似直接yaml文件部署的方案，而是采用helm方式安装。

 

来源：oschina

链接：https://my.oschina.net/beyondken/blog/4429294