fluentd

High kernel memory usage in Kubernetes Node

由 戏子无情 提交于 2020-12-12 06:44:05
问题 I am pretty desperate searching for a solution to this. I am running a Kubernetes Cluster (v1.16.7) on AWS. Node specs are : It is an Amazon EC2 t3.medium instance with 4GB RAM and AMI: k8s-1.11-debian-stretch-amd64-hvm-ebs-2018-08-17 with kernel: 4.9.0-7-amd64 My main problem is that I see increased memory usage in the kernel, which leads to faster memory starvation issues in my node. More specifically: free -m : total used free shared buff/cache available Mem: 3895 3470 130 3 294 204 Swap:

High kernel memory usage in Kubernetes Node

由 僤鯓⒐⒋嵵緔 提交于 2020-12-12 06:43:29
问题 I am pretty desperate searching for a solution to this. I am running a Kubernetes Cluster (v1.16.7) on AWS. Node specs are : It is an Amazon EC2 t3.medium instance with 4GB RAM and AMI: k8s-1.11-debian-stretch-amd64-hvm-ebs-2018-08-17 with kernel: 4.9.0-7-amd64 My main problem is that I see increased memory usage in the kernel, which leads to faster memory starvation issues in my node. More specifically: free -m : total used free shared buff/cache available Mem: 3895 3470 130 3 294 204 Swap:

Fluentd is not filtering as intended before writing to Elasticsearch

由 天大地大妈咪最大 提交于 2020-11-29 19:31:46
问题 Using: Elasticsearch 7.5.1. Fluentd 1.11.2 Fluent-plugin-elasticsearch 4.1.3 Springboot 2.3.3 I have a Springboot artifact with Logback configured with an appender that, in addition to the app STDOUT, sends logs to Fluentd: <appender name="FLUENT_TEXT" class="ch.qos.logback.more.appenders.DataFluentAppender"> <filter class="ch.qos.logback.classic.filter.ThresholdFilter"> <level>INFO</level> </filter> <tag>myapp</tag> <label>myservicename</label> <remoteHost>fluentdservicename</remoteHost>

Fluentd is not filtering as intended before writing to Elasticsearch

由 纵然是瞬间 提交于 2020-11-29 19:20:12
问题 Using: Elasticsearch 7.5.1. Fluentd 1.11.2 Fluent-plugin-elasticsearch 4.1.3 Springboot 2.3.3 I have a Springboot artifact with Logback configured with an appender that, in addition to the app STDOUT, sends logs to Fluentd: <appender name="FLUENT_TEXT" class="ch.qos.logback.more.appenders.DataFluentAppender"> <filter class="ch.qos.logback.classic.filter.ThresholdFilter"> <level>INFO</level> </filter> <tag>myapp</tag> <label>myservicename</label> <remoteHost>fluentdservicename</remoteHost>

Fluentd is not filtering as intended before writing to Elasticsearch

由 余生长醉 提交于 2020-11-29 19:20:03
问题 Using: Elasticsearch 7.5.1. Fluentd 1.11.2 Fluent-plugin-elasticsearch 4.1.3 Springboot 2.3.3 I have a Springboot artifact with Logback configured with an appender that, in addition to the app STDOUT, sends logs to Fluentd: <appender name="FLUENT_TEXT" class="ch.qos.logback.more.appenders.DataFluentAppender"> <filter class="ch.qos.logback.classic.filter.ThresholdFilter"> <level>INFO</level> </filter> <tag>myapp</tag> <label>myservicename</label> <remoteHost>fluentdservicename</remoteHost>

How to get ${kubernetes.namespace_name} for index_name in fluentd?

由 ◇◆丶佛笑我妖孽 提交于 2020-11-25 03:46:53
问题 Did anyone manage to get kubernetes.namespace_name as indexName? I tried this and its not working. index_name ${kubernetes.namespace_name}.%Y%m%d 回答1: Please follow below mentioned steps for complete installation. I have Added below fluentd.conf configmap file with below line. logstash_prefix clustername-${record['kubernetes']['namespace_name']} Fluentd-DaemonSet apiVersion: apps/v1 kind: DaemonSet metadata: name: fluentd namespace: logging labels: k8s-app: fluentd-logging version: v1 spec:

How to get ${kubernetes.namespace_name} for index_name in fluentd?

由 ╄→гoц情女王★ 提交于 2020-11-25 03:46:05
问题 Did anyone manage to get kubernetes.namespace_name as indexName? I tried this and its not working. index_name ${kubernetes.namespace_name}.%Y%m%d 回答1: Please follow below mentioned steps for complete installation. I have Added below fluentd.conf configmap file with below line. logstash_prefix clustername-${record['kubernetes']['namespace_name']} Fluentd-DaemonSet apiVersion: apps/v1 kind: DaemonSet metadata: name: fluentd namespace: logging labels: k8s-app: fluentd-logging version: v1 spec:

听说，你的Loki还是单体?(下篇)

由 自作多情 提交于 2020-10-28 08:18:28
正文共729字 预计阅读时间：2分钟😂 相信大家看过 《听说，你的Loki还是单体?(上篇)》 之后对Loki的分布式架构有了一定的认识 ，那么本篇主要就是对上篇内容的实践。小白主要提供 docker-compose 和 helm 两种方式将部署Loki集群的Demo版本。 在正式部署之前，我们还是先来看下Loki整体架构如下图： 我们本次部署清单里面主要涉及到的组件如下： 组件 副本数 说明 Cassandra 1 Loki Index存储 Minio 1 Loki S3存储 Consul 1 Loki 组件状态和哈希环存储 Redis 1 Loki 缓存 Gateway 2 Loki 网关 Distributor 3 Loki 组件 Ingester 3 Loki 组件 Querier 3 Loki 组件 Query-Frontend 2 Loki 组件 Table-Manager 1 Loki 组件 下载部署代码 $ git clone https://github.com/CloudXiaobai/loki-cluster-deploy.git 声明：以下部署均适用于demo环境，大家切勿直接用于生产环境 对于生产环境，请务必先解决Cassandra和Consul服务的高可用 通过docker-compose部署 启动服务 $ cd loki-cluster-deploy
订阅 fluentd