nxlog

日志采集落地方案

由 眉间皱痕 提交于 2020-08-13 12:15:12
日志采集落地方案 问题背景 团队日志经过规范化后，接下来就需要一个采集方案。本文主要是分享我们团队的方案，供有相同需求的同行参考一下，希望对部分人有所帮助或者启发。本文是接着上一篇讲的，需要了解日志规范化相关内容的，请移步上一篇文章。 日志规范化落地方案 采集端-nxlog 虽然现在已经进入容器化时代，filebeat, fluentd在私有云环境应用比较多，但是我们当时还没有容器化，所以我们选择了nxlog，目前实践下来，nxlog出色的完成了这个任务。使用下来，总结nxlog优秀的特性如下： 高性能，低消耗，稳定 多平台支持（因为我们是C#，java混合环境） 强大灵活的脚本语言 缺陷（社区版）： 单条日志大小有限制 无法统计关键指标 总体来说够用。下面以我们自己的日志格式为例，给出对应的nxlog配置主体部分。 <Extension syslog> Module xm_syslog </Extension> <Extension json> Module xm_json </Extension> #======================================================================== #applog ##==========================================================

IIS服务器配置NXLog进行syslog转发

由 谁说胖子不能爱 提交于 2020-04-27 11:22:50
NXLog是个跨平台日志传输插件，支持linux、windows平台的大部分系统日志及常见的web日志，支持tcp、udp、http(s)等协议传输。 本文通过NXLog将IIS的日志以syslog的形式转发至日志审计服务器。 一、系统环境 操作系统：Window Server2012 r2 Enterprise IIS：7.0 NXLog：nxlog-ce-2.10.2150，下载地址： https://nxlog.co/products/nxlog-community-edition/download （Windows请选择msi文件下载） 二、NXLog的整体实现原理 NXLog整体采用插件架构模式，通过加载可调用的模块， 实现从各种不同的日志来源读取日志数据(Input)，对日志数据进行解析和转换(Prosessor)，最后进行输出(Output)。 三、配置方法 IIS的日志为文本类型的逐行文件，各字段使用“空格符”进行分割，而其中的日期和时间分属于两个不同的字段。因此IIS的日志可以通过Input的im_file模块进行读取。 在NXLog的安装路径C:\Program Files\nxlog（或C:\Program Files (x86)\nxlog）下找到conf\nxlog.conf，用文本编辑器进行编辑。 增加Input日志来源：

Using nxlog to ship logs in to logstash from Windows using om_ssl

由 折月煮酒 提交于 2019-12-18 18:05:15
问题 I have been looking at options to ship logs from Windows, I have already got logstash set up, and I currently ship logs from Linux (CentOS) servers to my ELK stack using the logstash-forwarder and ssl encryption. For compliance reasons encryption is pretty much essential in this environment. I was hoping to use logstash-forwarder in Windows as well, but after compiling with Go I ran in to issues shipping Event Logs, and I found some people saying that it wasn't possible because of file

NXlog ignores multiline tomcat stacktraces while sending to Papertrail

由 拥有回忆 提交于 2019-12-13 05:19:26
问题 I am able get the stacktrace[exceptions] from rsyslog (v-8.13.0) to a common nxlog server which sends the logs to a papertrail server. But the nxlog server ignores these multiline logs to papertrail. I know there is a module (xm_multiline) in nxlog which can parse this, but not sure on how to integrate this with my current nxlog configuration file: ######################################## # Global directives # ######################################## User nxlog Group nxlog LogFile /var/log

Nxlog im_dbi is not working

由 我是研究僧i 提交于 2019-12-13 01:26:12
问题 I am able to insert data into PostgreSQL using nxlog(om_dbi). But I am not able to select data(or fetch data) from PostgreSQL using nxlog. I tried many options nothing is working. And in nxlog document also for IM_DBI module description has only "FIXME" mentioned. Document Link: http://nxlog.org/documentation/nxlog-community-edition-reference-manual-v20928#im_dbi Please help me to solve this. Logs: <Input dbiin> Module im_dbi SavePos TRUE SQL SELECT * FROM NEW_TABLE Driver pgsql Option host

Message missing in NXLOG log shipping

由 蹲街弑〆低调 提交于 2019-12-12 01:53:22
问题 I have following SQL logs in ERRORLOG file, 2014-12-19 14:27:21.76 spid52 Starting up database 'MyDatabase'. 2014-12-19 14:27:22.06 spid52 Setting database option COMPATIBILITY_LEVEL to 110 for database 'MyDatabase'. 2014-12-19 14:27:22.06 spid52 Setting database option ANSI_NULL_DEFAULT to OFF for database 'MyDatabase'. 2014-12-19 14:27:22.06 spid52 Setting database option ANSI_NULLS to OFF for database 'MyDatabase'. 2014-12-19 14:27:22.09 spid52 Setting database option ANSI_PADDING to OFF

Parse sql log from log file using logstash

由 ≡放荡痞女 提交于 2019-12-11 17:41:07
问题 I want to read MSSQL logs from log file, Written the NXLOG code as following, <Input sql-logs> Module im_file File 'C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\Log\ERRORLOG*' ReadFromLast TRUE Exec if $raw_event =~ /^#/ drop(); \ else \ { \ sql->parse_csv(); \ #$EventTime = parsedate($date + " " + $time); \ to_json (); \ } </Input> Where module SQL is, <Extension sql> Module xm_csv Fields $date, $time, $hostname, $message FieldTypes string, string, string, string Delimiter

Using SSL to ship from NXlog to Logstash

由 99封情书 提交于 2019-12-02 17:07:37
问题 I have NXLog shipping my Windows Events to another Logstash machine working fine in just TCP. But I want to encrypt the traffic using a self signed certificate. I think I have a basic understanding of SSL but confused by the NXLog docs. The NXLog om_ssl docs shows: <Output sslout> Module om_ssl Host localhost Port 23456 CAFile %CERTDIR%/ca.pem CertFile %CERTDIR%/client-cert.pem CertKeyFile %CERTDIR%/client-key.pem KeyPass secret AllowUntrusted TRUE OutputType Binary </Output> Does the

Using SSL to ship from NXlog to Logstash

由 不羁的心 提交于 2019-12-02 08:12:57
I have NXLog shipping my Windows Events to another Logstash machine working fine in just TCP. But I want to encrypt the traffic using a self signed certificate. I think I have a basic understanding of SSL but confused by the NXLog docs. The NXLog om_ssl docs shows: <Output sslout> Module om_ssl Host localhost Port 23456 CAFile %CERTDIR%/ca.pem CertFile %CERTDIR%/client-cert.pem CertKeyFile %CERTDIR%/client-key.pem KeyPass secret AllowUntrusted TRUE OutputType Binary </Output> Does the CertKeyFile mean that the NXLog "client" need the Private key used to generate the CAFile? I thought the
订阅 nxlog