JunOS has strong flexibility on many features. One of them is logging. It support flexible logging options. This post summarizes some concepts I learned from my work and studying.
1.Understand Juniper SRX logging Type:
1.1 System Logging
Junos OS supports configuring and monitoring of system log messages (also called syslog messages). You can configure files to log system messages and also assign attributes, such as severity levels, to messages. Reboot requests are recorded to the system log files, which you can view with the show log command. SRX Series devices can send system log messages from the control plane (Routing Engine) to one or more destinations. Destinations can include local files on the SRX Series device (because the SRX Series device is a syslog server), remote syslog servers, user terminals, and the system console.
| [email protected]> show configuration system syslog archive size 750k files 2; user * { any emergency; } host 10.9.0.33 { any any; change-log none; interactive-commands none; explicit-priority; } host 10.9.8.52 { any any; source-address 10.9.8.20; } file messages { any critical; authorization info; explicit-priority; } file interactive-commands { interactive-commands error; } } |
1.2 Traffic Logging (Event Mode)
You can use traffic logs to track usage patterns or troubleshoot issues for a specific policy. You can configure a policy so that traffic information is logged when a session begins (session-init) and/or closes (session-close). To generate traffic logs for multiple policies, you must configure each policy to log traffic information. You also must configure syslog messages with a severity level of info or any. In the default configuration, these messages and all other logging messages are sent to a local log file named messages.
| [email protected]> show configuration system syslog archive size 750k files 2; user * { any emergency; } host 10.9.0.33 { any any; change-log none; interactive-commands none; explicit-priority; } host 10.9.8.52 { any any; source-address 10.9.8.20; } file messages { any critical; authorization info; explicit-priority; } file interactive-commands { interactive-commands error; } file traffic-create { any any; match RT_FLOW_SESSION_CREATE; structured-data; } file traffic-deny { any any; match RT_FLOW_SESSION_DENY; } file traffic-flow { user info; match RT_FLOW; archive size 1000k files 5 world-readable; structured-data; } |
| [email protected]> show log ? Possible completions: <[Enter]> Execute this command <filename> Name of log file IKELOG Size: 270913, Last changed: Feb 15 2015 PKITRACE Size: 138153, Last changed: Oct 02 02:22:41 PKITRACE.0.gz Size: 98723, Last changed: Sep 27 05:12:14 __jsrpd_commit_check__ Size: 6456, Last changed: Dec 21 2014 appidd Size: 0, Last changed: May 13 2014 authd_libstats Size: 0, Last changed: May 13 2014 authd_profilelib Size: 0, Last changed: May 13 2014 authd_sdb.log Size: 0, Last changed: May 13 2014 authlib_jdhcpd_trace.log Size: 0, Last changed: Jan 18 2015 bin_messages Size: 7, Last changed: May 13 2014 chassisd Size: 1173869, Last changed: Oct 01 22:54:45 cosd Size: 98079, Last changed: Sep 20 11:36:47 dcd Size: 251523, Last changed: Sep 21 14:25:57 default-log-messages Size: 612840, Last changed: Oct 02 02:19:39 default-log-messages.0.gz Size: 1027366, Last changed: Sep 20 18:45:01 default-log-messages.1.gz Size: 1323072, Last changed: Sep 20 18:30:00 dfwc Size: 0, Last changed: May 13 2014 e2e_events Size: 239, Last changed: Sep 20 11:45:31 eccd Size: 0, Last changed: May 13 2014 ext/ Last changed: May 13 2014 flowc/ Last changed: May 13 2014 fwauthd_chk_only Size: 298, Last changed: Dec 21 2014 ggsn/ Last changed: May 13 2014 gprsd_chk_only Size: 1335, Last changed: Dec 21 2014 gres-tp Size: 23569, Last changed: Sep 20 11:36:47 group_db.log Size: 0, Last changed: May 13 2014 helplog Size: 64, Last changed: Nov 17 2014 hostname-cached Size: 408, Last changed: Dec 21 2014 httpd.log Size: 1533, Last changed: Sep 20 18:36:17 idpd Size: 0, Last changed: May 13 2014 idpd.addver Size: 185, Last changed: Sep 20 19:15:01 idpd_err Size: 208962, Last changed: Sep 20 19:15:11 idpd_err.1 Size: 1048851, Last changed: Sep 20 18:55:14 ifstraced Size: 120, Last changed: Dec 21 2014 indb Size: 967833, Last changed: Dec 21 2014 install Size: 3927, Last changed: Dec 21 2014 interactive-commands Size: 82, Last changed: Sep 21 14:25:52 inventory Size: 17170, Last changed: Sep 20 11:45:34 ipfd Size: 97046, Last changed: Sep 28 10:01:08 ipfd_chk_only Size: 32, Last changed: Dec 21 2014 jdhcpd_era_discover.log Size: 8892, Last changed: Oct 01 20:07:42 jdhcpd_era_discover.log.0 Size: 43387, Last changed: Aug 13 23:11:34 jdhcpd_era_discover.log.1 Size: 25529, Last changed: Jun 19 14:49:47 jdhcpd_era_discover.log.2 Size: 422808, Last changed: Apr 17 16:00:00 jdhcpd_era_discover.log.3 Size: 0, Last changed: Jan 18 2015 jdhcpd_era_solicit.log Size: 595, Last changed: Sep 20 11:36:47 jdhcpd_era_solicit.log.0 Size: 595, Last changed: Jul 19 13:01:48 jdhcpd_era_solicit.log.1 Size: 595, Last changed: May 17 12:26:15 jdhcpd_era_solicit.log.2 Size: 595, Last changed: Jan 18 2015 jdhcpd_era_solicit.log.3 Size: 0, Last changed: Jan 18 2015 jdhcpd_sdb.log Size: 0, Last changed: Jan 18 2015 jsrpd Size: 841811, Last changed: Sep 28 10:01:17 kmd Size: 369441, Last changed: Sep 20 18:36:26 license Size: 0, Last changed: May 13 2014 license_subs_trace.log Size: 16976, Last changed: Sep 20 11:36:47 lsys-cpu-utilization-log Size: 0, Last changed: May 13 2014 mastership Size: 13036, Last changed: Sep 20 11:36:47 messages Size: 687915, Last changed: Oct 02 02:26:11 messages.0.gz Size: 38283, Last changed: Sep 26 05:45:00 messages.1.gz Size: 38105, Last changed: Sep 24 22:15:00 nsd_chk_only Size: 1021282, Last changed: Sep 29 18:26:35 nstraced Size: 58027, Last changed: Sep 20 11:43:30 nstraced_chk_only Size: 370, Last changed: Mar 18 2015 pcre_db.log Size: 0, Last changed: May 13 2014 pf Size: 1152, Last changed: Dec 21 2014 pfed Size: 0, Last changed: May 13 2014 pfed_jdhcpd_trace.log Size: 0, Last changed: Jan 18 2015 pgmd Size: 385, Last changed: Dec 21 2014 pkid Size: 828994, Last changed: Dec 21 2014 rexp_db.log Size: 0, Last changed: May 13 2014 rsi.1400.0118 Size: 4620227, Last changed: Jan 18 2015 rsi_2015_02_04 Size: 4762354, Last changed: Feb 04 2015 rtlogd Size: 3952, Last changed: Sep 29 18:26:56 smartd.trace Size: 133439, Last changed: Oct 01 23:51:05 traffic-create Size: 9307887, Last changed: Oct 02 02:26:11 traffic-create.0.gz Size: 593738, Last changed: Oct 02 02:15:00 traffic-create.1.gz Size: 679624, Last changed: Oct 02 02:00:00 traffic-deny Size: 733722, Last changed: Oct 02 02:26:11 traffic-deny.0.gz Size: 30893, Last changed: Oct 02 02:00:00 traffic-deny.1.gz Size: 29997, Last changed: Oct 02 01:30:00 traffic-flow Size: 14043535, Last changed: Oct 02 02:26:11 traffic-flow.0.gz Size: 1110300, Last changed: Oct 02 02:15:00 traffic-flow.1.gz Size: 1194867, Last changed: Oct 02 02:00:00 traffic-flow.2.gz Size: 1223703, Last changed: Oct 02 01:45:00 traffic-flow.3.gz Size: 1205868, Last changed: Oct 02 01:30:00 traffic-flow.4.gz Size: 1196097, Last changed: Oct 02 01:15:01 user Show recent user logins utmd-av Size: 960, Last changed: Sep 20 11:36:47 utmp Size: 0, Last changed: May 13 2014 | Pipe through a command |
1.3 Notes from Juniper KB:
| System Logging | Traffic Logging | |
|---|---|---|
| SRX Branch Devices SRX100 SRX110 SRX210 SRX220 SRX240 SRX550 SRX650 | KB16502 | KB16509 |
| SRX High-End Devices SRX1400 SRX3400 SRX3600 SRX5600 SRX5800 | KB16502 | KB16506 |
2. Understand Juniper SRX Logging Methods:
 |
| Control Plane and Data Plane |
2.1 Control Plane Logging
The control plane logs have to do with events triggered by daemons on the control plane. This includes messages about the underlying hardware (chassisd), general-purpose messages (messages), and various protocol daemons like IDPD, appidd, and so on. Control plane logging is on by default to log locally, but you can override this with your own logfiles, syslog hosts, and criteria for different log messages. All logs are stored in the /var/log directory on the control plane. The configuration has been described at section 1.1
Services on the control plane:
- Management Daemon (MGD): Provides the interface between the UI components and the backend configuration and is responsible for acting on the Junos configuration to the system itself.
- Routing Protocol Daemon (RPD) : All routing protocols including RIP, OSPF, IS-IS, BGP, PIM, IPv6 counterparts, and so on.
- User interfaces: Console, Telnet, SSH, J-Web, NetConf.
- Filesystem interfaces: FTP/SCP.
- Syslogd: Logging subsystem on the control plane, different than what is on the data plane. This generates the OS and application logs on the control plane.
- Networking services: DNS, DHCP, NTP, ICMP, ARP/ND, SNMP.
- Chassisd: Controls the hardware operations of the data plane and interfaces with the components to ensure they are active and operating properly.
- JSRPD: This is the high availability daemon that runs the HA functionality between two SRX chassis in an HA cluster.
2.2 Data Plane Logging
Data plane logs are primarily those generated by components that process traffic on the data plane. These include the firewall logs (RT_LOG, which stands for Real-Time Log because it is not stored on the data plane) from the flowd process, IPS logs, UTM logs, and logs from other security components like Screens. Data plane logging is off by default and must be configured. Typically, it is recommended that you send logs off the SRX to a syslog host due to the large volume of logs that can be generated from the data plane, particularly on high-end SRX platforms like the 5800. In fact, it can take an entire infrastructure of syslog servers to handle the large volume of syslog messages that the high-end SRX can generate per second. For this reason, there are two different mechanisms that we can use to log messages to the control plane, as discussed in the next section.
Services on the data plane:
- Intrusion Detection and Prevention Daemon (IDPD)
- IKED
- PKID
2.2.1 Event Mode
Event mode – control plane log processing – used on low end devices. Optionally even rate can be specified. Once event mode is enabled under “security” then the logging to local file can configure under “system syslog” as above at section 1.2. You also can configure that security traffic logs are handled through the eventd process and sent with system logs though control panel Routing Engine.
| [email protected]> show configuration security log { mode event; event-rate 1000; format sd-syslog; source-address 10.9.8.20; stream securitylog { format sd-syslog; category all; host { 10.9.8.52; } } stream LogCollector { host { 10.9.20.17; } } stream TO-10.9.20.33 { format sd-syslog; category all; host { 10.9.20.33; } } } |
2.2.2 Stream Mode
Stream mode – data plane logging – Normally used on high end SRX devcies but can be configured on any SRX devices. Under security the syslog parameters can be specified, e.g. syslog server, syslog format, facility.
Note: SRX can only log to the control plane (Event mode) or log out the data plane (Stream mode) at one time
Security logs such as traffic and IDP logs are able to be streamed through the traffic interface ports to a remote syslog server. SRX devices do not send streamed session logs to the Routing Engine (RE). Because system logging is performed on the RE, session or traffic logs cannot be written to the RE file system. Therefore, all traffic logging must be sent to a remote syslog server. Because fxp0 belongs to the RE, the remote syslog server must be reachable by an interface on an IOC. Traffic logging cannot be sent out through fxp0.
When the logging mode is set to stream, security traffic logs generated in the data plane are streamed out a revenue traffic port directly to a remote server. That also means your local log file will stop logging. Match condition configuration in System -> Syslog part does not work in Stream mode. Its as per design, the Routing engine is the one which puts the match condition and filters the log,
since when we use stream mode the traffic is streamed out of the data plane itself and doesn’t reach the RE the match condition dose not work when using stream mode and only works in event mode.
Basically, only thing works at System – Syslog section are those generated from control plane.
| [email protected]> show configuration security log { cache; mode stream; format sd-syslog; source-address 10.2.2.13; stream TO-10-0-0-4 { format sd-syslog; category all; host { 10.0.00.4; } } stream TO-10.4.20.33 { format sd-syslog; category all; host { 10.4.20.33; } } inactive: traceoptions { file jtac; flag all; } } |
Please keep one thing in mind. Maximum stream destination is three. If you are configuration more than three destination, you will get following error messages from CLI. If you are using JunOS Space, it wont allow you add more than three destination either.
| [email protected]> commit and-quit [edit security log] ‘stream’ number of elements exceeds limit of 3 error: commit failed: (number of elements exceeds limit) |
 |
| Control plane pushing configuration to data plane |
| [email protected]> show security log Security logging is disabled |
“show security log” will show you something about audit log but not policy logging after enabled cache in the security log section, else SRX will show you Security Log disabled.
After you enabled cache under security -> log configuration, as shown at the configuration of section 2.2.2, you will get output like below once you use command show security log:
| [email protected]> show security log Event time Message 2015-10-02 09:15:04 UTC UI_CMDLINE_READ_LINE: User ‘root’, command ‘xml-mode netconf need-trailer ‘ 2015-10-02 09:15:04 UTC UI_LOGOUT_EVENT: User ‘root’ logout 2015-10-02 09:15:04 UTC UI_LOGIN_EVENT: User ‘root’ login, class ‘super-user’ [55330], ssh-connection ‘10.4.20.21 7804 10.2.1.14 59097’, client-mode ‘cli’ 2015-10-02 09:15:04 UTC UI_CMDLINE_READ_LINE: User ‘root’, command ‘xml-mode netconf need-trailer ‘ |