Java Log4j 2

Java Log4j 2 Log Management

You can send and manage your Java logs using Log4j 2. We’ll show you how to setup the Log4j2 LogManager to forward these to Syslog, Rsyslog will then forward them to Loggly.

We automatically parse out the timestamp, method, fully classified class name, thread, and log level from log4j. The syslog protocol does not support multiline events, so rsyslog will combine them into a single line escaped by octal characters. The advantage of using Rsyslog is that it can send TCP events without blocking your application, can optionally encrypt the data, and even queue data to add robustness to network failure. This guide assumes you use Java Log4j version 2 or higher, the default log configuration and directories. For alternatives, please see the Advanced Options section.

Java Log4j2 Log Management Setup

1. Configure Syslog Daemon

If you haven’t already, run our automatic Configure-Syslog script below to setup rsyslog. Alternatively, you can Manually Configure Rsyslog or Syslog-ng.

curl -O http://www.ziyi11.site/install/configure-linux.sh
sudo bash configure-linux.sh -a SUBDOMAIN -u USERNAME

Replace:

• SUBDOMAIN: your account subdomain that you created when you signed up for Loggly

• USERNAME: your Loggly username

2. Configure Syslog Daemon for UDP Input

Open rsyslog’s configuration file

sudo vim /etc/rsyslog.conf

Uncomment these lines to accept UDP messages on the default port 514.

$ModLoad imudp
$UDPServerRun 514

Restart the rsyslog service so the changes take effect

sudo service rsyslog restart

3. Add Log4j2

Add Log4j2 dependency from the Maven repository :

<dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-api</artifactId>
  <version>2.5</version>
</dependency>
<dependency>
  <groupId>org.apache.logging.log4j</groupId>
  <artifactId>log4j-core</artifactId>
  <version>2.5</version>
</dependency>

Then run maven to install

mvn clean install

Add the following configuration in log4j2.xml

The second field in the conversion pattern is the syslog appname, in this case it’s set to java.

<Configuration>
  <Appenders>
    <Socket name="Loggly" host="localhost" port="514" protocol="UDP">
      <PatternLayout>
        <pattern>${hostName} java %d{yyyy-MM-dd HH:mm:ss,SSS}{GMT} %p %t
          %c %M - %m%n</pattern>
      </PatternLayout>
    </Socket>
  </Appenders>
  <Loggers>
    <Root level="INFO">
      <AppenderRef ref="Loggly" />
    </Root>
  </Loggers>
</Configuration>

If you are using an HTTP appender, you do not need to add a host or appname.

<Appenders> 
	<Http name="loggly" url="https://logs-01.loggly.com/inputs/TOKEN/tag/java"> 
		<PatternLayout> 
			<pattern>${hostName} java %d{yyyy-MM-dd HH:mm:ss,SSS}{GMT} %p %t %c %M - %m%n</pattern> 
		</PatternLayout> 
	</Http> 
</Appenders>

Add Logging Code

import org.apache.logging.log4j.LogManager;
private final static Logger log = LogManager.getLogger();
logger.info("Hello World from Log4j2!");
logger.info("{"message" : "Hello World from Log4j2"}");

4. Verify Events

Search Loggly for events with the java log type over the past hour. It may take a few minutes to index the event. If it doesn’t work, see the troubleshooting section below.

logtype:java

5. Next Steps

• Troubleshooting with Java Logs – Use your logs to find the most common exceptions, trace transactions, debug memory issues, and more.

Advanced Java Log4j Log Options

• When we setup rsyslog to handle multiline events, they come in with octet values. i.e.

Hello

World

becomes

Hello#012World

So you can use this config to scrub the octet values as below:

if re_match($msg,'(#012)') 

then?

{

? ? set $!ext = re_extract($msg,'(#012)',0,1,"");

? ? set $!msg= replace($msg, $!ext, "n");

}

else?

? ? set $!msg = $msg;

• Java Log4j – You can also send log using Java Log4j
• File Monitoring – You can also configure Log4j with a FileAppender, then monitor that file using Rsyslog. This gives you a local backup but doesn’t work well with multiline stacktraces.

• CustomAppname– You can set custom appName in Log4j2 using following conversion pattern

  <PatternLayout>
    <pattern>${hostName} CustomAppname %d{yyyy-MM-dd HH:mm:ss,SSS}{GMT} %p %t
        %c %M - %m%n</pattern>
  </PatternLayout>
  

• Syslog4j – The Syslog4j appender can send events up to 64k in size, but it doesn’t seem to follow the layout pattern to enable parsing.

• logglylog4j – To send events directly to Loggly over the HTTP/S Event API, supports multiline with proper newline display

• Java Logback – For Logback or SLF4J logging

• Loggly Libraries Catalog – Additional libraries are added here

• Search or post your own log4j examples, configuration, log4j tutorials, or other questions in the community forum.

Java Log4j Troubleshooting

If you don’t see any data show up in the verification step, then check for these common problems.

Check Log4j2:

• Wait a few minutes in case indexing needs to catch up

• Check syslog log file

• Verify Log4j2 is working

• Run “sudo tcpdump -i lo -A udp and port 514” to verify UDP events are being sent to localhost

Check Your Syslog Daemon:

• Troubleshooting Rsyslog

Still Not Working?

• Search or post your own questions on Java log4j and Apache log4j in the community forum.