Support Troubleshooting

                    Troubleshooting Rsyslog

                    Troubleshooting Rsyslog

                    Use these tips to troubleshoot problems with Rsyslog. You can use our automated test, check the configuration, send sample data, and check transmission. Additionally, you can read the Rsyslog manual, try their support forum (they offer professional Rsyslog support), or check out our Rsyslog manual configuration docs.

                    Wait a Few Minutes

                    Wait a few minutes after sending an event to give it time to index and appear in the search results. It normally happens within seconds, but sometimes it can take longer.

                    Check Loggly Status

                    If Loggly isn’t seeing data check our status page to make sure we are indexing data and search is running. You should see green dots and “All Systems Operational”.

                    Automated Verification

                    Our configure-syslog script can send a test event to Loggly, and then verify if it’s received using the Loggly search API. You can overwrite your existing loggly configuration to make sure there are no errors and verify it again. It may take a few minutes to run.

                    curl -O
                    sudo bash -a SUBDOMAIN -u USERNAME


                    • SUBDOMAIN: your account subdomain that you created when you signed up for Loggly
                    • USERNAME: your Loggly username

                    Check Rsyslog Configuration

                    Make sure you restarted rsyslog so your changes take effect

                    sudo service rsyslog restart

                    Make sure rsyslog is running. If this command returns nothing than it’s not running.

                    ps -A | grep rsyslog

                    Check the rsyslog configuration. If there are no errors listed, then it’s ok.

                    rsyslogd -N1

                    Make sure you have Rsyslog version 5.8 or higher

                    rsyslogd -version

                    Check the Linux system log for rsyslog errors. You should see an event that it started and no errors. Some logs may also be in /var/log/syslog.

                    sudo cat /var/log/messages | grep rsyslog

                    Make sure Loggly is configured in your rsyslog configuration. There should be an endpoint for either in your main rsyslog.conf file or an include to the 22-loggly.conf file.

                    sudo vim /etc/rsyslog.d/22-loggly.conf

                    Check the permissions of rsyslog and the file you want to monitor to be sure it can read that file. You may need to use alter the privilege in the rsyslog.conf file:

                    $PrivDropToUser adm

                    Send Sample Data

                    Verify rsyslog is sending data to Loggly by making a test event. Then search for that event in Loggly by searching for “TroubleshootingTest” in the last hour.

                    logger TroubleshootingTest

                    Check the Linux system log to see if Rsyslog recorded the test event

                    sudo cat /var/log/messages | grep TroubleshootingTest

                    If you are sending repeated test messages, you should turn off repeated message reduction in the rsyslog configuration.

                    $RepeatedMsgReduction off

                    If you are filtering events out with a lower priority, you should send test events with a high enough priority.

                    logger -p local0.error "TroubleshootingTest"

                    Check Data Transmission

                    Use netstat to verify Rsyslog has an established connection to Loggly. Specifically, check that Loggly can make a connection through your firewall on the proper port. It’s 514 for syslog, 6514 for TLS syslog, 80 for HTTP, and 643 for HTTPS.

                    sudo netstat -taupn | grep syslog

                    Rsyslog Configuration
                    Use telnet to verify we can make an outbound connection to Loggly. For syslog we use port 514. If you can’t connect it might be a network or firewall issue.

                    telnet 514

                    Rsyslog Tutorial
                    Use tcpdump to verify data is being sent to Loggly. If you send your events in cleartext while tcpdump is running, you should be able to see them in the left hand column.

                    sudo tcpdump -A dst

                    Rsyslog Logging
                    If your application logs syslog to rsyslog, you can also test to see if messages making it to rsyslog over UDP to localhost.

                    sudo tcpdump -i lo -A udp and port 514

                    Check Log Rotation

                    Some older version of rsyslog may have trouble resuming after a log is rotated. If you have log rotation setup, follow these instructions to force rsyslog to pick up the new file.

                    Read More

                    • Rsyslog-users – Mailing list for rsyslog describing many common support issues

                    Still Not Working?

                    Please search our community forum for more Rsyslog configuration answers or post your own question.

                    Thanks for the feedback! We'll use it to improve our support documentation.