Enhanced Logging With rsyslog On Debian Etch And phpLogcon For Viewing

Well everybody knows the issues of reading and searching in log files. If you have more than one machine it even gets worse. This tutorial describes how to install and configure rsyslog on Debian Etch, but it can be adapted to other distributions.

Exerpt from rsyslog site:

"Rsyslog, the enhanced syslogd for Linux and Unix.

Rsyslog is an enhanced multi-threaded syslogd supporting, among others, MySQL, syslog/tcp, RFC 3195, permitted sender lists, filtering on any message part, and fine grain output format control. It is quite compatible to stock sysklogd and can be used as a drop-in replacement. Its advanced features make it suitable for enterprise-class, encryption protected syslog relay chains while at the same time being very easy to setup for the novice user. An optional web interface - phpLogCon - can be used to visualize all data online."

In this tutorial we will be building rsyslog from source and write the necessary config files.


Step 1: See That The Necessary Tools Are Installed

apt-get install binutils cpp fetchmail flex gcc libarchive-zip-perl libc6-dev libcompress-zlib-perl libdb4.3-dev libpcre3 libpopt-dev linux-kernel-headers lynx m4 make ncftp nmap openssl perl perl-modules unzip zip zlib1g-dev autoconf automake1.9 libtool bison autotools-dev g++ mysql-server mysql-client libmysqlclient15-dev

Next set a password for your mysql root user:

mysqladmin -u root password your_mysqlroot_password 

Now we can create the rsyslog db:

mysqladmin -u root -p create rsyslog

Next we start the mysql command shell and create the rsyslog user:

mysql -u root -p
GRANT SELECT, INSERT, UPDATE, DELETE ON rsyslog.* TO 'rsyslog_user'@'localhost' IDENTIFIED BY 'rsyslog_user_password';
GRANT SELECT, INSERT, UPDATE, DELETE ON mail.* TO 'rsyslog_user'@'localhost.localdomain' IDENTIFIED BY 'rsyslog_user_password';

As you might have noticed we do not import any tables into the db. phpLogCon will do that for us. 


Step 2: Get The rsyslog Source And Build rsyslog

Get the source:

cd /tmp
wget http://www.rsyslog.com/Downloads-req-getit-lid-58.phtml 

Now let's build and install rsyslog:

tar xvzf rsyslog-1.9.6.tar.gz
cd rsyslog-1.9.6
make install 

Since rsyslog has no configuration examples (and files) I'll provide the examples here. All of these are just default configurations, use the documentation provided by rsyslog to customize the configuration to your specific needs.

First the rsyslog configuration file /etc/rsyslog.conf

vi /etc/rsyslog.conf

Just copy and paste the contents below:

Do not forget to change the line:

*.*       >,rsyslog,rsyslog_user,rsyslog_user_password

with the username and password you specified above.

#  /etc/rsyslog.conf    Configuration file for rsyslogd.
#                       For more information see
#                       /usr/share/doc/rsyslog/html/rsyslog_conf.html
# First some standard logfiles.  Log by facility.
$ModLoad MySQL
*.*       >,rsyslog,rsyslog_user,rsyslog_user_password
auth,authpriv.*                 /var/log/auth.log
*.*;auth,authpriv.none          -/var/log/syslog
#cron.*                         /var/log/cron.log
daemon.*                        -/var/log/daemon.log
kern.*                          -/var/log/kern.log
lpr.*                           -/var/log/lpr.log
mail.*                          -/var/log/mail.log
user.*                          -/var/log/user.log
# Logging for the mail system.  Split it up so that
# it is easy to write scripts to parse these files.
mail.info                       -/var/log/mail.info
mail.warn                       -/var/log/mail.warn
mail.err                        /var/log/mail.err
# Logging for INN news system
news.crit                       /var/log/news/news.crit
news.err                        /var/log/news/news.err
news.notice                     -/var/log/news/news.notice
# Some `catch-all' logfiles.
        news.none;mail.none     -/var/log/debug
        mail,news.none          -/var/log/messages
# Emergencies are sent to everybody logged in.
*.emerg                         *
# I like to have messages displayed on the console, but only on a virtual
# console I usually leave idle.
#       news.=crit;news.=err;news.=notice;\
#       *.=debug;*.=info;\
#       *.=notice;*.=warn       /dev/tty8
# The named pipe /dev/xconsole is for the `xconsole' utility.  To use it,
# you must invoke `xconsole' with the `-file' option:
#    $ xconsole -file /dev/xconsole [...]
# NOTE: adjust the list below, or you'll go crazy if you have a reasonably
#      busy site..
        *.=notice;*.=warn       |/dev/xconsole
# Include all config files in /etc/rsyslog.d/
$IncludeConfig /etc/rsyslog.d/

Next is the startup script /etc/init.d/rsyslog

vi /etc/init.d/rsyslog

Just copy and paste the contents below:

#! /bin/sh
# Provides:          syslog
# Required-Start:    $local_fs $time
# Required-Stop:     $local_fs $time
# Should-Start:      $network
# Should-Stop:       $network
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: enhanced syslogd
# Description:       Rsyslog is an enhanced multi-threaded syslogd.
#                    It is quite compatible to stock sysklogd and can be
#                    used as a drop-in replacement.
# Author: Michael Biebl <[email protected]>
# Do NOT "set -e"
# PATH should only include /usr/* if it runs after the mountnfs.sh script
DESC="enhanced syslogd"
# Exit if the package is not installed
[ -x "$RSYSLOGD_BIN" ] || exit 0
[ -x "$RKLOGD_BIN" ] || exit 0
# Read configuration variable file if it is present
[ -r /etc/default/$NAME ] && . /etc/default/$NAME
# Define LSB log_* functions.
# Depend on lsb-base (>= 3.0-6) to ensure that this file is present.
. /lib/lsb/init-functions
# Function that starts the daemon/service
        # Return
        #   0 if daemon has been started
        #   1 if daemon was already running
        #   2 if daemon could not be started
        start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON --test > /dev/null \
                || return 1
        start-stop-daemon --start --quiet --pidfile $PIDFILE --exec $DAEMON -- \
                $DAEMON_ARGS \
                || return 2
# Function that stops the daemon/service
        # Return
        #   0 if daemon has been stopped
        #   1 if daemon was already stopped
        #   2 if daemon could not be stopped
        #   other if a failure occurred
        start-stop-daemon --stop --quiet --retry=TERM/30/KILL/5 --pidfile $PIDFILE --name $NAME
        #rm -f $PIDFILE
        return "$RETVAL"
# Function that sends a SIGHUP to the daemon/service
do_reload() {
        start-stop-daemon --stop --signal HUP --quiet --pidfile $PIDFILE --name $NAME
        return 0
create_xconsole() {
        if [ ! -e /dev/xconsole ]
                mknod -m 640 /dev/xconsole p
case "$1" in
        log_daemon_msg "Starting $DESC" "$RSYSLOGD"
        case "$?" in
#               0|1) log_end_msg 0 ;;
                2) log_end_msg 1 ;;
        log_progress_msg "$RKLOGD"
        case "$?" in
                0|1) log_end_msg 0 ;;
                2) log_end_msg 1 ;;
        log_daemon_msg "Stopping $DESC" "$RKLOGD"
        do_stop "$RKLOGD" "$RKLOGD_PIDFILE"
        case "$?" in
#               0|1) log_end_msg 0 ;;
                2) log_end_msg 1 ;;
        log_progress_msg "$RSYSLOGD"
        do_stop "$RSYSLOGD" "$RSYSLOGD_PIDFILE"
        case "$?" in
                0|1) log_end_msg 0 ;;
                2) log_end_msg 1 ;;
        log_daemon_msg "Reloading $DESC" "$RSYSLOGD"
        do_reload "$RSYSLOGD" "$RSYSLOGD_PIDFILE"
        log_end_msg $?
        $0 stop
        $0 start
        echo "Usage: $SCRIPTNAME {start|stop|restart|reload|force-reload}" >&2
        exit 3

Next we create a default configuration in /etc/default/rsyslogd

vi /etc/default/rsyslog

Just copy and paste the contents below:

# Options to rsyslogd
# -m 0 disables 'MARK' messages.
# -r enables logging from remote machines
# -x disables DNS lookups on messages recieved with -r
# See rsyslogd(8) for more details
# Options to rklogd
# -2 prints all kernel oops messages twice; once for klogd to decode, and
#    once for processing with 'ksymoops'
# -x disables all klogd processing of oops messages entirely
# See rklogd(8) for more details

Next we create the directory /etc/rsyslog.d - this is needed for additional configuration options (not covered in the tutorial). 

mkdir /etc/rsyslog.d

Now we have all of the configuration files, but we need now to make sure rsyslog is started at boot time in the right moment. Please note that rsyslog is not compatible with klogd and sysklogd which are installed by default. These need to be disabled or de-installed. In this tutorial I will disable them.

Issue the following commands to start rsyslogd at the right time during system boot and to disable klogd and sysklogd.

ln -s  /etc/init.d/rsyslog /etc/rc3.d/S10rsyslog
mv /etc/rc3.d/S10sysklogd /etc/rc3.d/_S10sysklogd
mv /etc/rc3.d/S11klogd /etc/rc3.d/_S11klogd

This concludes the build and configuration part.


Step 3: Install Apache2 And PHP5

You can skip this if you have already installed apache2 and php5. 

apt-get install apache2 apache2-doc apache2-mpm-prefork apache2-utils libexpat1 ssl-cert
apt-get install libapache2-mod-php5 php5 php5-common php5-curl php5-dev php5-gd php5-idn php-pear php5-imagick php5-imap php5-json php5-mcrypt php5-memcache php5-mhash php5-ming php5-mysql php5-ps php5-pspell php5-recode php5-snmp php5-sqlite php5-tidy php5-xmlrpc php5-xsl 

You will be asked the following question:

Continue installing libc-client without Maildir support? <-- Yes


Step 4: Get phpLogCon And Install phpLogCon

cd /tmp
wget  http://www.phplogcon.org/Downloads-req-getit-lid-6.phtml

Next we will create a directory for phpLogCon and enable it in Apache2:

tar xvzf phplogcon-1.2.3.tar.gz
mkdir /var/www/phplogcon
cp -R phplogcon-1.2.3/* /var/www/phplogcon 

Configure apache2 for phpLogcon:

vi /etc/apache2/sites-enabled/your_site_conf 

Add the following alias to enable phplogcon: 

Alias /phplogcon "/var/www/phplogcon"

Insert the line above before </virtualhost> or if you have already aliases defined, insert the line there.

Note: if /var/www is your server root then you don't need to create an alias. 

Restart apache2 to enable the changes.

/etc/init.d/apache2 force-reload 


Step 5: Configure phpLogCon And Install The Tables In The rsyslog DB

Point your browser to http://yourdomain.tld/phplogcon and follow the steps on the screen. After the installation wizard has finished, the database is populated with all of the required tables and an administrative user is created.

Delete the install directory to enable phpLogCon:

rm -R /var/www/phplogcon/install


Step 6: Start rsyslog

Issue the following commands to start rsyslog and to stop klogd and sysklogd:

/etc/init.d/sysklogd stop
/etc/init.d/klogd stop
/etc/init.d/rsyslog start


Step 7: Enjoy rsyslog

Point your browser to http://yourdomain.tld/phplogcon and log in with the username and password you specified in the installation wizard. You should now see something like this:

phpLogCon Screenshot

Share this page:

5 Comment(s)