Posts Tagged ‘package’

Installing NRPE for Nagios/Icinga on Gentoo

You have two options for monitoring things like load average and logged in users on a remote host with Nagios or Icinga: SNMP (which you are probably already using for Cacti or a similar graphing/monitoring solution) or the Nagios Remote Plugin Executor (NRPE). NRPE gives one greater flexibility in the kind of data collected and actions executed. Unfortunately, NRPE is not included in Portage so we must compile and configure it on our own.

First, download the NRPE source tarball to your core monitoring server and install check_nrpe:

# mkdir /usr/src/nrpe
# cd /usr/src/nrpe
# wget [tarball]
# tar xf [tarball]
# cd nrpe-[version]
# ./configure
# make all
# cp src/check_nrpe /usr/[lib|lib64]/nagios/plugins/

Now add the command to your Icinga or Nagios config:

define command{
        command_name check_nrpe
        command_line $USER1$/check_nrpe -H $HOSTADDRESS$ -c $ARG1$
}

Compile and install the nrpe server on the target host:

# mkdir /usr/src/nrpe
# cd /usr/src/nrpe
# wget [tarball]
# tar xf [tarball]
# cd nrpe-[version]
# ./configure
# make all
# mkdir /etc/nrpe
# cp src/nrpe /usr/bin/
# cp sample-config/nrpe.cfg /etc/nrpe/

Install the nagios-plugins package from portage:

# emerge nagios-plugins

Edit /etc/nrpe/nrpe.cfg with a mind to security (particularly the allowed_hosts directive). The nagios-plugins package has already created the nagios user and group so there is no need to change the defaults. Update the command paths at the end of the file to reflect the location nagios-plugins installed to:

command[check_users]=/usr/lib/nagios/plugins/check_users -w 5 -c 10
command[check_load]=/usr/lib/nagios/plugins/check_load -w 15,10,5 -c 30,25,20
command[check_root]=/usr/lib/nagios/plugins/check_disk -w 20% -c 10% -p /dev/root
command[check_zombie_procs]=/usr/lib/nagios/plugins/check_procs -w 5 -c 10 -s Z
command[check_total_procs]=/usr/lib/nagios/plugins/check_procs -w 150 -c 200

Start the server as root with the daemoniz flag:

# nrpe -c /etc/nrpe/nrpe.cfg -d

We can see it has dropped down to the nagios user:

# ps aux | grep nrpe
nagios     570  0.0  0.0   4208   948 ?        Ss   16:55   0:00 nrpe -c /etc/nrpe/nrpe.cfg -d

Now we can test the configuration on the Nagios/Icinga core monitoring server:

# /usr/lib64/nagios/plugins/check_nrpe -H [ADDRESS]
NRPE v2.13

If the remote NRPE server’s configuration is working it should respond with NRPE [version]. Now we can set up some services on the monitoring server:

define service{
        host_name               myhost
        service_description     Users
        check_command           check_nrpe!check_users
        max_check_attempts      5
        check_interval          5
        retry_interval          1
        check_period            24x7
        notification_interval   30
        notification_period     24x7
        notification_options    w,c,r
        contact_groups          admins
        }

Note that the argument for check_command is one of the hard-coded commands in the nrpe server’s configuration file. Restart Icinga/Nagios to load the changes and begin monitoring:

# /etc/init.d/icinga restart

An init script will be required to make the NRPE daemon start on boot, create /etc/init.d/nrpe:

#!/sbin/runscript
# Copyright (c) 2012 http://foxpa.ws
# All rights released

description="Runs Nagios Remote Plugin Executor on Gentoo"

depend()
{
        need net
}

start()
{
        ebegin "Starting NRPE"
        start-stop-daemon --start --quiet --user=root --background --exec "/usr/bin/nrpe" -- -c /etc/nrpe/nrpe.cfg -d
        eend ${?}
}

stop()
{
        ebegin "Stopping NRPE"
        start-stop-daemon --stop --quiet --pidfile "/var/run/nrpe.pid"
        eend ${?}
}

Now make it executable and add it to the default runlevel:

# chmod +x /etc/init.d/nrpe
# rc-update add nrpe default

Alternatively, NRPE can be run by xinetd. From the README:

Running Under INETD or XINETD
-----------------------------

If you plan on running nrpe under inetd or xinetd and making use
of TCP wrappers, you need to do the following things:



1) Add a line to your /etc/services file as follows (modify the port
   number as you see fit)

        nrpe            5666/tcp        # NRPE



2) Add entries for the NRPE daemon to either your inetd or xinetd
   configuration files.  Which one your use will depend on which
   superserver is installed on your system.  Both methods are described
   below.  NOTE: If you run nrpe under inetd or xinetd, the server_port
   and allowed_hosts variables in the nrpe configuration file are
   ignored.


   ***** INETD *****
   If your system uses the inetd superserver WITH tcpwrappers, add an
   entry to /etc/inetd.conf as follows:

        nrpe    stream  tcp     nowait  <user> /usr/sbin/tcpd <nrpebin> -c <nrpecfg> --inetd

   If your system uses the inetd superserver WITHOUT tcpwrappers, add an
   entry to /etc/inetd.conf as follows:

        nrpe    stream  tcp     nowait  <user> <nrpebin> -c <nrpecfg> --inetd


   - Replace <user> with the name of the user that the nrpe server should run as.
        Example: nagios
   - Replace <nrpebin> with the path to the nrpe binary on your system.
        Example: /usr/local/nagios/nrpe
   - Replace <nrpecfg> with the path to the nrpe config file on your system.
        Example: /usr/local/nagios/nrpe.cfg


   ***** XINETD *****
   If your system uses xinetd instead of inetd, you'll probably
   want to create a file called 'nrpe' in your /etc/xinetd.d
   directory that contains the following entries:


        # default: on
        # description: NRPE
        service nrpe
        {
                flags           = REUSE
                socket_type     = stream        
                wait            = no
                user            = <user>
                server          = <nrpebin>
                server_args     = -c <nrpecfg> --inetd
                log_on_failure  += USERID
                disable         = no
                only_from       = <ipaddress1> <ipaddress2> ...
        }


   - Replace <user> with the name of the user that the nrpe server should run as.
   - Replace <nrpebin> with the path to the nrpe binary on your system.
   - Replace <nrpecfg> with the path to the nrpe config file on your system.
   - Replace the <ipaddress> fields with the IP addresses of hosts which
     are allowed to connect to the NRPE daemon.  This only works if xinetd was
     compiled with support for tcpwrappers.



3) Restart inetd or xinetd will the following command (pick the
   on that is appropriate for your system:

        /etc/rc.d/init.d/inet restart

        /etc/rc.d/init.d/xinetd restart

   OpenBSD users can use the following command to restart inetd:

        kill -HUP `cat /var/run/inet.pid`



4) Add entries to your /etc/hosts.allow and /etc/hosts.deny
   file to enable TCP wrapper protection for the nrpe service.
   This is optional, although highly recommended.

Delete All Entries for a Given Criterion in ip_conntrack Table

You may find yourself in a position where it is necessary to remove all the entries in Netfilter’s connection tracking table (ip_conntrack) for a particular criterion, like the source or destination IP.

For example, I recently detected a user on one of my networks engaged in what was likely a TCP denial of service attack against root name servers (despite the odd fact that the destination port was 80). Being a NATted user, all of their connections were being tracked. The default time-out for tracking an established connection being 5 days, simply disconnecting the user at the second layer would not relieve the congestion on my routers within an acceptable time frame.

#  cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count
98636

#  cat /proc/net/ip_conntrack | grep "xxx.xxx.xxx.xxx"
tcp      6 416408 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.5.147 sport=58967 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.5.147 dst=yyy.yyy.yyy.yyy sport=80 dport=58967 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 416406 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.9.239 sport=58967 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.9.239 dst=yyy.yyy.yyy.yyy sport=80 dport=58967 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 416400 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.11.231 sport=58968 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.11.231 dst=yyy.yyy.yyy.yyy sport=80 dport=58968 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 416387 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.14.37 sport=58968 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.14.37 dst=yyy.yyy.yyy.yyy sport=80 dport=58968 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 416381 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.11.103 sport=58968 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.11.103 dst=yyy.yyy.yyy.yyy sport=80 dport=58968 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 416275 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.9.57 sport=58967 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.9.57 dst=yyy.yyy.yyy.yyy sport=80 dport=58967 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 415776 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.1.52 sport=58967 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.1.52 dst=yyy.yyy.yyy.yyy sport=80 dport=58967 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 417319 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.43.60 sport=58967 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.43.60 dst=yyy.yyy.yyy.yyy sport=80 dport=58967 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 417319 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.43.13 sport=58968 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.43.13 dst=yyy.yyy.yyy.yyy sport=80 dport=58968 packets=0 bytes=0 mark=0 secmark=0 use=1

...

It is possible to clear tracking entries en masse by removing then reloading the iptables rule that requires them to be tracked in the first place, but on a production gateway this is even less acceptable than waiting for them to expire. Fortunately, we can interact with the ip_conntrack table via conntrack-tools.

Against common sense, conntrack-tools is not available in the repositories for (at least version 5.2) of ClearOS, my favourite router distro. I grabbed a couple recent versions in RPM form but they didn’t feel like playing ball so I ended up with version 0.9.5 from ftp://ftp.pbone.net/mirror/archive.fedoraproject.org/fedora/linux/updates/8/i386.newkey/conntrack-tools-0.9.5-3.fc8.i386.rpm

Apparently newer versions allow one to use -D intuitively, i.e.:

# conntrack -D -s xxx.xxx.xxx.xxx

But this is not the case for at least versions including and prior to 0.97 – these require the d, dport, s and sport flags.

This wonderful person provides a way to pipe the output of conntrack -L (which lists entries the way I’d like to delete them, i.e. -s only) into sed which then breaks the output lines up and awk runs them with conntrack -D appropriately. I had to do some cleanup to get it to work due to the way their blog software mangles punctuation (a lot of my first posts here are mangled in the same way – pobody’s nerfect!):

 conntrack -L -s xxx.xxx.xxx.xxx | sed 's/=/ /g'| awk '{system("conntrack -D -s "$6" -d "$8" -p "$1" --sport="$10" --dport="$12)}'

It should be pretty clear how conntrack -L -s can be modified to work with the destination address or more complicated pattern matching.

Now we can see the ip_conntrack table is at a more reasonable level:

[root@router ~]#  cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count
46676

Portage Errors: sys-devel/patch is blocking sys-libs/glibc

If you run into this error while updating glibc:

# emerge --update gcc glibc

 * IMPORTANT: 6 news items need reading for repository 'gentoo'.
 * Use eselect news to read news items.

Calculating dependencies -[root@hd-t3672cl ~]# xm console dns2
... done!
[ebuild     U ] dev-libs/mpfr-3.0.1_p4 [2.4.1_p5]
[ebuild     U ] sys-devel/gcc-config-1.5-r2 [1.4.1]
[ebuild  N    ] virtual/os-headers-0 
[ebuild  N    ] dev-libs/mpc-0.8.2 
[ebuild  NS   ] sys-devel/gcc-4.5.3-r2 [4.3.4] USE="cxx hardened mudflap nls nptl openmp (-altivec) -bootstrap -build -doc (-fixed-point) -fortran -gcj -graphite -gtk (-libssp) -lto (-multilib) -multislot -nocxx -nopie -nossp -objc -objc++ -objc-gc -test -vanilla" 
[ebuild     U ] sys-libs/glibc-2.13-r4 [2.10.1-r1] USE="hardened*" 
[ebuild     U ] sys-devel/gcc-4.3.6-r1 [4.3.4] USE="cxx%* hardened* nptl* -fortran* (-libssp) -nossp%" 
[blocks B     ] <sys-devel/patch-2.6 ("<sys-devel/patch-2.6" is blocking sys-libs/glibc-2.13-r4)

 * Error: The above package list contains packages which cannot be
 * installed at the same time on the same system.

  ('ebuild', '/', 'sys-libs/glibc-2.13-r4', 'merge') pulled in by
    >=sys-libs/glibc-2.8 required by ('ebuild', '/', 'sys-devel/gcc-4.5.3-r2', 'merge')
    >=sys-libs/glibc-2.8 required by ('ebuild', '/', 'sys-devel/gcc-4.3.6-r1', 'merge')
    glibc


For more information about Blocked Packages, please refer to the following
section of the Gentoo Linux x86 Handbook (architecture is irrelevant):

http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?full=1#blocked

 * IMPORTANT: 6 news items need reading for repository 'gentoo'.
 * Use eselect news to read news items.

Update patch first and carry on:

# emerge --update patch
# emerge --update glibc gcc
Return top
foxpa.ws
Online Marketing Toplist
Internet
Technology Blogs - Blog Rankings

Internet Blogs - BlogCatalog Blog Directory

Technology blogs
Bad Karma Networks

Please Donate!


Made in Canada  •  There's a fox in the Gibson!  •  2010-12