=^.^=

Transparent HTTP/DNS Anonymity, Encryption and Filtering with Tor, Privoxy, Squid, DansGuardian and dnsmasq on ClearOS

• karma

Warning: There is a problem with UDP-based DNS query interception and a fix will be posted shortly! See comments for more info.

In this article we will use Tor and Privoxy with dnsmasq and netfilter on ClearOS to transparently encrypt and anonymize web and DNS traffic on an entire private network or a single host without necessarily having to reconfigure any of its constituents. We will also cover daisy-chaining Squid to speed up browsing and DansGuardian to provide content filtering against browser attacks and on-the-fly virus scanning via ClamAV.

For your convenience I have provided a virtual machine image with these modifications at the end of the article.

Introduction: Pros

There are a number of reasons one might choose to use this method over locally installing Tor, which may include:

  • Browsers do not require configuration
  • Not all software that makes use of HTTP supports using proxies
  • Anonymous software updates
  • Composition of network to be protected frequently changes (guests with laptops etc.)
  • Squid cache "speeds up" slow Tor requests
  • Clients may lack technical skill required to properly implement Tor locally and utilize it wisely
  • Users with multiple browsers installed can not inadvertently break network policy simply by switching or installing new browsers
  • The side-benefits of running behind ClearOS: drop-in IPS, transparent gw-gw VPN with IPSEC, PPTP server etc.
  • Centralized content filtering against browser exploits, phishing sites etc.
  • On-the-fly anti-virus protection with ClamAV
  • Allows packet inspectors like BotHunter to inspect torified traffic for malicious activity

Introduction: Cons

  • Traffic sent over Tor tends to be very slow. Reducing the number of hops increases speed but decreases anonymity.
  • TLS/SSL still requires browser configuration
  • Traffic other than DNS and HTTP on ports 53 and 80 must be configured to use the prox(y/ies).
  • Communication between hosts and the gateway is not Tor encrypted, of particular concern on wireless networks.

Introduction: whatis

  • ClearOS - A full-featured, easy-to-use turnkey Linux router distribution. It is based on CentOS (RHEL) and provides a simple web-based configuration front-end to help drive its powerful network services. Personal favourites include Intrusion Prevention, bandwidth and QoS management, connection failover and load balancing.
  • Tor - Tor provides anonymity for networked applications by routing traffic through encrypted, multi-layered tunnels between nodes operated by Tor supporters.
  • Privoxy - From their website: "Privoxy is a non-caching web proxy with advanced filtering capabilities for enhancing privacy, modifying web page data and HTTP headers, controlling access, and removing ads and other obnoxious Internet junk."
  • Squid - A versatile caching proxy. It is available with a webconfig interface for ClearOS users through the default repositories.
  • dnsmasq - A multi-talented DHCP, caching-forwarding DNS and TFTP/PXE/BOOTP server that is a default component of ClearOS installations.
  • DansGuardian - A content filtering proxy that also gives us the ability to plug in virus scanning with ClamAV.
  • ClamAV - An anti-virus solution designed to run on UNIX servers to provide filter-style scanning in applications as diverse as MTAs and PHP uploads.

Introduction: The Objective

By the end of this article we will have a working router, either on dedicated physical hardware or as a virtual machine running on a host. In both scenarios it will be possible to transparently route all HTTP and DNS traffic through Tor from any hosts configured to use the ClearOS box as their gateway. By running a virtual machine on a single host one can effectively put a self-contained firewall appliance between one's self and the upstream network. In fact, this method allows those with more than one NIC installed to eliminate an existing embedded firewall/router appliance and serve an entire private network without significantly altering the host machine's configuration. Protected, portable hotspots can be created with little more than a laptop and a switch or secondary wifi card simply by bridging the virtual machine's internal and external interfaces with the appropriate NICs.

We will be using netfilter and NAT to transparently redirect requests destined for remote servers to proxies running locally on the router:

We will be daisy-chaining a series of proxies together. One may choose to omit one or two depending on available resources and your intent. For the purposes of this article, we will cover configuring all three of the following; if you choose to omit a proxy you must adjust your configuration (including port forwarding for transparency) to reflect this.

  • Privoxy - This is the proxy we will be connecting to Tor, it is not optional. Default port tcp/8118.
  • Squid - Provides caching. Optional, module is required as a dependency for DansGuardian in ClearOS repos but does not need to be enabled to use it if bypassing the content filter webconfig panel. Default port tcp/3128
  • DansGuardian - Content filtering, ClamAV anti-virus module. Both optional. Default port tcp/8080.

Introduction: Prerequisites

You must start this process with a working ClearOS installation. You can use my ClearOS virtual machines for paravirtualized Xen or hardware emulators to set up a test environment. If you will be installing ClearOS on dedicated hardware you may find my personal installation checklist helpful, if a bit dated. For a production environment I recommend going with the Xen paravirtualized image but if your objective is to create a more portable solution the HVM image with VirtualBox is an alright pick.*

* See VirtualBox (Windows) for instructions on converting the raw disk image into a VDI.

Configuration: ClearOS: Networking

How you configure your networking on the ClearOS host will depend on whether you are virtualizing or using real hardware, how many NICs the physical server has and whether you intend to replace your existing gateway. The specifics go beyond the scope of this article but the end result must be the same: hosts on the private network must at least be configured to use the ClearOS installation as their gateway.

It is possible to put the ClearOS box behind an existing NATted router by creating a separate subnet for its LAN side. For example, the existing network is 192.168.0.0/24 and you have a ClearOS VM on your desktop with two virtual network interfaces bound to your one physical interface. You would configure the external interface of the ClearOS VM for 192.168.0.100/24 gw 192.168.0.1 then configure the internal (LAN) interface for 192.168.1.1/24. Next you would configure the hosts (including the machine on which the VM is running if you desire) you will be forcing through Tor to use IPs on the 192.168.1.0/24 subnet with 192.168.1.1 as the gateway. Be wary of offering DHCP from both the existing router and ClearOS.

Configuration: ClearOS: netfilter

We will need to store some iptables rules in such a way that they are loaded on startup. ClearOS provides two ideologically-correct places to do this. If you installed the Firewall - Custom rules module you will have a webconfig interface for manipulating rules stored in /etc/rc.d/rc.firewall.custom. The format of the file is simply

iptables options # descriptive comment

Traditionally, user-defined rules have been stored in /etc/rc.d/rc.firewall.local. This option is available whether or not you have installed the custom rules module and is formatted in much the same fashion, though one may do away with the hash mark and comment.

If you choose to omit one or more of the proxies outlined in this procedure you will have to manually configure netfilter to redirect TCP traffic on port 80 to the listening port of the last proxy in the chain. The "magic rule" looks like this:

iptables -t nat -A PREROUTING -i eth1 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports {PORT_NUMBER}

Where {PORT_NUMBER} is the intended target. Be mindful of existing rules that may need to be deleted.

Configuration: ClearOS: Installing the Software

ClearOS Modules

Assuming you now have a working ClearOS instance you will need to register with ClearSDN to get software updates and the rest of the modules we need (unless you selected them when doing the initial ClearOS installation).

If you will be using Squid and DansGuardian install the the Web Proxy Module and Content Filtering Module under Clear Center > Modules in the webconfig (if you did not do so during the server installation). Also install Antivirus file scanner module if you wish to use DansGuardian in conjunction with ClamAV to scan HTTP traffic for viruses on the fly.

If you would like to provide users attempting to use HTTPS friendly failure pages telling them how to enable their proxy locally install the Web server module.

Privoxy from RPM

Log into the ClearOS machine via SSH and install privoxy:

# rpm -iv ftp://ftp.muug.mb.ca/mirror/centos/5.5/os/i386/CentOS/privoxy-3.0.3-9.3.el5.i386.rpm

Make privoxy start on boot:

# chkconfig --level 2345 privoxy on

Tor from RPM

Installing Tor from RPM takes care of user account creation and init scripts automagically.

To install Tor you will first need libevent:

# yum install libevent

We will be using the torify command on the router to send our DNS requests over Tor later in the article. You will need to install tsocks or torify will not work:

# rpm -iv ftp://ftp.univie.ac.at/systems/linux/dag/redhat/el5/en/i386/dag/RPMS/tsocks-1.8-7.beta5.2.el5.rf.i386.rpm

Now Tor (adjust for the latest version):

# rpm -iv http://deb.torproject.org/torproject.org/rpm/centos5/tor-0.2.1.29-tor.0.rh5_5.i386.rpm

Make Tor start on boot:

# chkconfig --level 2345 tor on

If you don't like the thought of ClearOS calling home disable suvad - bear in mind this will negatively impact your ability to receive software updates. First, remove your ClearSDN registration information via webconfig by loading this URL:

http://LAN-address:81/admin/register.php?Reset=yes

Next run:

# /etc/init.d/suvad stop
# chkconfig --level 2345 suvad off

Bear in mind that even if you decide to leave suvad on the basic ClearSDN subscription includes and automatically configures Clear's dynamic DNS service. One may wish to disable this feature if one does not want systemname.poweredbyclear.com always pointing to your current public IP.

Tor from Source

A while back one had to modify Tor's source to get one-hop circuits running but it seems this recommendation has since been implemented and one-hop circuits can be configured without modification, provided the intended exit node is configured to allow this.

If you still wish to compile Tor from source on ClearOS we must first install its build environment:

# yum groupinstall "Development Tools"

If you have already installed Tor from RPM remove it:

[root@system html]# cp -r /etc/tor/ /etc/tor.bak
[root@system html]# cp /etc/init.d/tor /etc/init.d/tor.bak
[root@system html]# rpm -e tor
tor (pid 2058) running
Stopping tor: /usr/bin/torctl stop: tor stopped
[  OK  ]
/var/tmp/rpm-tmp.25797: line 12: fg: no job control
warning: /etc/tor/torrc saved as /etc/tor/torrc.rpmsave

Download the latest source tarball from the Tor project's website:

# cd /usr/src
# wget https://www.torproject.org/dist/tor-0.2.1.28.tar.gz
# tar zxf tor-0.2.1.28.tar.gz
# cd tor-0.2.1.28

Before we begin compiling we need to install libevent (if it is not already available) and libevent-devel:

# yum install libevent libevent-devel

Now OpenSSL and OpenSSL-devel:

# yum install openssl openssl-devel

On older software one had to modify src/or/control.c, removing or commenting out the following lines (big ups to sprawl for their article in 2600):

if (circ && (circuit_get_cpath_len(circ)<2 || hop==1)) {
   connection_write_str_to_buf(
                   "551 Can't attach stream to one-hop circuit.\r\n", conn);
   return 0;
 }

As we can see this block has been replaced:

  /* Is this a single hop circuit? */
  if (circ && (circuit_get_cpath_len(circ)<2 || hop==1)) {
    routerinfo_t *r = NULL;
    char* exit_digest;
    if (circ->build_state &&
        circ->build_state->chosen_exit &&
        circ->build_state->chosen_exit->identity_digest) {
      exit_digest = circ->build_state->chosen_exit->identity_digest;
      r = router_get_by_digest(exit_digest);
    }
    /* Do both the client and relay allow one-hop exit circuits? */
    if (!r || !r->allow_single_hop_exits ||
        !get_options()->AllowSingleHopCircuits) {
      connection_write_str_to_buf(
      "551 Can't attach stream to this one-hop circuit.\r\n", conn);
      return 0;
    }
    ap_conn->chosen_exit_name = tor_strdup(hex_str(exit_digest, DIGEST_LEN));
  }

Compile the software and install it:

# ./configure
# make
# make install

Now we need to make a non-privileged user for Tor to run under.

# useradd -d /var/lib/tor -s /sbin/nologin -r tor

Let's use the init script provided by the RPM, modifications in bold:

#!/bin/sh
#
# tor    The Onion Router
#
# Startup/shutdown script for tor. This is a wrapper around torctl;
# torctl does the actual work in a relatively system-independent, or at least
# distribution-independent, way, and this script deals with fitting the
# whole thing into the conventions of the particular system at hand.
# This particular script is written for Red Hat/Fedora Linux, and may
# also work on Mandrake, but not SuSE.
#
# These next couple of lines "declare" tor for the "chkconfig" program,
# originally from SGI, used on Red Hat/Fedora and probably elsewhere.
#
# chkconfig: 2345 90 10
# description: Onion Router - A low-latency anonymous proxy
#

# Library functions
if [ -f /etc/rc.d/init.d/functions ]; then
   . /etc/rc.d/init.d/functions
elif [ -f /etc/init.d/functions ]; then
   . /etc/init.d/functions
fi

# Increase open file descriptors a reasonable amount
ulimit -n 8192

TorCTL=/usr/bin/torctl

# torctl will use these environment variables
TorUSER=tor
export TorUSER

if [ -x /bin/su ] ; then
    SUPROG=/bin/su
elif [ -x /sbin/su ] ; then
    SUPROG=/sbin/su
elif [ -x /usr/bin/su ] ; then
    SUPROG=/usr/bin/su
elif [ -x /usr/sbin/su ] ; then
    SUPROG=/usr/sbin/su
else
    SUPROG=/bin/su
fi

case "$1" in

    start)
    action $"Starting tor:" $TorCTL start
    RETVAL=$?
    ;;

    stop)
    action $"Stopping tor:" $TorCTL stop
    RETVAL=$?
    ;;

    restart)
    action $"Restarting tor:" $TorCTL restart
    RETVAL=$?
    ;;

    reload)
    action $"Reloading tor:" $TorCTL reload
    RETVAL=$?
    ;;

    status)
    $TorCTL status
    RETVAL=$?
    ;;

    *)
    echo "Usage: $0 (start|stop|restart|reload|status)"
    RETVAL=1
esac

exit $RETVAL

We're going to need the torctl script too, open /usr/bin/torctl and paste (modifications in bold):

#!/bin/sh
#
# Tor control script designed to allow an easy command line interface
# to controlling The Onion Router
#
# The exit codes returned are:
#       0 - operation completed successfully. For "status", tor running.
#       1 - For "status", tor not running.
#       2 - Command not supported
#       3 - Could not be started or reloaded
#       4 - Could not be stopped
#       5 -
#       6 -
#       7 -
#       8 -
#
# When multiple arguments are given, only the error from the _last_
# one is reported.
#
#
# |||||||||||||||||||| START CONFIGURATION SECTION  ||||||||||||||||||||
# --------------------                              --------------------
# Name of the executable
EXEC=tor
#
# the path to your binary, including options if necessary
TorBIN="/usr/local/bin/$EXEC"
#
# the path to the configuration file
TorCONF="/usr/local/etc/tor/torrc"
#
# the path to your PID file
PIDFILE="/var/run/tor/tor.pid"
#
# The path to the log file
LOGFILE="/var/log/tor/tor.log"
#
# The path to the datadirectory
TorDATA="/var/lib/tor"
#
TorARGS="--pidfile $PIDFILE --log \"notice file $LOGFILE\" --runasdaemon 1"
TorARGS="$TorARGS --datadirectory $TorDATA"

# If user name is set in the environment, then use it;
# otherwise run as the invoking user (or whatever user the config
# file says)... unless the invoking user is root. The idea here is to
# let an unprivileged user run tor for her own use using this script,
# while still providing for it to be used as a system daemon.
if [ "x`id -u`" = "x0" ]; then
    TorUSER=tor
fi

if [ "x$TorUSER" != "x" ]; then
    TorARGS="$TorARGS --user $TorUSER"
fi

# We no longer wrap the Tor daemon startup in an su when running as
# root, because it's too painful to make the use of su portable.
# Just let the daemon set the UID and GID.
START="$TorBIN -f $TorCONF $TorARGS"

#
# --------------------                              --------------------
# ||||||||||||||||||||   END CONFIGURATION SECTION  ||||||||||||||||||||

ERROR=0
ARGV="$@"
if [ "x$ARGV" = "x" ] ; then
    ARGS="help"
fi

checkIfRunning ( ) {
    # check for pidfile
    PID=unknown
    if [ -f $PIDFILE ] ; then
        PID=`/bin/cat $PIDFILE`
        if [ "x$PID" != "x" ] ; then
            if kill -0 $PID 2>/dev/null ; then
               STATUS="$EXEC (pid $PID) running"
               RUNNING=1
            else
                STATUS="PID file ($PIDFILE) present, but $EXEC ($PID) not running"
                RUNNING=0
            fi
        else
            STATUS="$EXEC (pid $PID?) not running"
            RUNNING=0
        fi
    else
       STATUS="$EXEC apparently not running (no pid file)"
       RUNNING=0
    fi
    return
}

for ARG in $@ $ARGS
do
    checkIfRunning

    case $ARG in
    start)
        if [ $RUNNING -eq 1 ]; then
            echo "$0 $ARG: $EXEC (pid $PID) already running"
            continue
        fi
        if eval "$START" ; then
            echo "$0 $ARG: $EXEC started"
            # Make sure it stayed up!
            /bin/sleep 1
            checkIfRunning
            if [ $RUNNING -eq 0 ]; then
                echo "$0 $ARG: $EXEC (pid $PID) quit unexpectedly"
            fi
        else
            echo "$0 $ARG: $EXEC could not be started"
            ERROR=3
        fi
        ;;
    stop)
        if [ $RUNNING -eq 0 ]; then
            echo "$0 $ARG: $STATUS"
            continue
        fi
        if kill -15 $PID ; then
            echo "$0 $ARG: $EXEC stopped"
        else
            /bin/sleep 1
            if kill -9 $PID ; then
                echo "$0 $ARG: $EXEC stopped"
            else
                echo "$0 $ARG: $EXEC could not be stopped"
                ERROR=4
            fi
        fi
        # Make sure it really died!
        /bin/sleep 1
        checkIfRunning
        if [ $RUNNING -eq 1 ]; then
            echo "$0 $ARG: $EXEC (pid $PID) unexpectedly still running"
            ERROR=4
        fi
        ;;
    restart)
        $0 stop start
        ;;
    reload)
        if [ $RUNNING -eq 0 ]; then
            echo "$0 $ARG: $STATUS"
            continue
        fi
        if kill -1 $PID; then
            /bin/sleep 1
            echo "$EXEC (PID $PID) reloaded"
        else
            echo "Can't reload $EXEC"
            ERROR=3
        fi
        ;;
    status)
        echo $STATUS
        if [ $RUNNING -eq 1 ]; then
            ERROR=0
        else
            ERROR=1
        fi
        ;;
    log)
        cat $LOGFILE
        ;;
    help)
        echo "usage: $0 (start|stop|restart|status|help)"
        /bin/cat <<EOF

start      - start $EXEC
stop       - stop $EXEC
restart    - stop and restart $EXEC if running or start if not running
reload     - cause the running process to reinitialize itself
status     - tell whether $EXEC is running or not
log        - display the contents of the log file
help       - this text

EOF
        ERROR=0
        ;;
    *)
        $0 help
        ERROR=2
        ;;

    esac

done

exit $ERROR

Now make both scripts executable:

# chmod +x /usr/bin/torctl
# chmod +x /etc/init.d/tor

Now copy the sample configuration file:

# cp /usr/local/etc/tor/torrc.sample /usr/local/etc/tor/torrc

Create these directories if they were not created for you:

# mkdir /var/log/tor
# mkdir /var/run/tor
# mkdir /var/lib/tor
# chown tor: /var/log/tor
# chown tor: /var/run/tor
# chown tor: /var/lib/tor -R
# chmod 750 /var/log/tor
# chmod 750 /var/run/tor
# chmod 750 /var/lib/tor

Add Tor to startup:

# chkconfig --add tor
# chkconfig --level 2345 tor add

Configuration: Tor

Tor is configured by default to listen to port 9050 on localhost. If you would like hosts on the private network to be able to have direct access to Tor you must change or add a second SocksListenAddress directive for your router's LAN address (in this case 192.168.1.1) in /etc/tor/torrc.

Enable the control port by uncommenting this line in /etc/tor/torrc:

ControlPort 9051

Run this command, replacing password with the password you would like to use for the controller:

tor --hash-password password

Ignore the warning about running Tor as root; the actual daemon has been configured by its RPM to use a non-privileged account. Ucomment and change the HashedControlPassword to the output you were provided. Start Tor:

# /etc/init.d/tor start

Verify you can log into the controller, replace password with the unencrypted password you gave tor --hash-password:

[root@system log]# telnet localhost 9051
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
authenticate "password"
250 OK

By default Tor builds random 3-hop circuits. You can use the controller to increase performance or anonymity by increasing or decreasing the number of hops your circuits use, and even the geographical distance between hops. I recommend you read Tor Control Protocol for more information.

Quit telnet by holding down control and pressing the ] key. Type quit and hit enter.

It is important that your system time is accurate; you will find it difficult to establish circuits if your clock is skewed. Let's force an NTP update before Tor is started by modifying its init script at /etc/init.d/tor:

    start)
    /usr/sbin/ntpdate -s -b -u 0.gentoo.pool.ntp.org 1.gentoo.pool.ntp.org 2.gentoo.pool.ntp.org 3.gentoo.pool.ntp.org
    action $"Starting tor:" $TorCTL start
    RETVAL=$?
    ;;

Note: choose time servers you trust. You may not be able to start Tor and subsequently torify ntpd if the first update does not go out vanilla. ClearOS uses these time servers:

time1.clearsdn.com
time2.clearsdn.com
time3.clearsdn.com
time4.clearsdn.com

To keep the system time accurate and optionally provide NTP service for your local network activate the NTP daemon:

# chkconfig --level 2345 ntpd on
# /etc/init.d/ntpd start

NOTE Xen users experiencing clock skew between the VM and dom0 should add this to Tor's init script before the call to ntpdate (/etc/rc.d/rc.local is run too late):

echo 1 > /proc/sys/xen/independent_wallclock

Configuration: Tor: torify

If you have configured Tor to only listen on your LAN address you must configure tsocks accordingly in /etc/tsocks.conf:

# This is the configuration for libtsocks (transparent socks)
# Lines beginning with # and blank lines are ignored
#
# This sample configuration shows the simplest (and most common) use of
# tsocks. This is a basic LAN, this machine can access anything on the
# local ethernet (192.168.0.*) but anything else has to use the SOCKS version
# 4 server on the firewall. Further details can be found in the man pages,
# tsocks(8) and tsocks.conf(5) and a more complex example is presented in
# tsocks.conf.complex.example

# We can access 192.168.0.* directly
local = 192.168.0.0/255.255.255.0

# Otherwise we use the server
server = 192.168.0.1

Configure tsocks to connect to your LAN address on every host in the private network where you would like to use torify without the overhead of a local Tor deployment.

Configuration: Privoxy

Privoxy listens for connections on localhost by default. Set the listen-address parameter in /etc/privoxy/config to the LAN address of your router if you would like the ability to manually configure browsers to use it - a good idea if you expect to be using TLS/SSL connections. Configure Privoxy to connect to Tor:

# listen-address 127.0.0.1:8118
listen-address 192.168.1.1:8118
...
forward-socks4a / 127.0.0.1:9050 .

The period at the end is intentional. Start Privoxy:

# /etc/init.d/privoxy start

Configuration: Squid

Log in to the ClearOS webconfig at https://lan-ip:81 and configure Web Proxy under the Gateway menu. Set a suitable cache size and enable transparent mode. Optionally, you may wish to enable the banner and pop-up filter. If you will be using DansGuardian to provide anti-virus protection and content filtering enable the Contant Filter. Make the service start on boot, then start it.

Next we must configure squid to channel its requests through privoxy. Add this to /etc/squid/squid.conf:

cache_peer localhost parent 8118 7 no-query default

header_replace User-Agent Mozilla/5.0 (en) Gecko/20070515 Firefox/2.0.0.4
header_access User-Agent deny all
header_access From deny all
header_access Server deny all
header_access WWW-Authenticate deny all
header_access Link deny all
header_access Cache-Control deny all
header_access Proxy-Connection deny all
header_access X-Cache deny all
header_access X-Cache-Lookup deny all
header_access Via deny all
header_access Forwarded-For deny all
header_access X-Forwarded-For deny all
header_access Pragma deny all
header_access Keep-Alive deny all
header_access Referer deny all
# header_access Cookie deny all

Add this line if you are record-conscious would like to disable Squid's cache:

cache deny all

Then update any cache_dir values:

#cache_dir ufs /var/spool/squid 500 16 256
cache_dir null /tmp

Alternatively, one can use tmpfs/shm for the cache_dir. With this method the entire cache tree is stored in RAM; evidence disappears when the machine is shut down rather than being written to potentially recoverable storage media. Not only does this let us reap the benefits of caching in lieu of the low speeds Tor is notorious for, it produces faster cache hits. The down side is that if the router is compromised before it can be shut down the contents of /dev/shm may also be compromised. To use this method you will need enough free RAM to satisfy the cache constraints we configured earlier in proxy server webconfig module. Do not add the cache deny allACL to the configuration script. Instead, configure your cache_dir to reflect:

#cache_dir ufs /var/spool/squid 500 16 256
cache_dir ufs /dev/shm/squid 500 16 256

Now edit /etc/init.d/squid to create the directory on tmpfs with the proper permissions every time squid starts:

start() {
        automagic
        mkdir /dev/shm/squid
        chown squid:squid /dev/shm/squid
        chmod 750 /dev/shm/squid
        for adir in $CACHE_SWAP; do

Now restart Squid:

# /etc/init.d/squid restart

Try loading http://whatismyip.com etc. from a host configured to use the ClearOS gateway. If everything worked successfully you should see the IP of a Tor exit node at this stage.

Configuration: DansGuardian

In webconfig click on the Content Filter link under the Gateway menu. Configure it to start on boot then start it manually. You can take advantage of the Blanket Block to enforce a web-based authentication scheme as I outlined in this article: Transparent Proxy for Hot Spot/Public Network Web-Based Authentication on ClearOS. Enable the virus scanner to take advantage of ClamAV.

We will need to remove a rule which this process inserts automatically. Use one of the custom rule files (see Configuration: ClearOS: netfilter) to run:

iptables -t nat -D PREROUTING -s ! 127.0.0.1 -p tcp -m tcp --dport 3128 -j REDIRECT --to-ports 82

Now we must direct DansGuardian to hand off connections to the last proxy in the chain before it; port 3128 for Squid or 8118 for Privoxy. Edit /etc/dansguardian-av/dansguardian.conf:

# Network Settings
#
# the IP that DansGuardian listens on.  If left blank DansGuardian will
# listen on all IPs.  That would include all NICs, loopback, modem, etc.
# Normally you would have your firewall protecting this, but if you want
# you can limit it to a certain IP. To bind to multiple interfaces,
# specify each IP on an individual filterip line.
filterip =

# the port that DansGuardian listens to.
filterport = 8080

# the ip of the proxy (default is the loopback - i.e. this server)
proxyip = 127.0.0.1   

# the port DansGuardian connects to proxy on
proxyport = 3128
#proxyport = 8118

Configuration: dnsmasq: Torifying dnsmasq

I was pleasantly surprised to find that dnsmasq - ClearOS' default caching DNS and DHCP server - not only worked with torify but continued to bind to localhost and the LAN interface properly. It's as simple as editing this line in /etc/init.d/dnsmasq to show:

start)
 echo -n $"Starting $prog: "
 daemon torify $dnsmasq
 RETVAL=$?

Restart dnsmasq:

# /etc/init.d/dnsmasq restart

Make sure dnsmasq is using the tsocks library:

[root@trollup ~]# lsof | grep tsocks

dnsmasq 31015 nobody mem REG 3,1 63136 12419 /lib/libtsocks.so.1.8

Queries made through dnsmasq via the ClearOS box are now sent out over Tor and results are cached in RAM locally. Make sure the resolv.conf on the gateway starts with 127.0.0.1 but includes additional name servers so that dnsmasq knows where to send its queries.

[root@system ~]# cat /etc/resolv.conf
nameserver 127.0.0.1
nameserver 208.94.147.150
nameserver 208.94.147.151

Configuration: dnsmasq: Transparent Query Redirection

Add the following to your firewall configuration:

iptables -t nat -A PREROUTING -i eth1 -p tcp -m tcp --dport 53 -j REDIRECT --to-ports 53
iptables -t nat -A PREROUTING -i eth1 -p udp -m udp --dport 53 -j REDIRECT --to-ports 53

Where eth1 is the ClearOS box's LAN interface.

Configuration: dnsmasq: Verifying Transparency

Delete the rules we just added with iptables:

iptables -t nat -D PREROUTING -i eth1 -p tcp -m tcp --dport 53 -j REDIRECT --to-ports 53
iptables -t nat -D PREROUTING -i eth1 -p udp -m udp --dport 53 -j REDIRECT --to-ports 53

UNIX

On a *nix host configured to use the ClearOS box as the gateway determine the name servers for any given domain:

karma@bzp ~ $ whois google.ca

...

Name servers:
    ns1.google.com
    ns2.google.com
    ns3.google.com
    ns4.google.com

Query one of them for the domain you chose, in this case google.ca:

karma@bzp ~ $ dig google.ca @ns1.google.com

; < < > > DiG 9.7.1 < < > > google.ca @ns1.google.com
;; global options: +cmd
;; Got answer:
;; -> > HEADER < <- opcode: QUERY, status: NOERROR, id: 64208
;; flags: qr aa rd; QUERY: 1, ANSWER: 5, AUTHORITY: 0, ADDITIONAL: 0
;; WARNING: recursion requested but not available

;; QUESTION SECTION:
;google.ca.                     IN      A

;; ANSWER SECTION:
google.ca.              300     IN      A       74.125.226.80
google.ca.              300     IN      A       74.125.226.81
google.ca.              300     IN      A       74.125.226.82
google.ca.              300     IN      A       74.125.226.84
google.ca.              300     IN      A       74.125.226.83

;; Query time: 68 msec
;; SERVER: 216.239.32.10#53(216.239.32.10)
;; WHEN: Sun Dec 19 20:05:29 2010
;; MSG SIZE  rcvd: 107

Note the presence of the aa flag. This indicates that we are looking at an authoritative answer from DNS servers configured to serve this zone. Now load the rules back in:

iptables -t nat -A PREROUTING -i eth1 -p tcp -m tcp --dport 53 -j REDIRECT --to-ports 53
iptables -t nat -A PREROUTING -i eth1 -p udp -m udp --dport 53 -j REDIRECT --to-ports 53

Try the query again:

karma@bzp ~ $ dig google.ca @ns1.google.com

; < < > > DiG 9.7.1 < < > > google.ca @ns1.google.com
;; global options: +cmd
;; Got answer:
;; -> > HEADER < <- opcode: QUERY, status: NOERROR, id: 48247
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 0

Note how the aa flag has been replaced by ra, indicating a recursive lookup.

Windows

On a windows host configured to use the ClearOS box as the gateway, open the command line and run:

nslookup

Find the DNS server for any given domain name. Set it in nslookup:

> server xxx.xxx.xxx.xxx

Now query the domain name:

> domain-name.com

You should see no trace of "Non-authoritative answer:" in the output. Now, add the rules again:

iptables -t nat -A PREROUTING -i eth1 -p tcp -m tcp --dport 53 -j REDIRECT --to-ports 53
iptables -t nat -A PREROUTING -i eth1 -p udp -m udp --dport 53 -j REDIRECT --to-ports 53

Do not change the DNS server nslookup is configured to query. Run the query again:

> domain-name.com

Immediately before the Name and Address(es) portion of the output you will see "Non-authoritative answer:" which indicates that your requests are being routed through dnsmasq.

Configuration: dnsmasq: Verifying Anonymity

Verifying that your DNS queries are being routed through Tor is somewhat more complicated than with web traffic. You will either need to run packet capture software (see my article on Remote Ethernet Packet Capture with Wireshark and tshark over SSH for ideas; on ClearOS run yum install wireshark to get tshark) or set up a name server that logs queries.

I used BIND because I already had that deployed. You don't need to have a name server on public address space - or even a registered domain to do this. Simply get a working BIND installation on an address which is reachable by the ClearOS box and configure its /etc/resolv.conf to do lookups against it after 127.0.0.1.

Include the following in your named.conf's logging block:

  channel request_log
  {
    file "/var/log/named/request.log" versions 3 size 5m;
    severity info;
    print-time yes;
    print-severity yes;
    print-category yes;
  };
  category queries
  {
    request_log;
  };

Now create and set the permissions on request.log:

# touch /var/log/named/request.log
# chown named: /var/log/named/request.log
# chmod 660 /var/log/named/request.log

Now reload the configuration and enable query logging:

# rndc reload
# rndc querylog

Make a request for a unique-looking subdomain of some zone your BIND server is configured to serve so you will be able to pick it out if you are using busy live servers. Otherwise you can:

# tail -f /var/log/named/request.log

to watch requests as they come through. If, according to this log, the request you made does not originate from your address(es) it is safe to assume they are leaving a Tor exit node.

System logs

It may suit one to take the system logs into account. On a traditional production system logging is important but if we're trying to make a disposable, anonymizing virtual machine it may be preferable to leave as few forensic traces as possible. One could disable syslog but some daemons write directly to file and may complain or fail to start if their logfiles are unwritable. We can satisfy everyone by loading a blank /log/ directory tree into tmpfs and deciding whether or not to disable syslog based on one's paranoia and available RAM.

First, we'll need to turn off syslog:

# /etc/init.d/syslog stop

Next, clear out the existing log files.

# yes | rm /var/log/*
# yes | rm /var/log/*/*

Create wtmp:

# touch /var/log/wtmp
# chown root:utmp /var/log/wtmp
# chmod 664 /var/log/wtmp

Add a symbolic link for dansguardian-av:

# ln -s /var/log/dansguardian /var/log/dansguardian-av

Now, move the sanitized directory elsewhere:

# mv /var/log /var/log.tree

Copy the log directory tree to tmpfs:

# cp -ax /var/log.tree /dev/shm/log

Create a symlink from tmpfs:

# ln -s /dev/shm/log /var/log

Now, if you choose, remove syslog from startup:

# chkconfig --level 2345 syslog off

There may be other logging considerations to take care of depending on your situation - such as snort packet logging, which is enabled by default.

We need to copy the new /log/ tree to tmpfs before anything else starts every time the system is booted, so let's create a very crude init script in /etc/init.d/logtree:

#!/bin/bash
cp -ax /var/log.tree /dev/shm/log

Now link it into runlevel 3:

# chmod +x /etc/init.d/logtree
# ln -s /etc/init.d/logtree /etc/rc.d/rc3.d/S05logtree

If you would like to reduce the risk of having your logs intercepted while the machine is live we can disable logrotate and add these lines to /etc/crontab:

*/05 * * * * root yes | rm /var/log/*
*/05 * * * * root yes | rm /var/log/*/*

Restart cron:

# /etc/init.d/crond restart

HTTPS

Due to the way TLS is implemented and proxies are designed to handle such requests it is not, to the best of my knowledge, feasible to transparently handle TLS connections. This presents a serious problem if users on your protected network will be signing in to HTTPS-protected sites as they will break anonymity.

One must block traffic to the port:

iptables -t nat -A PREROUTING -i eth1 -p tcp -m tcp --dport 443 -j DROP

If your client(s) will be using HTTPS they must now configure either privoxy or squid as the proxy in their browser(s). Privoxy will only be available if you chose to bind it to your LAN address, at port 8118. If you are not taking advantage of squid's cache this may provide some performance gain. Otherwise, configure the browser(s) to use port 3128 (Squid) or 8080 (DansGuardian).

Failing Gracefully

Rather than block HTTPS traffic altogether we can redirect TLS requests to a local web server on the ClearOS box that shows a page instructing users to configure their browsers to use one of the available proxies. This costs resources (mostly RAM) but can save your client's calling you for support. You need to have installed the Web server module as outlined in Configuration: ClearOS: Installing the Software.

Log into webconfig at https://lan-ip:81/ . Click the Web Server link in the Server menu. Click the Start and Automatic buttons. Make sure SSL is enabled. Now add this to your firewall configuration:

iptables -t nat -A PREROUTING -i eth1 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 443

You may choose to replace or edit the default page that comes with ClearOS:

# nano /var/www/html/index.html

Edit the blurb to include details on configuring browsers to use one of the proxies you have chosen to end the daisy-chain with. I thought the ClearOS template was so cute I kept it:

...

<body>

<div id='dialogintro1' class='sb'>
<p class='blurb'>
Sorry, you can't make TLS/SSL connections without configuring your browser to use this proxy:<br />
<strong>Server</strong>192.168.1.1<br />
<strong>Port</strong>8080
</p>
<p align='center'><a href='http://www.clearfoundation.com/'><img src="logo.png" alt="Powered by ClearOS"></a></p>
<br>
</div>

<script type='text/javascript'>
        dialogbox.render(dialogintro1);
</script>

</body>

Now when a user behind the ClearOS box attempts to make an SSL connection their browser will complain about an invalid certificate. If they accept it they will be told why SSL isn't working and what to do about it.

Acknowledgements

I was first inspired to do something like this when I read sprawl's Tor Control Protocol article in the Winter 2009-10 Edition of 2600.

These articles helped along the way:

A special thank you to the countless developers who have contributed in one way or another to the software involved in this article.

Virtual Machine Image

DOWNLOAD clear-trollup.tar.lzma (342MB) c9a56f4b55084e97b8a54f54aff1f512

This file is a sparse, raw full-disk image which can be used natively by some virtualization platforms (Xen, QEMU) or converted for use by others (VirtualBox, VMware). It is based on the ClearOS 5.2 foxpa.ws HVM Edition.

Note to Windows Users: 7zip (and probably others) has a problem handling sparse tar files. Download the gnuwin32 version of tar for the command line at ??http://gnuwin32.sourceforge.net/packages/gtar.htm

c:\Users\karma\Desktop>"c:\Program Files\GnuWin32\bin\tar.exe" xf clear-trollup.tar

Modified from the original README:

Caveats and considerations:
 - comes with iftop and tshark
 - custom snort rules with SIDs beginning with 4000000
 - http://foxpa.ws/2010/06/04/bad-snort-rules/ applied
 - user username is user, password is user (must SU on physical console)
 - root password is root
 - user password is user; log in as user then SU to root on PTYs
 - automatic updates are disabled by default
 - graphical console disabled
 - eth0 ext DHCP
 - eth1 lan 192.168.254.1/24
 - admin https://192.168.254.1:81
 - grant users shell access in webconfig enabled

The VM uses all of the proxies covered in this article and fails SSL connections gracefully. Unfortunately, this requires that the VM has on the order of 400MB of RAM to work with. Remember to update IP/port binding directives and netfilter to reflect your new configuration should you drop one or more proxies from the chain. On-the-fly virus-scanning is enabled by default and logs are written to tmpfs at /dev/shm/tmfs.

Suvad has been disabled from startup, it must be re-enabled and started before you will be able to register your VM with ClearSDN:

# chkconfig --level 2345 suvad on
# /etc/init.d/suvad start

If you encounter issues registering your system try loading this URL in your browser twice:

https://129.168.254.1:81/admin/register.php?Reset=yes

The LDAP system may stall on startup if the VM is unable to resolve DNS queries. Ensure that the external interface is configured correctly (edit /etc/sysconfig/network-scripts/ifcfg-eth0) and /etc/resolv.conf lists working name servers before you start the VM. The VM is configured to acquire a DHCP address for its external interface from your existing router/DHCP server and uses two public name servers by default.

If you are connecting the internal and external interfaces to distinct Layer 2 networks (i.e. using it as an actual gateway) you may wish to remove these lines from /etc/sysctl.conf:

net.ipv4.conf.default.arp_filter=1
net.ipv4.conf.all.arp_filter=1

To mount the partition inside the full disk image use kpartx:

# kpartx -a clear-trollup.hdd

Find the partition's device node in /dev/mapper and mount it:

# mount /dev/mapper/loop1p1 /mnt/rawroot

Once you have unmounted the partition remove the loop device:

# kpartx -d clear-trollup.hdd

kpartx is a part of the miltipath-tools package. On Gentoo run:

# emerge miltipath-tools

Your kernel must support Device Mapper.

After you have configured your VM it is a wise idea to generate new keys for SSH. run:

# rm /etc/ssh/ssh_host_*
# /etc/init.d/sshd restart

Virtual Machine Image: Xen Paravirtualized

You will require a PAE or 64-bit compatible hypervisor and dom0 to run this image paravirtualized. The first time the machine is created you must use the -c flag to see the pyGRUB bootloader or mx will complain about an invalid default kernel having been specified. You will have 5 seconds to choose the domU kernel and initrd combo. Once logged into the VM edit /boot/grub/grub.conf to boot the domU configuration by default.

This is the sample configuration that accompanies the image:

# ClearOS 5.2 foxpa.ws TROllup edition Jan 2011 http://foxpa.ws/

name = "clear-trollup"

vcpus = 1
memory = 512
vif = [ 'mac=00:16:3e:ff:00:01,bridge=xenbr0', 'mac=00:16:3e:ff:00:02,bridge=xenbr0' ]
disk = ['file:/var/xen/clear-trollup/clear-trollup.hdd,hda,w']

bootloader = "/usr/bin/pygrub"
root = "/dev/hda1 ro"
extra = "xencons=ttyS"

# REMEMBER update your /boot/grub/grub.conf to load the domU kernel automatically

Be sure to update the MAC addresses and file paths.

Virtual Machine Image: Hardware Virtualization

The regular (HVM) kernel entry in grub.conf has the divider=10 RHEL-specific kernel command line argument to reduce the default timer frequency from 1000Hz to 100Hz.

HVM users will probably encounter a recurring error message on the main PTY indicating that agetty is re-spawning too fast on /dev/ttyS0. Open /etc/inittab and comment this line:

s0:12345:respawn:/sbin/mingetty ttyS0

Tor's init script may complain when it goes to enable the independent clock feature specific to Xen PV before setting the time with ntpdate. Comment this line in /etc/init.d/tor:

echo 1 > /proc/sys/xen/independent_wallclock

Virtual Machine Image: Hardware Virtualization: VirtualBox (Windows)

On Linux, and with older versions of VB I recall being able to use raw disk images unmodified. It seems that this is no longer the case so before you set up your virtual machine we will have to convert the disk image into a suitable format.

On *nix run:

$ VBoxManage convertfromraw clear-trollup.hdd clear-trollup.vdi --format VDI

On Windows:

C:\Users\karma>cd "c:\Program Files\Oracle\VirtualBox"
c:\Program Files\Oracle\VirtualBox>VBoxManage.exe convertfromraw clear-trollup.hdd clear-trollup.vdi --format VDI

Open VirtualBox then click on the New button to start the New VM Wizard.

Give the VM a name, set the OS to Linux and the Version to Red Hat (32 bit).

Click next and specify a reasonable amount of RAM for the VM. With all proxies used, antivirus scanning, HTTPS graceful failure and IDS/IPS the VM will need up to 400MB. Click Next and you will be prompted to locate the disk image.

Click Next then Finish and you will exit the Wizard.

Select the VM and click the Settings button. You may wish to reduce the amount of video memory under the Display tab. Click on the Storage tab. Remove clear-trollup.vdi from the SATA controller, then add it to the IDE controller:

Click on the Network tab. Change the Attached to: drop-down to Bridged Adapter. Select the NIC you will be putting the external interface of the VM on.

Click on the Adapter 2 tab. Enable the adapter and attach it to a Bridged Adapter. If you will be using the VM to protect only the PC it is installed on or other hosts on the same layer 2 network with an existing router you may choose to bridge this interface to the same physical NIC as Adapter 1. If you will be using the VM as your subnet's primary router/firewall Adapter 1 must be connected to the uplink NIC (i.e. plugged into your cable modem) and Adapter 2 should connect to the local layer 2 network (i.e. switch).

Click on the Serial Ports tab and check the Enable Serial Port box to avoid annoying respawning messages due to a Xen-related modification to /etc/inittab. Click OK to exit the settings window and start the virtual machine.

Once init has finished its run change the IP address of the physical adapter you have bridged to Adapter 2 (eth1) to something on the 192.168.254.0/24 subnet:

In your browser navigate to https://192.168.254.1:81 and log in with the username root and password root. Go to IP Settings under the Network menu and update your configuration if necessary (i.e. provide a static IP for eth0). Continue to set a new password, user accounts and so on. Reboot the virtual machine once you have reconfigured it to your specifications.

Load up your web browser and navigate to, say, http://whatismyip.org/. If the IP does not match your public address congratulations, you have successfully configured your VM.

Dedication

This article and VM are dedicated to Bradley Manning, who reminded us all that no matter how creative your OPSEC is mouthing off to some asshat will probably get you nailed.

Fix: Multiple NICs on Same Layer 2 Network Broadcast All MACs on ARP request

• karma

By default when a Linux host is connected to the same layer 2 network by two network interfaces (i.e. plugged into the same or connected switch (and VLAN) or attached to a bridge in similar configuration) ARP requests for any IP on the NICs in question will produce a single response for every interface's MAC address. Obviously, this poses a problem called ARP flux where an IP may seem to migrate from MAC to MAC.

This example uses a DHCP server of similar stock to the one covered in Transparent Proxy for Hot Spot/Public Network Web-Based Authentication on ClearOS . It has an external interface (192.168.222.22) and an internal interface (192.168.111.1) on which DHCP clients are collected. Since we are collecting DHCP users then routing them through another private network both interfaces are connected to the same switch. When an ARP request for either IP is made both will respond at first. Observe:

bzp ~ # arping 192.168.222.22 -I br0
ARPING 192.168.222.22 from 192.168.222.53 br0
Unicast reply from 192.168.222.22 [00:16:3E:22:00:01]  0.950ms
Unicast reply from 192.168.222.22 [00:16:3E:22:00:02]  0.987ms
Unicast reply from 192.168.222.22 [00:16:3E:22:00:02]  0.930ms
^CSent 2 probes (1 broadcast(s))
Received 3 response(s)
bzp ~ # arping 192.168.111.1 -I br0
ARPING 192.168.111.1 from 192.168.111.99 br0
Unicast reply from 192.168.111.1 [00:16:3E:22:00:01]  0.946ms
Unicast reply from 192.168.111.1 [00:16:3E:22:00:02]  0.981ms
Unicast reply from 192.168.111.1 [00:16:3E:22:00:02]  0.887ms
Unicast reply from 192.168.111.1 [00:16:3E:22:00:02]  0.891ms
Unicast reply from 192.168.111.1 [00:16:3E:22:00:02]  0.953ms
^CSent 4 probes (1 broadcast(s))
Received 5 response(s)

We can correct this by adding the following lines to /etc/sysctl.conf:

net.ipv4.conf.default.arp_filter=1
net.ipv4.conf.all.arp_filter=1

Now run:

# sysctl -p

The issue should be corrected:

bzp ~ # arping 192.168.111.1 -I br0
ARPING 192.168.111.1 from 192.168.111.99 br0
Unicast reply from 192.168.111.1 [00:16:3E:22:00:02]  1.171ms
Unicast reply from 192.168.111.1 [00:16:3E:22:00:02]  0.991ms
^CSent 2 probes (1 broadcast(s))
Received 2 response(s)
bzp ~ # arping 192.168.222.22 -I br0
ARPING 192.168.222.22 from 192.168.222.22 br0
Unicast reply from 192.168.222.22 [00:16:3E:22:00:01]  0.919ms
Unicast reply from 192.168.222.22 [00:16:3E:22:00:01]  0.980ms
^CSent 2 probes (1 broadcast(s))
Received 2 response(s)

Note that one of the MACs will "win out" after the first broadcast because the target machine will start sending unicast responses from the correct interface; if you are consistently seeing multiple MACs responding on every ARPing it is safe to say you are probably looking at an IP address conflict instead.

BotHunter Headless on ClearOS with TOR

• karma

I was inspired to play with BotHunter by this article: ?https://www.whataboutbob.org/public/?p=102

I'm not fond of running GUIs on firewalls (let alone virtual machines) and I've been writing about using TOR with ClearOS for an upcoming article so we will be building upon rstangarone's article today. There is nothing stopping you from installing what you need to, say, tunnel X through SSH but that goes beyond the scope of this article.

You may wish to set this up in one of my pre-installed ClearOS Virtual Machines before implementing it on production systems: paravirtualized Xen, hardware emulators.

As put forward in the BotHunter documentation:

BotHunter is the first, and still the best, network-based malware infection diagnosis system out there. It tracks the two-way communication flows between your computer(s) and the Internet, comparing your network traffic against an abstract model of malware communication patterns. Its goal is to catch bots and other coordination-centric malware infesting your network, and it is exceptionally effective.

I hope their money is where their mouth is. Interestingly:

Optional: You are prompted to install Tor if it has not been installed previously. BotHunter may be configured to use Tor to interact anonymously with the BotHunter repository services.

BotHunter is based on what is described in the documentation as a heavily customized Snort implementation. I toyed for some time with the thought of unifying ClearOS's Intrusion Prevention/Detection System implementation with it to conserve resources but ultimately decided time was better spent doing other things and swallowed the hit. You may feel differently however, if you succeed please drop me a line. The BotHunter installer will attempt to compile its Snort from source; ensure that the ClearOS build environment has been installed:

# yum install "Developer Tools"

Let's find then install the latest version of TOR for ClearOS. You will first need libevent:

# yum install libevent

If you will be using the torify command you will need to install tsocks:

# rpm -iv ftp://ftp.univie.ac.at/systems/linux/dag/redhat/el5/en/i386/dag/RPMS/tsocks-1.8-7.beta5.2.el5.rf.i386.rpm

Now TOR (adjust for the latest version):

# rpm -iv http://deb.torproject.org/torproject.org/rpm/centos5/tor-0.2.1.28-tor.0.rh5_5.i386.rpm

Make TOR start on boot:

# chkconfig --level 2345 tor on

Tor is configured by default to listen to port 9050 on localhost. If you would like hosts on the private network to be able to route connections through this deployment you may add a second SocksListenAddress directive, set to your router's LAN address in /etc/tor/torrc. Edit the /etc/tor/tor-tsocks.conf file on your client machines to route connections made via torify to the LAN address. Note that you should only do this for private addresses otherwise you may be opening an inadvertent public proxy.

Enable the control port by uncommenting this line in /etc/tor/torrc:

ControlPort 9051

Run this command, replacing password with the password you would like to use for the controller:

tor --hash-password password

Ignore the warning about running TOR as root; the init script has been configured by the RPM to use a non-privileged account. Uncomment and change the HashedControlPassword to the output you were provided. Start TOR:

# /etc/init.d/tor start

Before we install BotHunter we must provide ClearOS with a java environment:

# yum install java

Now let's download the latest version of BotHunter. We'll be following the installation instructions available at http://www.bothunter.net/doc/users_guide-UNIX.html. Go to http://www.bothunter.net/download.html and fill out the form to get your download link.

# wget {your download link}
# tar zxf {your filename}
# cd BotHunter/

Here's the README that came with mine:

[root@router BotHunter]# cat README.txt 

            BotHunter(*) Internet Release
              www.bothunter.net
              Unix Version 1.5.0
              February 25, 2010

* HARDWARE RECOMMENDATIONS

  Your system should have a modern Intel Pentium-class or
  Motorola PowerPC processor, at least 1 GB RAM, and at least
  1 Ethernet NIC/WIC for network monitoring.

* OS AND SOFTWARE REQUIREMENTS

  BotHunter is available for use on the following operating
  systems:

  Linux:    tested on Fedora, Red Hat Enterprise Linux, Debian,
            and SuSE distributions

  FreeBSD:  tested on Product Release 7.0

  MacOS 10: tested on Tiger and Leopard, Mac OS 10.4 and 10.5

  BotHunter requires a Sun-compatible Java Runtime Environment
  (JRE) Release 1.5 or later.

  Linux: the Linux distribution of Sun's Java JRE is available at

           http://java.sun.com/javase/downloads/index.jsp

  Mac OS:  for Mac OS X, Xcode must be installed on your system;
           it may be obtained from

           http://developer.apple.com/tools/xcode/

  FreeBSD: for installing a recent version of Java, we recommend
           that you consult

           http://www.freebsd.org/java/

* NETWORK REQUIREMENTS

  Installation requires Internet connectivity for downloading the
  necessary libraries, packages, and BotHunter ruleset updates.
  Your target platform should have a promiscuous mode tap, such as
  a span port or access to broadcast LAN traffic.  Ideally, your
  machine should be attached to a monitoring position on an internal
  network egress point to observe successful connection flows (e.g.,
  behind the firewall) between your internal hosts and external
  entities.

* INSTALLATION

  The following is a summary of the minimum steps necessary to install,
  configure, and start BotHunter, in its default configuration for live
  traffic monitoring.  This installation procedure should be performed
  by the root user.  You will also need to know the IP address netmask
  of the network you wish to protect, and the IP addresses of your
  email and DNS servers.

  BotHunter's installation process will NOT upgrade a previous
  installation. If you have a previous installation of BotHunter, you
  should remove the previous user installation or install BotHunter
  into a different user account.

  While installation requires root privilege, BotHunter does not
  require root privilege to run.  Instead, this installation creates
  a nonprivileged user account that runs BotHunter.

  Note: you may type '?' at any prompt for a detailed explanation of
  what is expected.

  1.  Untar the BotHunter Unix distribution.

  2.  Begin the root installation procedure.

      root% java -jar botHunterInstall.jar

      Read the EULA and if acceptable click YES.

  3.  Confirm that you wish to perform this root install.

  4.  Optional: You are prompted to install Tor if it has not been
      installed previously.  BotHunter may be configured to use Tor
      to interact anonymously with the BotHunter repository services.

  5.  Indicate the new nonprivileged user account with which you wish
      to install BotHunter (default user account = cta-bh). BotHunter
      will then install dependent packages. If you choose to install
      BotHunter over a preexisting user account, this account must
      use csh(1).

  6.  Enter your Trusted Network Mask: Provide a (comma separated)
      local network mask list, plus the IP addresses of all external
      NetBIOS shares with which your internal machines are allowed to
      communicate.

      example: 192.168.1.0/24,10.10.0.10/16

  7.  Enter the (comma separated) IP addresses of the email server(s)
      used by systems inside your network.

  8.  Enter the (comma separated) list of DNS servers used by systems
      inside your network.

  9.  Enter your network interface that BotHunter will use to monitor
      your network.

  10. Indicate whether you wish BotHunter to start automatically on
      reboot.

  11. Optional: As a last step, you may now set user cta-bh's password:

      root% /usr/bin/passwd cta-bh

  12. su to the user account that you created during the BotHunter
      installation:

      root% su -l cta-bh

  14. To set up BotHunter in its default configuration (LIVEPIPE
      mode), use the BotHunter shell alias:

      cta-bh% BotHunter

* CONFIGURING AND OPERATING BOTHUNTER

  You are now ready to configure and operate BotHunter. Please read
  the BotHunter Unix User's Guide, available at

  http://www.bothunter.net/doc/users_guide-UNIX.html  

  for details on how to configure and operate BotHunter.

  The User Guide is also available under the doc directory of this
  installation package.

* GOOD LUCK

  Thank you for your interest and support.

  BotHunter Developers: Phillip Porras, Martin Fong, Keith Skinner,
  Steven Dawson, Vinod Yegneswaran, Guofei Gu.

----------------------------------------------------------------------
(*) BotHunter is a U.S. Registered Trademark of
    SRI International, 333 Ravenswood Ave., Menlo Park, CA 94025

We'll need to make a non-privileged user account and working directory for BotHunter to run under.

# mkdir /opt/bh
# useradd -d /opt/bh -s /sbin/nologin -r bh
# chown bh: /opt/bh

Now run the installer:

java -jar botHunterInstall.jar

If you chose not to start BotHunter automatically at the end of the installation procedure run:

# /etc/init.d/zzzBotHunter_bh start

Check on the status of BotHunter:

# cd /opt/bh/BotHunter/LIVEPIPE_CONFIG
# sudo -u bh java -jar ../botHunterInstall.jar status

Note that the paths must be the same as used above. You will only be able to query the daemon if you connect to it as the user it is running under.

[root@router LIVEPIPE_CONFIG]# sudo -u bh java -jar ../botHunterInstall.jar status
Issuing net query signal.

CTA BotHunter 1.5.0 status #2 as of 2011/01/13 18:42:46 EST
  Process elapsed time:               0 00:08:12
  Memory usage:                       43880 Kbytes
  Input events read:                  2
  Input events parsed:                2
  Local text BotHunter profiles:      1
  NetQuery requests made:             2
  NetQuery responses received:        1
  Repository messages queued:         1
  Messages sent to repository:        2
  Sensor connected to repository:     true
  Most recently seen author ID:       ***********
  Most recently seen observer ID:     ********
 
CTA BotHunter: Process is active.

Now let's add an alias to our ~/.bashrc:

alias cwdBotHunter='sudo -u bh java -Xmx104m -jar /opt/bh/BotHunter/botHunterInstall.jar'
alias BotHunter='cd /opt/bh/BotHunter/LIVEPIPE_CONFIG; cwdBotHunter'

Start a new bash instance (log out/log in or run 'bash'). You should now be able to run

BotHunter status

as root from any location.

I want e-mail notifications so let's open /opt/bh/BotHunter/LIVEPIPE_CONFIG/CTA_BotHunter/CTA_BotHunter.config and add these lines:

# ----------------------------------------------------------------------
# e-mail parameters
[email protected]
mailHost=xxx.xxx.xxx.xxx
[email protected] 
mailSubject=BotHunter Profile %m(score) %df(yyyyMMdd_HHmmssSSS)
mailSubjSubs=true

Save the file then restart BotHunter to apply your changes:

# /etc/init.d/zzzBotHunter_bh restart

That's all, folks!

ISC Contacts L'il Ol' Me!

• karma

Someone from ISC contacted me a couple days ago regarding the ongoing DRDoS attack one of my client's DNS servers has been involved in for months, see the comments section of ISC.org DRDoS Update 2: Problems with .nl Netherlands TLD. I was quite surprised to hear from them and now wonder if perhaps the more responsible thing to do would have been to contact them from the outset. At any rate I dropped the netfilter rules very briefly this morning to obtain a fresh sample of packets. Naturally, my client was not entirely thrilled with the idea. Here's the nice letter I sent them:

Hello,

I was pleasantly surprised to see someone from ISC asking for
information pertaining to an ongoing DRDoS attack against one of my
client's servers at
http://foxpa.ws/2011/01/03/isc-org-drdos-update-2-problems-with-nl-netherlands-tld/

I will be happy to provide you with what I know, sample packet data
and cooperation in implementing and testing suggestions.

I think I am looking at a (D)RDoS that is using spoofed ARP-based ANY
queries for isc.org with the intent to obfuscate its source and
amplify its payload. The following series of links more or less
documents the evolution of my theory and how I have tried to contain
the attack:

http://foxpa.ws/2010/07/20/making-the-case-for-access-controlled-recursive-lookups-with-bind/
http://foxpa.ws/2010/07/21/thwarting-the-isc-org-dns-ddos/
http://foxpa.ws/2010/11/17/isc-org-any-request-drdos-update/
http://foxpa.ws/2011/01/03/isc-org-drdos-update-2-problems-with-nl-netherlands-tld/

Please find attached a libpcap formatted file with fresh packets
sampled this morning (the filter was adjusted to only record ANY
transactions). Please remove any identifying marks from the packets if
you distribute them.

It may be noteworthy that the bursts of traffic seem to be focused on
the daytime (EST) hours and generally dwindle down at night.

Hopefully this has helped, please do not hesitate to get in touch with
me if I can be of further service.

Cheers,

K

It would be fantastic if the whiz kids over there could come up with a better idea of dealing with this! Fingers crossed. =)

ISC.org DRDoS Update 2: Problems with .nl Netherlands TLD

• karma

I provided a netfilter rule in my last update that indiscriminately drops packets containing the string "isc.org" on port 53/UDP which I have had to implement on a client's network due to one of their name servers being targeted as an amplifier in a UDP bandwidth-starvation DRDoS attack. I was afraid this sort of blunt-object type of approach might have unforeseen ramifications when I wrote a snort rule of similar design and today one has reared its ugly head. It turns out this string is part of the payload in transactions with (at least) the .nl root servers. This would not normally be much of a problem for a small Canadian network but for the fact the registered name servers for kijiji.ca (a popular free classifieds site) are all under .nl.

As you can see, my friends haven't taken the hint and buggered off to greener pastures yet; the spike correlates with my removing and replacing the netfilter rule. You'll notice the inbound stays steady after the outbound drops off, that's because the packets are only being dropped at the name servers rather than the gateway as usual (I double-wrap for his pleasure).

I have a number of options and currently I find none of them particularly satisfactory. For the interim the solution has been to provide clients with a secondary name server off the network. More to come?