Archive for May, 2012

gen_initramfs_list.sh: Cannot open ‘/usr/share/v86d/initramfs’

If you encounter this error while compiling a newer kernel:

  CC      init/calibrate.o
  LD      init/built-in.o
  HOSTCC  usr/gen_init_cpio
  /usr/src/linux-3.2.12-gentoo/scripts/gen_initramfs_list.sh: Cannot open '/usr/share/v86d/initramfs'
make[1]: *** [usr/initramfs_data.cpio] Error 1
make: *** [usr] Error 2

Emerge v86d:

# emerge v86d

Credit due: big my secret: Problem of make kernel about v86d initramfs

Delete All Entries for a Given Criterion in ip_conntrack Table

You may find yourself in a position where it is necessary to remove all the entries in Netfilter’s connection tracking table (ip_conntrack) for a particular criterion, like the source or destination IP.

For example, I recently detected a user on one of my networks engaged in what was likely a TCP denial of service attack against root name servers (despite the odd fact that the destination port was 80). Being a NATted user, all of their connections were being tracked. The default time-out for tracking an established connection being 5 days, simply disconnecting the user at the second layer would not relieve the congestion on my routers within an acceptable time frame.

#  cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count
98636

#  cat /proc/net/ip_conntrack | grep "xxx.xxx.xxx.xxx"
tcp      6 416408 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.5.147 sport=58967 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.5.147 dst=yyy.yyy.yyy.yyy sport=80 dport=58967 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 416406 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.9.239 sport=58967 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.9.239 dst=yyy.yyy.yyy.yyy sport=80 dport=58967 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 416400 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.11.231 sport=58968 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.11.231 dst=yyy.yyy.yyy.yyy sport=80 dport=58968 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 416387 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.14.37 sport=58968 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.14.37 dst=yyy.yyy.yyy.yyy sport=80 dport=58968 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 416381 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.11.103 sport=58968 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.11.103 dst=yyy.yyy.yyy.yyy sport=80 dport=58968 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 416275 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.9.57 sport=58967 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.9.57 dst=yyy.yyy.yyy.yyy sport=80 dport=58967 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 415776 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.1.52 sport=58967 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.1.52 dst=yyy.yyy.yyy.yyy sport=80 dport=58967 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 417319 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.43.60 sport=58967 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.43.60 dst=yyy.yyy.yyy.yyy sport=80 dport=58967 packets=0 bytes=0 mark=0 secmark=0 use=1
tcp      6 417319 ESTABLISHED src=xxx.xxx.xxx.xxx dst=192.168.43.13 sport=58968 dport=80 packets=1 bytes=40 [UNREPLIED] src=192.168.43.13 dst=yyy.yyy.yyy.yyy sport=80 dport=58968 packets=0 bytes=0 mark=0 secmark=0 use=1

...

It is possible to clear tracking entries en masse by removing then reloading the iptables rule that requires them to be tracked in the first place, but on a production gateway this is even less acceptable than waiting for them to expire. Fortunately, we can interact with the ip_conntrack table via conntrack-tools.

Against common sense, conntrack-tools is not available in the repositories for (at least version 5.2) of ClearOS, my favourite router distro. I grabbed a couple recent versions in RPM form but they didn’t feel like playing ball so I ended up with version 0.9.5 from ftp://ftp.pbone.net/mirror/archive.fedoraproject.org/fedora/linux/updates/8/i386.newkey/conntrack-tools-0.9.5-3.fc8.i386.rpm

Apparently newer versions allow one to use -D intuitively, i.e.:

# conntrack -D -s xxx.xxx.xxx.xxx

But this is not the case for at least versions including and prior to 0.97 – these require the d, dport, s and sport flags.

This wonderful person provides a way to pipe the output of conntrack -L (which lists entries the way I’d like to delete them, i.e. -s only) into sed which then breaks the output lines up and awk runs them with conntrack -D appropriately. I had to do some cleanup to get it to work due to the way their blog software mangles punctuation (a lot of my first posts here are mangled in the same way – pobody’s nerfect!):

 conntrack -L -s xxx.xxx.xxx.xxx | sed 's/=/ /g'| awk '{system("conntrack -D -s "$6" -d "$8" -p "$1" --sport="$10" --dport="$12)}'

It should be pretty clear how conntrack -L -s can be modified to work with the destination address or more complicated pattern matching.

Now we can see the ip_conntrack table is at a more reasonable level:

[root@router ~]#  cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count
46676

ip_conntrack: table full, dropping packet.

Connections in to and out of your network are working sporadically. Your router’s dmesg is flooded with “ip_conntrack: table full, dropping packet.” What do you do?

This condition occurs when the connection tracking table has reached its limit. Connection tracking is a function of Netfilter that stores information like the source and destination IP addresses, port numbers, protocol type, state and timeout of a two-way connection. This facility lets us create sophisticated and informed Netfilter rules in a way that is not possible to accurately derive on a packet header-by-header basis.

The conntrack table takes the form of a memory structure; if there were no constraints on the size of the table it could conceivably start knocking off userspace processes if it became too large (i.e. under DoS conditions). Entries in the conntrack table expire either when their timeout has been reached or the connection has been properly closed. In cases where connections are not being closed according to protocol (poor network connectivity, DoS, spoof attack, etc.) the table can fill rapidly causing an intermittent denial of service condition on your network.

The most prominent symptom of a full connection tracking table is that your old, running connections (secure shell sessions) will continue to function while it becomes impossible to establish new ones. Worse, as the entries continue to time out and the table keeps filling up you may “get lucky” and establish a new connection here and there, making the situation much more confusing.

Depending on the situation you may have one or two options. If you have gobs and gobs of RAM available or the (for example) attack is low-volume you can adjust the entry limit of the table. First, check what the current limit is:

# cat /proc/sys/net/ipv4/ip_conntrack_max
65536

You can see how full the table currently is by running:

# cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count
62168

ip_conntrack_max is determined as a multiple of how much RAM the system boots up with but generally stops at 65536 regardless. You may find that this isn’t even enough for a high volume network under normal conditions. We can adjust the limit temporarily thus:

# echo 131072 > /proc/sys/net/ipv4/ip_conntrack_max

If this turns out to be your magic bullet and you’re sure no other actions need to be taken to mitigate your particular situation add the following line to /etc/sysctl.conf:

net.ipv4.netfilter.ip_conntrack_max = 131072

To load the value from sysctl.conf run:

# sysctl -p

If you don’t have the option of throwing more RAM at the problem you may be forced to make an executive decision in the interest of preserving network services for legitimate clients. You can decrease the load on the conntrack table by removing rules that use stateful logic (i.e. containing “-t nat” or “-m state”). The brute force option is to rmmod the ip_conntrack module:

# rmmod ip_conntrack

However this may not be possible in all environments. The other option is to flush your rules and set the default policy to allow:

# iptables -P
# iptables -F

This is also typically the effect of

# /etc/init.d/iptables stop
or
# /etc/init.d/firewall stop
Return top
foxpa.ws
Online Marketing Toplist
Internet
Technology Blogs - Blog Rankings

Internet Blogs - BlogCatalog Blog Directory

Bad Karma Networks

Please Donate!


Made in Canada  •  There's a fox in the Gibson!  •  2010-12