pitter patter on the keyboard

There appears to be a benign segfault from opendkim when Zimbra is shut down:

opendkim[4057]: segfault at 1f0 ip 0000003044209220 sp 00007fffa235bd48 error 4 in libpthread-2.12.so[3044200000+17000]
opendkim[10772]: segfault at 1f0 ip 0000003044209220 sp 00007fffbd4e47a8 error 4 in libpthread-2.12.so[3044200000+17000]

The only solid reference to this I could find was at http://www.zimbra.com/forums/administrators/58839-mta-logger-logswatch-wont-start-also-opendkim-issue.html. These users appear to be using CentOS 6.3 as well which may indicate the problem is with libpthread and not in fact opendkim.

After dealing with kernel panics (from what I am hoping was merely OOMs) by moving my new Zimbra VM to a different box and giving it 4 cores and 4 gigs of ram and 4 gigs of swap I was about 80 accounts deep into my second Courier-IMAP Maildir Migration when it started throwing this error:

service.FAILURE (system failure: ZimbraLdapContext)

dmesg shows:

[ 1299.261973] slapd[10678]: segfault at 444d2d54 eip b778f154 esp b1587e28 error 4

Upon restarting Zimbra everything seems to be fine. Unfortunately, the one reference to this error I have found (http://wiki.zimbra.com/wiki/Uninstall_Instructions_for_Unix_and_Windows_Account_Management_in_Admin_UI) states:

Once this is done, you’re almost out of the woods, but this last step is very, very important. You MUST run slapindex to update the indexes in your Zimbra LDAP database, or you run the risk of having segfault/protection errors that crash the slapd process. So far, this has mostly been observed using Zimbra Network Edition running on Ubuntu 8.04 Server LTS. The exact command is ‘/opt/zimbra/openldap-2.3.43.10z/sbin/slapindex’. The command will probably throw you an error message about “loglevel”. Open the referenced slapd file and temporarily change the log-level to an actual number (49152 is what I usually set it to). Then revert that change after slapindex has run. Start Zimbra again via ‘zmcontrol start’ and watch the processes for about 30 minutes to make sure nothing is amiss. If you get slapd errors, run slapindex again, it’s usually the cause of the problem.

The article has very little to do with what I’m trying to accomplish, but given the versions of Ubuntu and Zimbra I think it’s a fair bet that either/and:

• We’re having the same problem anyway
• Running slapindex might solve my problem too

Unfortunately, the migration moves quite slowly so adding slapindex to the script is out of the question.

I’ll be giving ZCS 8 a shot on RHEL 6 tomorrow. Hooray. >.>

I noticed this nasty wee bugger last week:

[notice] seg fault or similar nasty error detected in the parent process

The first time it happened the drop in traffic did not correlate with the daily init scripts but when the second occurance this morning did I finally had somewhere to start. The default logrotate script for apache on gentoo (and many other flavours) does a graceful reload once the log files have been rotated to release their inodes. Now I could reproduce the issue:

# /etc/init.d/apache2 reload
 * Gracefully restarting apache2 ...
# /etc/init.d/apache2 restart
 * Stopping apache2 ...
httpd (pid 2505?) not running                                                [ ok ]
 * Starting apache2 ...
 * start-stop-daemon: /usr/sbin/apache2 is already running
# /etc/init.d/apache2 stop   
 * Stopping apache2 ...
httpd (pid 2505?) not running
# /etc/init.d/apache2 start
 * Starting apache2 ...
 * start-stop-daemon: /usr/sbin/apache2 is already running

Poking around this seems to usually be caused by loading bad/wrong/nonexistent libraries in the php.ini – and I am guilty of carrying over old php.inis on upgrade. Updating and switching to the distributed php.ini for my installed version of php seems to have solved this issue.

Sometimes it really is my fault :p