Import Courier-IMAP Maildir E-Mail and IMAP Folders to Zimbra
I have the good fortune of being forced to migrate over 1,100 e-mail accounts from a 15 year old qmail server with Courier-IMAP.
A script is provided at http://wiki.zimbra.com/wiki/Courier-IMAP_Maildir_to_zmmailbox. It should be easy to run as root and import over NFS. Unfortunately, I ran into trouble with the --noVerify flag, possibly due to the older version of Zimbra I'm importing to:
* Running import process... 0 ...some messages did not import correctly: check /tmp//import-example.com-user--6108
The output of the log looks like:
addMessage --flags "u" --noValidation "/Inbox" "/mnt/example.com/user/Maildir//cur/1362778133.14175.smtp.example.com:2,"
To test it you're going to have to look at the prefix in the script:
/opt/zimbra/bin/zmmailbox -m [email protected] -z addMessage --flags "u" --noValidation "/Inbox" "/mnt/example.com/user/Maildir//cur/1362778133.14175.smtp.example.com:2,"
Which gave me the following error:
ERROR: zclient.CLIENT_ERROR (unknown folder: --noValidation)
I tried putting the --noValidation flag in every position with no luck, and it is listed in the Zimbra wiki article for zmmailbox at http://wiki.zimbra.com/wiki/Zmmailbox so I am left to assume my version simply does not support it. It doesn't seem to be consequential anyway so I simply removed it from the script.
#!/bin/bash # # courier/vpopmail Maildir to Zimbra Import # # This script can be stored anywhere, but you should run it while in the root # of the domain's users. It looks for the file vpasswd which contains a # line-separated list of users and uses that to import. You can also run the # script with a user name to process a single user. Additionally, you can # specify a folder name (courier format) to process a single folder for that # user. # We assume the folder structure is like this: # Inbox: <working directory>/<user>/Maildir/<cur|new> # Subfolder: <working directory>/<user>/Maildir/.Subfolder/<cur|new> # If this is not what your structure looks like, you need to change the # "folderpath" variable construction down further in this script. # This is the command to run to run mailbox commands. ZMCMD='/opt/zimbra/bin/zmmailbox -z' # This will be used for temporary/log files during the import process TEMP='/tmp/' # We assume the working directory's name is the domain. # Otherwise, override this with your actual domain name. domain=`basename ${PWD}` echo Process ID: $$ if $1 != "" ; then USERS=$1; else USERS=`cat vpasswd | cut -f1 -d:` fi for user in ${USERS}; do echo "Beginning User: $user..." if $2 != "" ; then FOLDERS="$user/Maildir/$2/cur"; else FOLDERS=`find $user -type d -name cur | sort` fi echo "$FOLDERS" | while read line; do folderdir=`echo ${line} | cut -f3 -d"/"` if ${folderdir} == "cur" ; then folderdir=""; fi folder=`echo ${folderdir} | sed 's/^\.//; s%\.%/%g; s%\&-%\&%g'` folderpath=${PWD}/${user}/Maildir/${folderdir}/ # If the folder name is blank, this is the top level folder, # Zimbra calls it "Inbox" (so do most clients/servers). if $folder == "" ; then folder="Inbox"; fi # In Courier IMAP, all folders must be children of the root # folder, which means Trash, Junk, Sent, Drafts are typically # under Inbox. This is not the case with Zimbra, so we will # slide these mailboxes to the top level so they behave properly, # For all "non-special" mailboxes, we will keep them as children # so they remain where the user had them before. if [[ $folder != "Trash" && $folder != "Junk" && $folder != "Sent" && $folder != "Drafts" && $folder != "Inbox" ]] ; then folder="Inbox/${folder}"; fi echo "* Working on Folder $folder..." # Courier allows heirarchy where non-folders (literally nothing) are # able to have children. Zimbra does not. It's also possible that # we will process the folders out of heirarchical order for some reason # Here we separate the path and make sure all the parent folders exist # before trying to create the folder we're working on. parts=(`echo $folder | sed 's% %\x1a%g; s%/% %g'`); hier=""; for i in "${parts[@]}"; do hier=`echo ${hier}/$i | sed 's%^/%%; s%\x1a% %g'`; ${ZMCMD} -m ${user}@${domain} getFolder "/${hier}" >/dev/null 2>&1 || ( echo -n " + Creating folder $hier... " && ${ZMCMD} -m ${user}@${domain} createFolder "/${hier}" ) done # Figure out how many messages we have count=`find "${folderpath}new/" "${folderpath}cur/" -type f | wc -l`; imported=0; echo " * $count messages to process..." # Define the temporary file names we will need importfn="${TEMP}/import-$domain-$user-$folderdir-$$" implogfn="${TEMP}/import-$domain-$user-$folderdir-$$-log" impflogfn="${TEMP}/import-$domain-$user-$folderdir-$$-flaglog" impflagfn="${TEMP}/import-$domain-$user-$folderdir-$$-flags" touch "$importfn" # Determine the courier extended flag identifiers ("keywords") flagid=0 if -f "${folderpath}courierimapkeywords/:list" ; then extflags="YES" cat "${folderpath}courierimapkeywords/:list" 2>/dev/null | while read line; do # A blank line indicates the end of the definitions. if "${line}" == "" ; then break; fi # To avoid escape character madness, I'm swapping $ with % here. flag=`echo ${line} | sed 's/\\\$/%/'` echo courierflag[${flagid}]="'$flag'"; flagid=$(( flagid + 1 )); # Create the tag if it doesn't start with '%' if `echo ${flag} | grep '%'` == "" ; then echo -n " + Attemping to create tag ${flag}... " >&2 ${ZMCMD} -m ${user}@${domain} createTag "${flag}" >&2 fi done > "$impflagfn" source "$impflagfn" fi echo -n " * Queuing messages for import... " # Find all "cur" or "new" messages in this folder and import them. find "${folderpath}new/" "${folderpath}cur/" -type f | while read msg; do flags=""; tags=""; msgid=`echo $msg | cut -d: -f1 | sed s%.*/%%` # Determine the old maildir style flags oldflags=`echo $msg | cut -d: -f2` # Replied if `echo ${oldflags} | grep 'R'` != "" ; then flags="${flags}r"; fi # Seen if `echo ${oldflags} | grep 'S'` == "" ; then flags="${flags}u"; fi # Trashed if `echo ${oldflags} | grep 'T'` != "" ; then flags="${flags}x"; fi # Draft if `echo ${oldflags} | grep 'D'` != "" ; then flags="${flags}d"; fi # Flagged if `echo ${oldflags} | grep 'F'` != "" ; then flags="${flags}f"; fi # Determine the courier-imap extended flags for this message if ${extflags} == "YES" ; then oldflags2=`grep $msgid "${folderpath}courierimapkeywords/:list" 2>/dev/null | cut -d: -f2` for flag in ${oldflags2}; do # Forwarded if ${courierflag[$flag]} == '%Forwarded' ; then flags="${flags}w"; fi # Sent by me if ${courierflag[$flag]} == '%MDNSent' ; then flags="${flags}s"; fi # Convert non-system flags to Zimbra tags if `echo ${courierflag[$flag]} | grep '%'` == "" ; then tags="${tags},${courierflag[$flag]}" fi done # Clean up the tag list for the command line if ${tags} != "" ; then tags=`echo ${tags} | sed "s/^,\?/--tags \'/; s/\$/\'/"`; fi fi # Log the result of flag processing for debugging if $flags != "" || $tags != "" ; then echo `date +%c` "$msg had flags $oldflags and $oldflags2, now $flags and $tags in folder $folder" >> "$impflogfn" fi # Add the command to the queue file to import this message echo "addMessage --flags \"${flags}\" ${tags} \"/$folder\" \"${msg}\"" >> "$importfn" imported=$(( $imported + 1 )); printf "\b\b\b\b\b\b\b\b%7d " $imported; done echo "...done"; # Since we redirect the queue file to the mailbox tool, we end with "quit" echo "quit" >> "$importfn" # We're counting "prompts" from the zmmailbox utility here. The first # one comes up before a message is imported, so we start at -1 to offset # its existence. imported=-1; # We do this redirect because running the command for each message is very # slow. We can't just pass the directory to the command, despite Zimbra's # support because we can't tag or flag the messages that way. echo -n " * Running import process... " ${ZMCMD} -m $user@$domain < "${importfn}" 2> "${implogfn}" | while read; do imported=$(( $imported + 1 )); printf "\b\b\b\b\b\b\b\b%7d " $imported; done if -s "${implogfn}" ; then echo "...some messages did not import correctly: check $importfn"; else echo "...done"; fi done done echo "Import Process Complete!"
Now my output for a single account is:
Process ID: 11495 Beginning User: user... * Working on Folder Inbox/Archive... + Creating folder Inbox/Archive... 285 * 0 messages to process... * Queuing messages for import... ...done * Running import process... 0 ...done * Working on Folder Drafts... * 0 messages to process... * Queuing messages for import... ...done * Running import process... 0 ...done * Working on Folder Junk... * 0 messages to process... * Queuing messages for import... ...done * Running import process... 0 ...done * Working on Folder Sent... * 3 messages to process... * Queuing messages for import... 3 ...done * Running import process... 3 ...done * Working on Folder Trash... * 0 messages to process... * Queuing messages for import... ...done * Running import process... 0 ...done * Working on Folder Inbox/new folder... + Creating folder Inbox/new folder... 289 * 5 messages to process... * Queuing messages for import... 5 ...done * Running import process... 5 ...done * Working on Folder Inbox... * 1 messages to process... * Queuing messages for import... 1 ...done * Running import process... 1 ...done Import Process Complete!
It is important to note that running the import script multiple times will result in the e-mails being imported multiple times and it takes a fair amount of time to perform this procedure on one account nevermind one thousand so a strategy should be formulated for dealing with runoff mail before the MX/target is switched.
Comments
There are no comments for this item.