Manually Parse Multipart Form Data to Populate PHP Global $_POST/$_PUT Variables with a Simulated Request Body from stdin or File

For testing purposes I needed to simulate a multipart POST request from the command line. The body of a multipart request is huge and full of newlines so it's practical to dump it into a text file and redirect it to stdin. Multipart can't be parsed simply with parse_str() so it's necessary to manually parse the input. It seems most people running into this problem are trying to implement an ideologically-correct RESTful PUT interface but I found a tidy, global-agnostic class by misiek08 at https://gist.github.com/misiek08/7988b3b9a9911e35d0b3 and made a minor correction.

I've converted it into a static class and added a couple of functions to suit my use case:
class HttpMultipartParser { public static function populate_post_stdin() { $parsed = self::parse_stdin(); $_POST = $parsed['variables']; $_FILES = $parsed['files']; } public static function populate_put_stdin() { $parsed = self::parse_stdin(); $_PUT = $parsed['variables']; $_FILES = $parsed['files']; } public static function parse_stdin() { $stream = fopen('php://stdin', 'r'); return self::parse_multipart($stream); } public static function parse_multipart($stream, $boundary = null) { $return = array('variables' => array(), 'files' => array()); $partInfo = null; while(($lineN = fgets($stream)) !== false) { if(strpos($lineN, '--') === 0) { if(!isSet($boundary) || $boundary == null) { $boundary = rtrim($lineN); } continue; } $line = rtrim($lineN); if($line == '') { if(!empty($partInfo['Content-Disposition']['filename'])) { self::parse_file($stream, $boundary, $partInfo, $return['files']); } elseif($partInfo != null) { self::parse_variable($stream, $boundary, $partInfo['Content-Disposition']['name'], $return['variables']); } $partInfo = null; continue; } $delim = strpos($line, ':'); $headerKey = substr($line, 0, $delim); $headerVal = ltrim($line, $delim + 1); $partInfo[$headerKey] = self::parse_header_value($headerVal, $headerKey); } fclose($stream); return $return; } public static function parse_header_value($line, $header = '') { $retval = array(); $regex = '/(^|;)\s*(?P<name>[^=:,;\s"]*):?(=("(?P<quotedValue>[^"]*(\\.[^"]*)*)")|(\s*(?P<value>[^=,;\s"]*)))?/mx'; $matches = null; preg_match_all($regex, $line, $matches, PREG_SET_ORDER); for($i = 0; $i < count($matches); $i++) { $match = $matches[$i]; $name = $match['name']; $quotedValue = $match['quotedValue']; if(empty($quotedValue)) { $value = $match['value']; } else { $value = stripcslashes($quotedValue); } if($name == $header && $i == 0) { $name = 'value'; } $retval[$name] = $value; } return $retval; } public static function parse_variable($stream, $boundary, $name, &$array) { $fullValue = ''; $lastLine = null; while(($lineN = fgets($stream)) !== false && strpos($lineN, $boundary) !== 0) { if($lastLine != null) { $fullValue .= $lastLine; } $lastLine = $lineN; } if($lastLine != null) { $fullValue .= rtrim($lastLine, '\r\n'); } $array[$name] = $fullValue; } public static function parse_file($stream, $boundary, $info, &$array) { $tempdir = sys_get_temp_dir(); $name = $info['Content-Disposition']['name']; $fileStruct['name'] = $info['Content-Disposition']['filename']; $fileStruct['type'] = $info['Content-Type']['value']; $array[$name] = &$fileStruct; if(empty($tempdir)) { $fileStruct['error'] = UPLOAD_ERR_NO_TMP_DIR; return; } $tempname = tempnam($tempdir, 'php_upl'); $outFP = fopen($tempname, 'wb'); if($outFP === false) { $fileStruct['error'] = UPLOAD_ERR_CANT_WRITE; return; } $lastLine = null; while(($lineN = fgets($stream, 4096)) !== false) { if($lastLine != null) { if(strpos($lineN, $boundary) === 0) break; if(fwrite($outFP, $lastLine) === false) { $fileStruct = UPLOAD_ERR_CANT_WRITE; return; } } $lastLine = $lineN; } if($lastLine != null) { if(fwrite($outFP, rtrim($lastLine, '\r\n')) === false) { $fileStruct['error'] = UPLOAD_ERR_CANT_WRITE; return; } } $fileStruct['error'] = UPLOAD_ERR_OK; $fileStruct['size'] = filesize($tempname); $fileStruct['tmp_name'] = $tempname; } }

Things I Learned the Hard Way Doing Full Disk Encryption on Gentoo with DM-Crypt LUKS

The documentation for setting up full disk encryption on Gentoo is specific at best, spotty at worst and confusing in general. Without re-living the immense pain it was to configure over again in detail, here are some of the things I wish I knew before I started:

  • The 2TB partition size limit is not a Windows-only limitation. It comes from the old BIOS style partition scheme. You can either use parted to create >2T GPT partitions or use the whole raw disk if you plan on using it for a single mount point or if you fancy putting DM/LVM on top. Obviously that isn't an option if you plan on booting from this device.
  • If you don't think you need LVM you probably don't, especially if this is a workstation or personal computer. It's way easier to skip all of that and use old-fashioned device nodes directly.
  • Having a newline character in your keyfile is deadly. The init script that genkernel rolls into your initramfs uses cryptsetup luksOpen /dev/whatever whatever -d - or --keyfile - instead of simply piping the output of gpg as the wiki article has you do when you luksFormat. The former stops reading the input at a newline, the latter incorporates it into your key. This is a problem if, say, you took the output of openssl rand -base64 96 because you wanted to generate a 512 bit or larger key. There is already a newline in the middle of the cleartext, so I thought I was clever when I removed it. Not so; nano and many other text editors will always leave a newline at the end of the file. If you cat the cleartext and your command prompt doesn't run on to the end of it you still have a newline in there. Pipe your keyfile through tr -d '\n' before you do anything to be safe.
  • genkernel will make mrproper unless you use the --no-clean or --no-mrproper flags. It will back up your .config and start building your kernel the way it wants to unless you specify ramdisk.
  • The plain64 IV doesn't take arguments, specifying sha512 as the hash is redundant.
  • Genkernel will not magically read /etc/conf.d/dmcrypt and import your keyfiles or their locations. You may need to add the following to your kernel command line, replacing {UUID} with the UUID of your /boot partition (or wherever you are keeping the keyfile). You can obtain the UUID by running blkid. Using a UUID instead of a path will allow you to store your keyfile on removable media which may not have the same device node from time to time.
    real_root=/dev/mapper/root crypt_root=/dev/sda2 root_key=root.gpg root_keydev=UUID={UUID}

    If you are using grub2 append this to your GRUB_CMDLINE_LINUX variable in /etc/default/grub and if you are booting Xen remember to ALSO append it to your GRUB_CMDLINE_LINUX_XEN_DEFAULT variable.

  • Non-root partitions will be luksOpened and mounted during bootup by the dmcrypt init script.

Zombie Disk Image is Loopback-Mounted in a Guest Domain and can't be Mounted

Chances are if you're reading this you've just pulled yourself out of a xend crash or a similar event which has left one or more zombie VM disk images attached to their loop device. You're trying to restart the VM but you keep getting this message:

Error: Device 2049 (vbd) could not be connected.
File /xen/vm/disk.img is loopback-mounted through /dev/loop/5,
which is mounted in a guest domain,
and so cannot be mounted now.

losetup will only tell you what you already know:

# losetup -d /dev/loop/5
loop: can't delete device /dev/loop/5: Device or resource busy

If you don't know the ID number of the VM before it tanked you can find it in your xend logs. Execute this command to free the loop device:

# xenstore-rm backend/vbd/{VM ID}