pitter patter on the keyboard

I recently mentioned using APC with threaded Apache and PHP was incredibly unstable. For a few days I moved back to the prefork (process-based) Apache MPM to see if it was worth bringing back APC. This eventually ended up bringing the load average on the SQL VM to about 40 with what I would normally consider an appropriate number of MaxClients and the odd spike at about 10 when restricted to 128. Page load time was also noticeably increased so I decided to give XCache a whirl (thanks to LiteStar mentioning it in the comments of that article).

XCache is a stable, and (so far it seems) genuinely thread-safe opcode cache and datastore. It is developed my mOo, a lighttpd developer and hosted under the lighttpd domain. Depsite this, it is built as a PHP extension in the same manner as APC meaning it can be used with mod_php, fastcgi etc. Like APC, XCache features shared memory variable storage functions with a simple name, value scheme in addition to the opcode cache. I was able to quickly write an SQL wrapper using these that reduced the database hits in one of my apps by a factor of about 10,000 – making virtually the whole thing run out of RAM.

Also similarly to APC, XCache comes with bundled admin scripts which – while perhaps not as pretty as APC’s – I certainly find more insightful. Some folks find it faster, but I find it works so it looks like I’ll be sold for some time.

Cheers, Litey. And cheers, mOo.

UPDATE Or not… After a few weeks of faithful service xcache finally segfaulted on me:

apache2[17395]: segfault at 332e352e ip b5ba90f7 sp a4b4cb70 error 4 in xcache.so[b5b96000+1d000]
apache2[16754]: segfault at 4320526d ip b5ba4297 sp aab5ad60 error 4 in xcache.so[b5b96000+1d000]
apache2[27959]: segfault at 43205445 ip b5ba90f7 sp aab58b70 error 4 in xcache.so[b5b96000+1d000]
apache2[15215]: segfault at 2 ip b5ba90f7 sp a534db70 error 4 in xcache.so[b5b96000+1d000]
apache2[15536]: segfault at 362e3335 ip b5ba90f7 sp ac35bb70 error 4 in xcache.so[b5b96000+1d000]

I’m doing my best at this moment to reproduce the suituation.

UPDATE It turns out this might actually be a bug in PHP. mOo was kind enough to respond in detail:

A PHP 5.3 bug No.62432 ReflectionMethod random corrupt memory on high
concurrent was fixed that cause PHP itself unstable, bogus reported as
XCache bug. It is false positive to reproduce with XCache
loaded/enabled only, just because XCache make PHP run faster than
with/without other opcode cacher. PHP up to 5.3.14 is affected. It is
highly suggested to upgrade to newer PHP

https://bugs.php.net/bug.php?id=62432

so, upgrade to 5.4 if applicable. upgarde to
http://xcache.lighttpd.net/wiki/Release-2.0.1 if you still have
problem with PHP 5.4 + XCache 1.3.2

Which I actually realized when reading this while he was making his reply https://groups.google.com/forum/?fromgroups#!topic/xcache/pZHjUu3Dq3k:

PHP 5.3.14 is unstable, Please upgrade to new version. You have been warned I’ve tried hard to make it stable even in 2.0 big jump, yet anything that can go wrong goes wrong.

Just goes to show I need to do more reading before jumping the gun :p

I came across this script while looking for something that would let me quickly determine a user’s download speed. I’ve improved upon it by detecting latency (and HTTP header overhead) then subtracting it from the total time of the bulk download to yield more accurate results.

You must supply a large file to download, the larger the more accurate (or at least averaged) the reading. I created mine like so:

$ dd if=/dev/urandom of=./random.img bs=1M count=20

Enjoy:

<?php

$data_file = "random.img";    // $data_file should be some generic binary file - this script was tested with a 3 MB zip file of some Dilbert comic strips. The file should be sufficiently large so that the latency from the initial HTTP request itself does not skew the results by much.
$filesize = filesize($data_file);
$divisor = 1024; //set this to 1000 instead of 1024 if you want true KB and MB, rather than mis-labeled KiB and MiB that we are most used to seeing...
if(isset($_GET['latency']))
{
	print('L');
	exit(0);
}
elseif(isset($_GET['ajax'])){
    // if it's an ajax call we just read and print the binary file's contents...
    $fd = fopen ($data_file, 'r');
    $contents = fread ($fd, $filesize);
    ob_start(print $contents);
    ob_end_flush();
    fclose ($fd);
    exit(0);
}
else{
    // if this visit is not an AJAX request we display the page itself...
?><!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html;charset=utf-8" />
<title>Bandwidth Test</title>

<script type="text/javascript">
//<!--
var time;
var starttime;
var latency_starttime;
var endtime;
var downloadtime;
var kbps;
var ksec;
var mbps;
var msec;
var ajax_run = new XMLHttpRequest();
function addCommas(nStr){
    nStr += '';
    x = nStr.split('.');
    x1 = x[0];
    x2 = x.length > 1 ? '.' + x[1] : '';
    var rgx = /(\d+)(\d{3})/;
    while (rgx.test(x1)) {
        x1 = x1.replace(rgx, '$1' + ',' + '$2');
    }
    return x1 + x2;
}
function ajax_bandwidth_test(){
    cancel_any_pending_ajax();
    disable_table();
    send_ajax_call();
}
function cancel_any_pending_ajax(){
    if(ajax_run.readyState){
        if (ajax_run.readyState != 0 && ajax_run.readyState != 4){
            ajax_run.abort();
        }
    }
}
function disable_table(){
    var action_row = document.getElementById('action_row');
    var spans = action_row.getElementsByTagName('span');
    var link = action_row.getElementsByTagName('a')[0];
    link.style.zIndex = -1;
    if (spans.length>1){
        spans[1].className = 'disabled';
        spans[2].className = 'disabled';
    }
    var status_box = document.getElementById('status');
    status_box.innerHTML = "Testing Your Current Bandwidth...";
    document.title = "Bandwidth Test | Running...";
}
function send_ajax_call(){
    var result = '';
    var x = new Date();
    x = x.getTime();
    var latency_url = '<?php print $_SERVER['REQUEST_URI']; ?>?latency='+x;
    var url = '<?php print $_SERVER['REQUEST_URI']; ?>?ajax='+x; //you will need to change the "?" to a "&" if your URI already has some $_GET variables in it...

    if (window.XMLHttpRequest){
        // code for IE7+, Firefox, Chrome, Opera, Safari
        latency_run = new XMLHttpRequest();
        ajax_run = new XMLHttpRequest();
    }
    else{
        // code for IE6, IE5
        latency_run = new ActiveXObject("Microsoft.XMLHTTP");
        ajax_run = new ActiveXObject("Microsoft.XMLHTTP");
    }

    latency_run.onreadystatechange = function(){
        if (latency_run.readyState==4 && latency_run.status==200){
            result = latency_run.responseBody;    //ie won't work with responseText, but responseBody will work just fine...
            if (result!=''){
                // alert(latency_starttime);
            }
            else{
                alert('Speed test failed!');
            }
        }
        else if (latency_run.readyState==1){
            //connected to server, start the timer...
            time = new Date();
            latency_starttime = time.getTime();
            document.getElementById('status').innerHTML = "Testing Your Current Latency and HTTP Overhead... Connected..."
        }
        else if (latency_run.readyState==2){
            //server is processing the request...
            document.getElementById('status').innerHTML = "Testing Your Current Latency and HTTP Overhead... Loading..."
        }
        else if (latency_run.readyState==4 && latency_run.status!=200){
            alert('Speed test failed! (' + latency_run.status + ')');
        }
    }
    latency_run.open("GET",latency_url,false);
    latency_run.send();

    ajax_run.onreadystatechange = function(){
        if (ajax_run.readyState==4 && ajax_run.status==200){
            result = ajax_run.responseBody;    //ie won't work with responseText, but responseBody will work just fine...
            if (result!=''){
                //no point in printing the binary result to the page, just update the speed test results...
                update_results();
            }
            else{
                alert('Speed test failed!');
            }
        }
        else if (ajax_run.readyState==1){
            //connected to server, start the timer...
            time = new Date();
            starttime = time.getTime();
            document.getElementById('status').innerHTML = "Testing Your Current Bandwidth... Connected..."
        }
        else if (ajax_run.readyState==2){
            //server is processing the request...
            document.getElementById('status').innerHTML = "Testing Your Current Bandwidth... Loading..."
        }
        else if (ajax_run.readyState==4 && ajax_run.status!=200){
            alert('Speed test failed! (' + ajax_run.status + ')');
        }
    }
    ajax_run.open("GET",url,false);
    ajax_run.send();
}
function update_results(){
    time = new Date();
    endtime = time.getTime();
    latency = starttime - latency_starttime;
    alert(latency_starttime+' '+starttime);
    if (endtime == starttime){
        downloadtime = 0;
    }
    else{
        downloadtime = (endtime - starttime - (latency * 2))/1000;
    }
    kbytes_of_data = <?php print $filesize/$divisor; ?>;
    mbytes_of_data = <?php print $filesize/$divisor/$divisor; ?>;
    ksec = kbytes_of_data / downloadtime;
    msec = mbytes_of_data / downloadtime;
    kbps = addCommas(Math.round(ksec * 8192)/1000);
    mbps = addCommas(Math.round(msec * 8192)/1000);

    ksec = addCommas(Math.round(ksec*100)/100);
    msec = addCommas(Math.round(msec*100)/100);
    var action_row = document.getElementById('action_row');
    var cells = action_row.getElementsByTagName('td');
    var link = document.getElementById('trigger');
    var title = "Bandwidth Test | ";
    var text = '';
    if (msec>1){
        cells[1].innerHTML = "<span>" + mbps + " mbps</span>";
        cells[2].innerHTML = "<span>" + msec + " MB/s</span>";
        cells[3].innerHTML = "<span>" + latency + " ms</span>";
        title += msec + " MB/s";
        text += addCommas(Math.round(mbytes_of_data*100)/100) + ' MB of data transferred in ' + downloadtime + ' seconds...';
    }
    else{
        cells[1].innerHTML = "<span>" + kbps + " kbps</span>";
        cells[2].innerHTML = "<span>" + ksec + " KB/s</span>";
        cells[3].innerHTML = "<span>" + latency + " ms</span>";
        title += ksec + " KB/s";
        text += addCommas(Math.round(kbytes_of_data*100)/100) + ' KB of data transferred in ' + downloadtime + ' seconds...';
    }
    document.title = title;
    document.getElementById('status').innerHTML = text;
    link.style.zIndex = 1;
}
//-->

</script>
<style type="text/css">
*{margin:0;padding:0;}
body{padding:20px;background:#fff;}
table{border-collapse:collapse;position:relative;z-index:0;}
th,td{border:1px solid #000;cursor:default;}
th{padding:2px 5px;font-weight:bold;text-align:center;font-size:16px;}
td{width:120px;text-align:right;}
td span{padding:2px 5px;display:block;line-height:20px;}
td:first-child{width:110px;text-align:left;}
th:first-child{border-top:0;border-left:0;}
a{text-decoration:none;font-weight:bold;font-family:tahoma;font-size:12px;display:block;color:#333;padding:5px;text-align:center;position:relative;z-index:1;}
a:hover{background-color:#333;color:#fff;}
a,a:hover,a:focus,a:active{outline:0;}
#link_box{position:relative;}
#link_mask{display:block;width:100%;height:100%;background:transparent;position:absolute;top:0;left:0;z-index:0;padding:0;line-height:24px;font-size:500px;overflow:hidden;cursor:default;}
#status{margin-top:20px;}
.disabled{background-color:#333;color:#fff;}
.disclaimer td{border-right:0;border-bottom:0;border-left:0;font-size:11px;text-align:right;color:#333;width:auto;}
.disclaimer td em{font-weight:normal;font-style:italic;}
</style>
</head>
<body>
<table>
  <thead>
    <tr>

      <th> </th>
      <th>bits</th>
      <th>Bytes</th>
      <th>Latency</th>
    </tr>

  </thead>
  <tbody>
    <tr id="action_row">
      <td><div id="link_box"><a href="javascript:ajax_bandwidth_test();" id="trigger">Start Test</a><span id="link_mask"> </span></div></td>

      <td> </td>
      <td> </td>
      <td> </td>
    </tr>
    <tr class="disclaimer">
      <td colspan="4">[ <em>speeds are only relative to your connection speed with this website</em> ]</td>

    </tr>
  </tbody>
</table>
<p id="status"></p>
</body>
</html>
<?php
}
?>

I have always been fond of the One-Time Pad. There are a lot of paranoid people who think admins have nothing better to do than sit around reading private messages all day and I thought it would be neat (even if never used) to implement client-side OTP encryption. Not having the time to do this kind of thing myself I asked Tucker, a programmer who helps moderate Yiffy International if he would like to contribute the code. Fortunately for us he did, and he is graciously allowing me to re-post it here for your pleasure.

It should be noted that using the built in pad generator is the least secure method since it is easy to predict that pads generated with it used a browser’s RNG.

The particularly clever thing about this implementation is that special characters, numbers and spaces – even line breaks – are preserved through a mechanism we call “Z-encoding.”

Thanks Tucker, that’s super cool!

Here is the JavaScript element:

function format (string) {
  var retval = string.charAt (0);
  for (var i = 1; i < string.length; i ++) {
    if ((i % 48) == 0) retval += "\n";
    else if ((i % 6) == 0) retval += " ";
    retval += string.charAt (i);
  }
  return retval;
}

function unformat (string) {
  string = string.toUpperCase ();
  var retval = "";
  for (var i = 0; i < string.length; i ++) {
    if ((string.charCodeAt (i) >= 65) && (string.charCodeAt (i) <= 90)) {
      retval += string.charAt (i);
    }
  }
  return retval;
}

function checkformat (string) {
  for (var i = 0; i < string.length; i ++) {
    if ((string.charCodeAt (i) < 65) || (string.charCodeAt (i) > 90)) {
      return true;
    }
  }
  return false;
}

function inttoletters (number) {
  var bigbyte = Math.floor (number / 16);
  var smallbyte = number - (bigbyte * 16);
  return String.fromCharCode (65 + bigbyte, 65 + smallbyte);
}

function letterstoint (letter1, letter2) {
  var bigbyte = letter1 - 65;
  var smallbyte = letter2 - 65;
  return String.fromCharCode ((16 * bigbyte) + smallbyte);
}

function process (source) {
  source = source.toUpperCase ();
  var retval = "";
  for (var i = 0; i < source.length; i ++) {
    if ((source.charCodeAt (i) >= 65) && (source.charCodeAt (i) < 90)) {
      retval += source.charAt (i);
    } else if (source.charCodeAt (i) == 90) {
      retval += "ZZ";
    } else {
      retval += "Z" + inttoletters (source.charCodeAt (i));
    }
  }
  return retval;
}

function unprocess (source) {
  var retval = "";if (checkformat (source)) {
    dispmsg ("Error: Invalid characters in source string.");
    return "";
  } else {
    for (var i = 0; i < source.length; i ++) {
      if ((source.charCodeAt (i) >= 65) && (source.charCodeAt (i) < 90)) {
        retval += source.charAt (i);
      } else if (source.charCodeAt (i) == 90) {
        i ++;
        if (source.charCodeAt (i) == 90) {
          retval += source.charAt (i);
        } else {
          retval += letterstoint (source.charCodeAt (i), source.charCodeAt (i + 1));
          i ++;
        }
      }
    }
    return retval.toLowerCase ();
  }
}

function genprs (source) {
  var retval = "";
  if (checkformat (source)) {
    dispmsg ("Error: Invalid characters in source string.");
    return "";
  } else {
    for (var i = 0; i < source.length; i ++) {
      retval += String.fromCharCode (65 + Math.floor (Math.random () * 26));
    }
    dispmsg ("Ready.");
    return retval;
  }
}

function encrypt (source, pad) {
  var retval = "";
  var newchar = 0;
  if (checkformat (source)) {
    dispmsg ("Error: Invalid characters in source string.");
    return "";
  } else if (source.length > pad.length) {
    dispmsg ("Error: Too many characters in source string.");
    return "";
  } else {
    while (pad.length > source.length) source += String.fromCharCode (65 + Math.floor (Math.random () * 25));
    for (var i = 0; i < source.length; i ++) {
      newchar = source.charCodeAt (i) + pad.charCodeAt (i) - 65;
      if (newchar > 90) newchar -= 26;
      retval += String.fromCharCode (newchar);
    }
    dispmsg ("Ready.");
    return retval;
  }
}

function decrypt (source, pad) {
  var retval = "";
  var newchar = 0;
  if (source.length > pad.length) {
    dispmsg ("Error: Too many characters in source string.");
    return "";
  } else {
    for (var i = 0; i < source.length; i ++) {
      newchar = source.charCodeAt (i) - pad.charCodeAt (i) + 65;
      if (newchar < 65) newchar += 26;
      retval += String.fromCharCode (newchar);
    }
    dispmsg ("Ready.");
    return retval;
  }
}

function fullencrypt (source, pad) {
  return format (encrypt (process (source), unformat (pad)));
}

function fulldecrypt (source, pad) {
  return unprocess (decrypt (unformat (source), unformat (pad)));
}

function html_generate () {
  document.getElementById ("pad").value = format (genprs (process (document.getElementById ("sender").value)));
}

function html_format () {
  document.getElementById ("pad").value = format (unformat (document.getElementById ("pad").value));
}

function html_copypad () {
  document.getElementById ("pastepad").value = document.getElementById ("pad").value;
  document.getElementById ("reciever").value = document.getElementById ("sender").value;
}

function html_encrypt () {
  document.getElementById ("sender").value = fullencrypt (document.getElementById ("sender").value, document.getElementById ("pad").value);
}

function html_decrypt () {
  document.getElementById ("reciever").value = fulldecrypt (document.getElementById ("reciever").value, document.getElementById ("pastepad").value);
}

function dispmsg (msgcode) {
  document.getElementById ("message").innerHTML = msgcode;
}

Here is a demo HTML form:

<html>
<head>
<title>Encryption Demo</title>
<style>textarea { resize: none !important; } #message { color: red; }</style>
<script type="text/javascript" src="encrypt.js"></script>
<script type="text/javascript" src="main.js"></script>
</head>
<body>
<table><tr>
<td>Sender: <input onclick="html_encrypt()" type="button" value="Encrypt" /></td>
<td>One Time Pad: <input onclick="html_generate()" type="button" value="Generate" /> <input onclick="html_format()" type="button" value="Format" /></td>
</tr><tr>
<td><textarea id="sender" cols="80" rows="20"></textarea></td>
<td><textarea id="pad" cols="80" rows="20"></textarea></td>
</tr><tr>
<td>Reciever: <input onclick="html_decrypt()" type="button" value="Decrypt" /></td>
<td>Paste One Time Pad: <input onclick="html_copypad()" type="button" value="Copy Pad" /></td>
</tr><tr>
<td><textarea id="reciever" readonly="readonly" cols="80" rows="20"></textarea></td>
<td><textarea id="pastepad" readonly="readonly" cols="80" rows="20"></textarea></td>
</tr></table>
<span id="message">Ready.</span>
</body>
</html>

You can read more about our implementation at: http://forum.yiffy.tk/viewtopic.php?f=2&t=3712.