Documentary for Dinner: Vice Fringe: How to Sell Drugs (2012)

Vice Fringe’s How to Sell Drugs follows a day in the life of “Nims,” a New York City dealer.

Despite the title you won’t learn any real trade secrets – other than don’t be a little shit. This doc is much more focused on Nims’ humanity and taking sweet shots of drugs than the business. Nontheless, an entertaining 20 minutes.

ApacheBench Shows Lots of Failed Requests due to Length

Breathe easy. Smile. You’re probably here because you’ve just run ab and got output something like:

This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking **** (be patient)
Completed 100000 requests
Completed 200000 requests
Completed 300000 requests
Completed 400000 requests
Completed 500000 requests
Completed 600000 requests
Completed 700000 requests
Completed 800000 requests
Completed 900000 requests
Finished 1000000 requests


Server Software:        nginx/1.2.1
Server Hostname:        ****
Server Port:            80

Document Path:          ****
Document Length:        162 bytes

Concurrency Level:      5
Time taken for tests:   9502.884921 seconds
Complete requests:      1000000
Failed requests:        697730
   (Connect: 0, Length: 697730, Exceptions: 0)
Write errors:           0
Total transferred:      279019852 bytes
Total POSTed:           247000494
HTML transferred:       158019731 bytes
Requests per second:    105.23 [#/sec] (mean)
Time per request:       47.514 [ms] (mean)
Time per request:       9.503 [ms] (mean, across all concurrent requests)
Transfer rate:          28.67 [Kbytes/sec] received
                        25.38 kb/s sent
                        54.06 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   15  26.7     13    3183
Processing:     0   31  18.8     29    2319
Waiting:        0   30  15.9     28    1719
Total:          0   46  38.9     42    4333

Percentage of the requests served within a certain time (ms)
  50%     42
  66%     45
  75%     46
  80%     47
  90%     51
  95%     55
  98%    104
  99%    201
 100%   4333 (longest request)

697730 out of 1 million requests failed? No, not really.

ApacheBench expects to be run against something that produces consistent output. Chances are you’ve specified a script that has dynamic output and the length of that output has changed since the first pull.

Let’s have a nice cup of tea :)

SHM/tmpfs File-Based PHP Cache/Datastore in RAM

It seems like only yesterday XCache was my knight in shining armour but a burst of segfaults has prompted the creation of a backup plan.

UPDATE It turns out my problem was actually PHP’s fault. Put that armour back on!

We can use files on tmpfs to provide much the same function as XCache or APC’s shared datastore. Start by mounting a slice somewhere appropriate (this line is for fstab):

none                    /mnt/ram        tmpfs           defaults,noatime,size=256M      0 0

Next we’ll create some basic interface functions. Let $config['fcache_path'] be the path to your tmpfs mount or writeable directory:

function fcache_isset($key)
{
	global $config;
	return @file_exists($config['fcache_path'].$key);
}

function fcache_unset($key)
{
	global $config;
	return @unlink($config['fcache_path'].$key);
}

function fcache_get($key)
{
	global $config;
	$val = @file_get_contents($config['fcache_path'].$key);
	if(empty($val))
		return NULL;
	else
		return $val;
}

function fcache_set($key, $val='')
{
	global $config;
	if(!empty($val))
	{
		$tmp = tempnam($config['fcache_path'], $key);
		if(@file_put_contents($tmp, $val))
		{
			if(@rename($tmp, $config['fcache_path'].$key))
				return true;
			else
				return false;
		}
		else
		{
			return false;
		}
	}
	return true;
}

I use rename() instead of flock() to make atomic writes because according to the manual page:

On some operating systems flock() is implemented at the process level. When using a multithreaded server API like ISAPI you may not be able to rely on flock() to protect files against other PHP scripts running in parallel threads of the same server instance!

They mention IIS’ ISAPI specifically but I’ve had enough problems with Apache’s mpm_worker lately that I’m not willing to take the risk. Further, I’d rather have the query run twice than have any lock-related hangups.

Now that we have some very basic functions to interface with we can put them to work in something useful. The following is what I’ve whipped up to switch between XCache, this file-based cache and no cache at all when pulling standard mysql results. cache_set() could easily be replaced with cache_unset() preceeding every update query but I do things this way to make the code more readable to me. You can also increase performance by using only arrays instead of converting between arrays and objects but this software was written entirely using mysql_fetch_object() and the caching was an afterthought.

Let $config['cache'] contain the cache type.

function cache_get($key, $query)
{
	global $config;
	if($config['cache'] == 'xcache' and function_exists('xcache_get'))
	{
		$serialized = xcache_get($key);
		if($serialized != NULL)
		{
			$unserialized = unserialize($serialized);
			$object = (object) $unserialized;
			return $object;
		}
		else
		{
			$result = mysql_query($query);
			if($result === false)
				return false;
			if(mysql_num_rows($result) > 0)
			{
				$object = mysql_fetch_object($result);
				$array = (array) $object;
				$serialized = serialize($array);
				xcache_set($key, $serialized);
				return $object;
			}
			else
			{
				return true;
			}
		}
	}
	elseif($config['cache'] == 'fcache' and function_exists('fcache_get'))
	{
		$serialized = fcache_get($key);
		if($serialized != NULL)
		{
			$unserialized = unserialize($serialized);
			$object = (object) $unserialized;
			return $object;
		}
		else
		{
			$result = mysql_query($query);
			if($result === false)
				return false;
			if(mysql_num_rows($result) > 0)
			{
				$object = mysql_fetch_object($result);
				$array = (array) $object;
				$serialized = serialize($array);
				fcache_set($key, $serialized);
				return $object;
			}
			else
			{
				return true;
			}
		}
	}
	else
	{
		$result = mysql_query($query);
		if($result === false)
			return false;
		if(mysql_num_rows($result) > 0)
		{
			$object = mysql_fetch_object($result);
			return $object;
		}
		else
		{
			return true;
		}
	}
}

function cache_set($key, $query)
{
	global $config;
	if($config['cache'] == 'xcache' and function_exists('xcache_unset'))
	{
		$result = mysql_query($query);
		if($result === false)
			return false;
		xcache_unset($key);
		return true;
	}
	elseif($config['cache'] == 'fcache' and function_exists('fcache_unset'))
	{
		$result = mysql_query($query);
		if($result === false)
			return false;
		fcache_unset($key);
		return true;
	}
	else
	{
		$result = mysql_query($query);
		if($result === false)
			return false;
		return true;
	}
}

function cache_unset($key)
{
	global $config;
	if($config['cache'] == 'xcache' and function_exists('xcache_unset'))
	{
		xcache_unset($key);
		return true;
	}
	elseif($config['cache'] == 'fcache' and function_exists('fcache_unset'))
	{
		fcache_unset($key);
		return true;
	}
	else
	{
		return true;
	}
}

Please note that cache_get() checks if the returned value is NULL, it does NOT use (x|f)cache_isset() because that would introduce a serious race condition.

This implementation leaves out two important features that xcache has: garbage collection and timeouts. Garbage collection can be handled by a cron script and use of the find command to take out stale entries. Timeouts can be implemented by inserting a value into the file and comparing it against the file’s time stamp and the current time – a clever idea I got from looking over http://flourishlib.com/docs/fCache.

Will Bond’s fCache is probably what you’re looking for if you want to be able to port between all of the major datastores easily and have individual control over an item’s expiration. However, this implementation uses a rand()om number for garbage collection and may be subject to the race (or minor hangup depending on how file_put_contents() handles locking) condition we avoid here with atomic writes.

Here are some completely meaningless apache bench benchmarks against an AJAX app’s polling script on a live, production server:

Without datastore

Document Length:        105 bytes

Concurrency Level:      20
Time taken for tests:   122.880 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      2420000 bytes
Total POSTed:           2290229
HTML transferred:       1050000 bytes
Requests per second:    81.38 [#/sec] (mean)
Time per request:       245.761 [ms] (mean)
Time per request:       12.288 [ms] (mean, across all concurrent requests)  
Transfer rate:          19.23 [Kbytes/sec] received          
         18.20 kb/s sent       
         37.43 kb/s total      
 
Connection Times (ms)          
              min  mean[+/-sd] median   max   
Connect:       91  202 341.7    154    9170   
Processing:    19   43  40.3     32     746   
Waiting:       19   40  39.0     31     746   
Total:        114  245 344.1    194    9193   
 
Percentage of the requests served within a certain time (ms) 
  50%    194    
  66%    212    
  75%    224    
  80%    232    
  90%    264    
  95%    408    
  98%    480    
  99%   3170    
 100%   9193 (longest request)

XCache datastore

Document Length:        105 bytes

Concurrency Level:      20
Time taken for tests:   121.803 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      2420000 bytes
Total POSTed:           2290229
HTML transferred:       1050000 bytes
Requests per second:    82.10 [#/sec] (mean)
Time per request:       243.605 [ms] (mean)
Time per request:       12.180 [ms] (mean, across all concurrent requests)
Transfer rate:          19.40 [Kbytes/sec] received
                        18.36 kb/s sent
                        37.76 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       91  201 331.6    154    3418
Processing:    19   42  40.6     32     798
Waiting:       19   39  39.2     31     788
Total:        115  243 334.1    193    3459

Percentage of the requests served within a certain time (ms)
  50%    193
  66%    210
  75%    221
  80%    228
  90%    260
  95%    405
  98%    473
  99%   3163
 100%   3459 (longest request)

fcache

Document Length:        105 bytes

Concurrency Level:      20
Time taken for tests:   121.174 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      2420000 bytes
Total POSTed:           2291374
HTML transferred:       1050000 bytes
Requests per second:    82.53 [#/sec] (mean)
Time per request:       242.347 [ms] (mean)
Time per request:       12.117 [ms] (mean, across all concurrent requests)
Transfer rate:          19.50 [Kbytes/sec] received
                        18.47 kb/s sent
                        37.97 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       90  199 320.8    154    3407
Processing:    19   42  38.7     33     747
Waiting:       19   40  37.3     32     747
Total:        116  242 323.2    193    3486

Percentage of the requests served within a certain time (ms)
  50%    193
  66%    210
  75%    222
  80%    231
  90%    262
  95%    408
  98%    475
  99%   3161
 100%   3486 (longest request)

Interesting to see fcache narrowly beat out xcache but since the testing environment is not perfectly controlled the results are of course useless.

Return top

LINKS

foxpa.ws
Online Marketing Toplist
Internet
Technology Blogs - Blog Rankings

Internet Blogs - BlogCatalog Blog Directory

Technology blogs 		Bad Karma Networks

Please Donate!


Made in Canada  •  There's a fox in the Gibson!  •  2010-12