Posts Tagged ‘request’

Portable PHP HTTP(S) GET Request with HTTP Basic Authentication

I’m not sure where I found this class years ago (probably the php.net comments) but it’s a handy way to make simple HTTP GET requests in PHP without having to rely on the availability of allow_url_fopen or curl extensions. It supports automatic detection of SSL/TLS and non-default port numbers based on the URL you provide.

I’ve made a minor modification to include support for HTTP Basic Authentication.

class HTTPRequest
{
    var $_fp;          // HTTP socket
    var $_url;         // full URL
    var $_host;        // HTTP host
    var $_protocol;    // protocol (HTTP/HTTPS)
    var $_uri;         // request URI
    var $_port;        // port
    var $_user;        // HTTP Basic Auth User
    var $_pass;        // HTTP Basic Auth Password
   
    // scan url
    function _scan_url()
    {
        $req = $this->_url;
       
        $pos = strpos($req, '://');
        $this->_protocol = strtolower(substr($req, 0, $pos));
       
        $req = substr($req, $pos+3);
        $pos = strpos($req, '/');
        if($pos === false)
            $pos = strlen($req);
        $host = substr($req, 0, $pos);
       
        if(strpos($host, ':') !== false)
        {
            list($this->_host, $this->_port) = explode(':', $host);
        }
        else
        {
            $this->_host = $host;
            $this->_port = ($this->_protocol == 'https') ? 443 : 80;
        }
       
        $this->_uri = substr($req, $pos);
        if($this->_uri == '')
            $this->_uri = '/';
    }
   
    // constructor
    function HTTPRequest($url, $user='', $pass='')
    {
        $this->_url = $url;
        $this->_scan_url();
        $this->_user = $user;
        $this->_pass = $pass;
    }
   
    // download URL to string
    function DownloadToString()
    {
        $crlf = "\r\n";
       
        // generate request
        $req = 'GET ' . $this->_uri . ' HTTP/1.0' . $crlf
            .    'Host: ' . $this->_host . $crlf;
            if(!empty($this->_user))
                 $req .= "Authorization: Basic " . base64_encode($this->_user . ':' . $this->_pass) . $crlf;
       $req .= $crlf;
       
        // fetch
        $this->_fp = fsockopen(($this->_protocol == 'https' ? 'ssl://' : '') . $this->_host, $this->_port);
        fwrite($this->_fp, $req);
        while(is_resource($this->_fp) && $this->_fp && !feof($this->_fp))
            $response .= fread($this->_fp, 1024);
        fclose($this->_fp);
       
        // split header and body
        $pos = strpos($response, $crlf . $crlf);
        if($pos === false)
            return($response);
        $header = substr($response, 0, $pos);
        $body = substr($response, $pos + 2 * strlen($crlf));
       
        // parse headers
        $headers = array();
        $lines = explode($crlf, $header);
        foreach($lines as $line)
            if(($pos = strpos($line, ':')) !== false)
                $headers[strtolower(trim(substr($line, 0, $pos)))] = trim(substr($line, $pos+1));
       
        // redirection?
        if(isset($headers['location']))
        {
            $http = new HTTPRequest($headers['location']);
            return($http->DownloadToString($http));
        }
        else
        {
            return($body);
        }
    }
}

Usage:

$r = new HTTPRequest($url, [username], [password]);
$response = $r->DownloadToString();

The username and password variables are optional.

Reliable AJAX: Timeout and Retry XMLHttpRequest

A lot of AJAX seems to rely on ideal conditions: a server that is both running and in good health, a reliable high speed internet connection, a lack of packet loss etc. Unfortunately, this is not always the case – what happens when your user triggers an event and the daemon is restarting or they are hopping access points/cell towers? What happens if the server load is sky high and only half of the requests are getting through? AJAX isn’t a very robust technology but we can increase the odds of our XMLHttpRequest transactions succeeding with some rudimentary JavaScript.

Newer Msxml XMLHTTP implementations support runtime-definable connection time-outs but that covers at best half of your audience and is therefore irrelevant to our needs. We will be implementing our own cross-browser compatible time-outs with the setTimeout() function. Once the time-out expires we will kill the XHR and repeat the process again, with a slightly longer time-out. The process is repeated as many times as necessary; each time increasing the duration. This is called backoff – similar in concept to but much less complicated than the exponential backoff algorithm Ethernet uses to recover from packet collisions.

The tricky part is to set the initial time-out to a value which gives the browser as long as reasonably possible to send and receive the request under strained conditions. Depending on how long the round trip takes in ideal circumstances a safe value can range anywhere from (plus) less than one second to several seconds – this is going to be unique to your application, resources and network so test thoroughly. If the time-out is too low we will be doing more harm than good by stopping and re-sending the request before it has even had a chance to complete. It might be sensible for you to give the last attempt a huge time-out.

The other major factor to consider is whether “the user knows best” in the given situation or not. If you are giving a visual cue to the user that there is AJAX activity happening in the background or if they expect to see something on the page update once the request has completed they might notice that things are taking a little long to react and re-trigger the event (read: hammer on the button like a mindless idiot). If you don’t want them to interfere with the time-out and re-send procedure take that into account when setting up your trigger – the example we are going to use assumes the user knows best. If they hammer on the button the best case scenario is their request goes through faster than if they left it alone and the worst case scenario is once they stop hammering the time-out and re-load cycle will run its course from the top anyway.

// Initialize global variables
var attempts = 0;
var timeout = 1000;    // In miliseconds
var max_attempts = 5;  // Six from 0
var script_url = ''    // The location of the target processing script
var junk = '';         // The variables we're POSTing, probably set in trigger()
var t = '';

// Cross-Browser XMLHttpRequest()
function createXHR()
{
	try { return new XMLHttpRequest(); } catch(e) {}
	try { return new ActiveXObject("Msxml2.XMLHTTP.7.0"); } catch (e) {}
	try { return new ActiveXObject("Msxml2.XMLHTTP.6.0"); } catch (e) {}
	try { return new ActiveXObject("Msxml2.XMLHTTP.5.0"); } catch (e) {}
	try { return new ActiveXObject("Msxml2.XMLHTTP.4.0"); } catch (e) {}
	try { return new ActiveXObject("Msxml2.XMLHTTP.3.0"); } catch (e) {}
	try { return new ActiveXObject("Msxml2.XMLHTTP"); } catch (e) {}
	try { return new ActiveXObject("Microsoft.XMLHTTP"); } catch (e) {}
	alert("Please enable ActiveX or update your browser.");
	return null;
}

// User calls this function, i.e. onclick event handler
function trigger()
{	
	// Re-set the time-out in case the user interrupts the cycle
	clearTimeout(t);
	attempts = 0;
	timeout = 1000;

	driver();
}

// Handle the state changes of the XHR
function callback()
{
	if (xmlhttp.readyState==4 && xmlhttp.status==200)
	{
		// Success! Re-set the time-out for the next round.
		clearTimeout(t);
		attempts = 0;
		timeout = 1000;

		xmlhttp = '';

		// Perform whatever action we need to do on success.
	}
	else
	{
		// Error handling/Request status indication
	}
}

// The guts of the request
function driver()
{
	xmlhttp = createXHR();

	xmlhttp.open("POST",script_url,true);

	t = setTimeout(function(){timedout();}, timeout);
	xmlhttp.onreadystatechange=function(){callback();}

	xmlhttp.setRequestHeader("Content-type","application/x-www-form-urlencoded");
	xmlhttp.send(junk);
}

// Run this if we time out
function timedout()
{
	if(typeof(xmlhttp) == 'object')
		xmlhttp.abort();

	if(attempts < max_attempts)
	{
		// We timed out. Back off the time-out a bit.
		attempts++;
		timeout = timeout + 200;
		// We're adding 200ms to the timeout each iteration, YMMV.
		// Maybe if attempts == max_attempts make the timeout really huge.

		// Play it again, Sam!
		driver();
	}
	else
	{
		// We failed. Set the timeout back to default for the next round.
		timeout = 1000;
		attempts = 0;
		xmlhttp = '';
		alert("Timed out while sending request to server.");
	}
}

ApacheBench Shows Lots of Failed Requests due to Length

Breathe easy. Smile. You’re probably here because you’ve just run ab and got output something like:

This is ApacheBench, Version 2.0.40-dev <$Revision: 1.146 $> apache-2.0
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Copyright 2006 The Apache Software Foundation, http://www.apache.org/

Benchmarking **** (be patient)
Completed 100000 requests
Completed 200000 requests
Completed 300000 requests
Completed 400000 requests
Completed 500000 requests
Completed 600000 requests
Completed 700000 requests
Completed 800000 requests
Completed 900000 requests
Finished 1000000 requests


Server Software:        nginx/1.2.1
Server Hostname:        ****
Server Port:            80

Document Path:          ****
Document Length:        162 bytes

Concurrency Level:      5
Time taken for tests:   9502.884921 seconds
Complete requests:      1000000
Failed requests:        697730
   (Connect: 0, Length: 697730, Exceptions: 0)
Write errors:           0
Total transferred:      279019852 bytes
Total POSTed:           247000494
HTML transferred:       158019731 bytes
Requests per second:    105.23 [#/sec] (mean)
Time per request:       47.514 [ms] (mean)
Time per request:       9.503 [ms] (mean, across all concurrent requests)
Transfer rate:          28.67 [Kbytes/sec] received
                        25.38 kb/s sent
                        54.06 kb/s total

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   15  26.7     13    3183
Processing:     0   31  18.8     29    2319
Waiting:        0   30  15.9     28    1719
Total:          0   46  38.9     42    4333

Percentage of the requests served within a certain time (ms)
  50%     42
  66%     45
  75%     46
  80%     47
  90%     51
  95%     55
  98%    104
  99%    201
 100%   4333 (longest request)

697730 out of 1 million requests failed? No, not really.

ApacheBench expects to be run against something that produces consistent output. Chances are you’ve specified a script that has dynamic output and the length of that output has changed since the first pull.

Let’s have a nice cup of tea :)

Return top
foxpa.ws
Online Marketing Toplist
Internet
Technology Blogs - Blog Rankings

Internet Blogs - BlogCatalog Blog Directory

Technology blogs
Bad Karma Networks

Please Donate!


Made in Canada  •  There's a fox in the Gibson!  •  2010-12