Posts Tagged ‘vm’

Cheap VPS Showdown: Waveride vs VPS Database vs Iniz vs Prometeus vs domVPS

I decided it would be fun to deploy a global network of cheap, disposable reverse proxies and stumbled upon Low End Box. If you can get past the spring green it’s a wonderful site that catalogues VPS deals in the $10-and-under range, many of them exclusive to “LEB”.

I took out VPSes with the following providers:

  • Waveride (Austria) – 4G RAM 2T Xit 50G store
  • VPS Database (Russia) – 512M RAM 1T Xit 50G store
  • Iniz (The Netherlands) – 2G RAM 1T Xit 100G store
  • Prometeus (Italy) – 1G RAM 4T Xit 30G store
  • domVPS (Germany) – 1G RAM 1T Xit 60G store

Though some of the providers I list here sell KVM or other real virtual machines all of the ones I tried were OpenVZ containers. This was my first time with OpenVZ and at best I would call it a learning experience. I learned:

  • OpenVZ Containers are distinct from true virtual machines. They are one real machine that employs a bag of tricks to fool you and your software into believing you are in a distinct environment.
  • As such, they are about as flimsy as a chroot jail. (I’m going to take some heat for that one!)
  • OpenVZ Containers are perfect if you want to oversell the hell out of your machine. There is virtually no overhead, provisioning is instant and resource allocation can be slid around easily.
  • Since OpenVZ doesn’t provide the same level of hardware abstraction you get with real virtualization platforms snmpd requires some sort of buggery to work. I got it to work out of the box on two out of the five VPSes and gave up on the other three. Not for lack of trying.

I also picked up some useful stereotypes about the providers themselves:

  • Pretty much everyone uses SolusVM to manage the virtual machines.
  • Most of them use WHMCS for their CRM.
  • Integration between these systems is poor. Would be nice to manage tickets from inside SolusVM.
  • Some providers do not automatically start your container when the node is rebooted/powered on to prevent the server from seizing. OK then, you’re thinking, they must have the good sense to do a staggered startup instead. They do not.
  • Due to the low cost of their VPSes these providers tend to attract unsavory folks. Consequentially, some of them are frequently under attack.
  • Many cheap VPS providers are “just some dude” with a cheap dedicated and some turn-key software. Do not have high expectations for support, nevermind support response times. Or competence.

Now I had my sweet global network of reverse proxies all set up. It was time to roll out Round Robin DNS to distribute the load across them. As I explained in detail in my recent article Quick and Dirty (and Free!) Host Monitoring for DNS Failover and Round-Robin it is necessary to monitor for and pull downed servers from the A pool. The script I posted in that article has been running for about two weeks now, logging away. From this we can extrapolate some useful information about the reliability of these five providers.

The Environment
Let’s establish the testing environment a bit so the providers can’t cry foul. The script runs once every 5 minutes from a Xen PV VM on one of my many dedicated servers in Montreal. The DC’s network has a 100% uptime SLA. The uptime on this VM is:

11:39:36 up 239 days, 20:08,  1 user,  load average: 0.00, 0.00, 0.00

The latency from this server to the VPSes (as of today) is:

  • Waveride – 115ms
  • VPS Database – 133ms
  • Iniz – 85ms
  • Prometeus – 98ms
  • domVPS – 103ms

The load average on each VPS is at or near 0.00. I am moving about 1mbit/s at 95th percentile through each of them and performing no disk caching. My whole operation is running out of about 50MB of RAM on each VPS without using any disk i/o – and believe me disk i/o is where you really get to see the overselling so I think we’re being more than fair.

Just so you don’t think my host monitoring method is unreliable or faulty I should tell you that I also run Icinga (a fork of Nagios) at my office here in Toronto. It’s monitoring not only pings but also HTTP, HTTPS and DNS. The alert record matches up. Frankly, I’m sick of my cellphone screaming at me.

The Raw Data
For the first week of data collection I was using the script I posted unaltered. If ALL four pings failed a log entry would be made and the server would be pulled. If even one ping made it back it would stay.

Tue May 14 15:05:24 EDT 2013 178.238.236.domVPS is down
Tue May 14 15:25:24 EDT 2013 178.238.236.domVPS is down
Tue May 14 15:40:24 EDT 2013 178.238.236.domVPS is down
Tue May 14 16:15:23 EDT 2013 178.238.236.domVPS is down
Tue May 14 21:30:17 EDT 2013 37.247.53.Prometeus is down
Tue May 14 21:35:16 EDT 2013 37.247.53.Prometeus is down
Tue May 14 21:40:06 EDT 2013 151.236.10.Waveride is down
Tue May 14 21:45:14 EDT 2013 151.236.10.Waveride is down
Tue May 14 21:50:05 EDT 2013 151.236.10.Waveride is down
Tue May 14 21:55:05 EDT 2013 151.236.10.Waveride is down
Tue May 14 22:00:07 EDT 2013 151.236.10.Waveride is down
Tue May 14 22:05:04 EDT 2013 151.236.10.Waveride is down
Tue May 14 22:10:06 EDT 2013 151.236.10.Waveride is down
Tue May 14 22:15:04 EDT 2013 151.236.10.Waveride is down
Wed May 15 03:25:07 EDT 2013 151.236.10.Waveride is down
Wed May 15 03:30:06 EDT 2013 151.236.10.Waveride is down
Wed May 15 03:35:05 EDT 2013 151.236.10.Waveride is down
Wed May 15 03:40:06 EDT 2013 151.236.10.Waveride is down
Wed May 15 03:45:05 EDT 2013 151.236.10.Waveride is down
Wed May 15 03:50:07 EDT 2013 151.236.10.Waveride is down
Wed May 15 03:55:06 EDT 2013 151.236.10.Waveride is down
Wed May 15 04:00:05 EDT 2013 151.236.10.Waveride is down
Wed May 15 04:05:07 EDT 2013 151.236.10.Waveride is down
Wed May 15 04:10:06 EDT 2013 151.236.10.Waveride is down
Wed May 15 04:15:06 EDT 2013 151.236.10.Waveride is down
Wed May 15 04:20:08 EDT 2013 151.236.10.Waveride is down
Wed May 15 04:25:07 EDT 2013 151.236.10.Waveride is down
Wed May 15 05:25:14 EDT 2013 151.236.10.Waveride is down
Wed May 15 07:25:21 EDT 2013 95.47.118.Iniz is down
Wed May 15 07:30:20 EDT 2013 95.47.118.Iniz is down
Wed May 15 07:35:20 EDT 2013 95.47.118.Iniz is down
Wed May 15 07:40:20 EDT 2013 95.47.118.Iniz is down
Wed May 15 07:45:20 EDT 2013 95.47.118.Iniz is down
Wed May 15 07:50:20 EDT 2013 95.47.118.Iniz is down
Wed May 15 09:35:17 EDT 2013 37.0.122.VPSDatabase is down
Wed May 15 15:15:24 EDT 2013 178.238.236.domVPS is down
Thu May 16 04:20:17 EDT 2013 37.0.122.VPSDatabase is down
Thu May 16 06:50:17 EDT 2013 37.0.122.VPSDatabase is down
Thu May 16 07:35:18 EDT 2013 37.0.122.VPSDatabase is down
Thu May 16 12:40:14 EDT 2013 151.236.10.Waveride is down
Thu May 16 14:55:23 EDT 2013 178.238.236.domVPS is down
Thu May 16 17:35:18 EDT 2013 37.0.122.VPSDatabase is down
Fri May 17 02:10:17 EDT 2013 37.0.122.VPSDatabase is down
Fri May 17 04:10:17 EDT 2013 37.0.122.VPSDatabase is down
Fri May 17 04:20:17 EDT 2013 37.0.122.VPSDatabase is down
Fri May 17 04:30:17 EDT 2013 37.0.122.VPSDatabase is down
Fri May 17 04:35:17 EDT 2013 37.0.122.VPSDatabase is down
Fri May 17 05:35:17 EDT 2013 37.0.122.VPSDatabase is down
Fri May 17 05:50:18 EDT 2013 37.0.122.VPSDatabase is down
Fri May 17 08:50:17 EDT 2013 37.0.122.VPSDatabase is down
Fri May 17 09:55:18 EDT 2013 37.0.122.VPSDatabase is down
Fri May 17 12:00:17 EDT 2013 37.0.122.VPSDatabase is down
Fri May 17 14:45:17 EDT 2013 37.0.122.VPSDatabase is down
Fri May 17 16:05:14 EDT 2013 151.236.10.Waveride is down
Fri May 17 16:45:17 EDT 2013 37.0.122.VPSDatabase is down
Fri May 17 17:30:18 EDT 2013 37.0.122.VPSDatabase is down
Sat May 18 00:55:17 EDT 2013 37.0.122.VPSDatabase is down
Sat May 18 05:10:14 EDT 2013 151.236.10.Waveride is down
Sat May 18 05:15:14 EDT 2013 151.236.10.Waveride is down
Sun May 19 10:15:14 EDT 2013 151.236.10.Waveride is down
Sun May 19 13:45:24 EDT 2013 178.238.236.domVPS is down
Sun May 19 13:55:23 EDT 2013 178.238.236.domVPS is down
Sun May 19 14:45:23 EDT 2013 178.238.236.domVPS is down
Sun May 19 14:50:23 EDT 2013 178.238.236.domVPS is down
Sun May 19 15:10:23 EDT 2013 178.238.236.domVPS is down
Sun May 19 15:15:24 EDT 2013 178.238.236.domVPS is down
Sun May 19 19:25:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 19:40:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 20:20:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 20:25:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 20:30:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 20:35:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 20:40:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 20:45:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 20:50:18 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 20:55:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 21:00:18 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 21:05:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 21:10:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 21:15:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 21:20:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 21:25:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 21:30:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 21:35:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 21:40:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 21:45:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 21:50:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 21:55:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 22:00:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 22:05:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 22:10:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 22:15:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 22:20:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 22:25:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 22:30:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 22:35:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 22:40:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 22:45:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 22:50:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 22:55:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 23:00:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 23:05:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 23:10:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 23:15:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 23:20:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 23:25:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 23:30:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 23:35:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 23:40:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 23:45:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 23:50:17 EDT 2013 37.0.122.VPSDatabase is down
Sun May 19 23:55:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 00:00:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 00:05:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 00:10:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 00:15:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 00:20:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 00:25:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 00:30:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 00:35:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 00:40:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 00:45:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 00:50:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 00:55:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 01:00:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 01:05:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 01:10:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 01:15:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 01:20:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 01:25:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 01:30:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 01:35:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 01:40:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 01:45:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 01:50:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 01:55:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 02:00:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 02:05:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 02:10:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 02:15:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 02:20:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 02:25:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 02:30:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 02:35:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 02:40:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 02:45:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 02:50:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 11:30:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 11:40:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 11:45:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 11:50:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 11:55:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 12:05:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 12:10:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 12:15:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 12:25:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 12:30:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 12:35:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 12:40:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 12:45:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 12:50:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 12:55:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 13:00:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 13:05:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 13:10:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 13:15:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 13:20:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 13:25:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 13:30:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 13:35:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 13:40:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 13:45:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 13:50:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 13:55:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 14:00:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 14:05:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 14:10:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 14:15:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 14:20:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 14:25:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 14:30:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 14:35:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 14:40:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 14:45:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 14:50:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 14:55:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 15:00:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 15:05:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 15:10:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 15:15:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 15:20:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 15:25:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 15:30:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 15:35:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 15:40:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 15:45:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 15:50:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 15:55:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 16:00:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 16:05:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 16:10:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 16:15:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 16:20:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 16:25:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 16:30:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 16:35:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 16:40:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 16:45:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 16:50:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 16:55:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 17:00:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 17:05:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 17:10:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 17:15:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 17:20:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 17:25:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 17:30:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 17:35:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 17:40:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 17:45:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 17:50:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 17:55:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 18:00:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 18:05:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 18:10:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 18:15:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 18:20:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 18:25:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 18:30:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 18:35:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 18:40:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 18:40:30 EDT 2013 95.47.118.Iniz is down
Mon May 20 18:45:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 18:50:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 18:55:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 19:00:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 19:05:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 19:10:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 19:15:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 19:20:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 19:25:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 19:30:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 19:35:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 19:40:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 19:45:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 19:50:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 19:55:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 20:00:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 20:05:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 20:10:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 20:15:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 20:20:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 20:25:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 20:30:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 20:35:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 20:40:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 20:45:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 20:50:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 20:55:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 21:00:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 21:05:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 21:10:18 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 21:15:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 21:20:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 21:25:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 21:30:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 21:35:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 21:40:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 21:45:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 21:50:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 21:55:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 22:00:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 22:05:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 22:10:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 22:15:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 22:20:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 22:25:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 22:30:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 22:35:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 22:40:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 22:45:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 22:50:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 22:55:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 23:00:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 23:05:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 23:10:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 23:15:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 23:20:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 23:25:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 23:30:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 23:35:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 23:40:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 23:45:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 23:50:17 EDT 2013 37.0.122.VPSDatabase is down
Mon May 20 23:55:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 00:00:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 00:05:18 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 00:10:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 00:15:18 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 00:20:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 02:00:24 EDT 2013 178.238.236.domVPS is down
Tue May 21 07:35:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 07:40:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 07:45:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 07:50:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 07:55:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 08:05:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 08:10:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 08:15:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 08:20:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 09:20:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 09:30:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 09:40:18 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 10:00:18 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 10:05:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 10:10:18 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 10:15:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 10:20:18 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 10:25:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 10:30:18 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 10:35:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 10:50:18 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 11:00:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 11:05:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 11:15:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 11:25:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 11:30:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 11:35:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 11:40:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 11:45:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 11:50:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 11:50:30 EDT 2013 95.47.118.Iniz is down
Tue May 21 11:55:18 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 12:00:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 12:05:18 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 12:10:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 12:15:18 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 12:25:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 12:30:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 12:35:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 12:40:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 12:45:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 12:50:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 12:55:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 13:00:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 13:05:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 13:10:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 13:15:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 13:20:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 13:25:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 13:30:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 13:35:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 13:40:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 13:45:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 13:50:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 13:55:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 14:00:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 14:05:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 14:10:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 14:15:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 14:20:17 EDT 2013 37.0.122.VPSDatabase is down
Tue May 21 14:20:30 EDT 2013 95.47.118.Iniz is down
Tue May 21 14:25:17 EDT 2013 37.0.122.VPSDatabase is down

On the second week I changed the script to fail a server if any less than four of four pings were returned:

Tue May 21 14:27:23 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 14:30:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 14:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 14:40:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 14:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 14:45:21 EDT 2013 95.47.118.Iniz is down 2 of 4 packets succeeded.
Tue May 21 14:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 14:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 15:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 15:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 15:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 15:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 15:20:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 15:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 15:30:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 15:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 15:40:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 15:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 15:50:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 15:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 16:00:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 16:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 16:10:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 16:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 16:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 16:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 16:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 16:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 16:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 16:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 16:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 16:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 17:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 17:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 17:05:30 EDT 2013 95.47.118.Iniz is down 1 of 4 packets succeeded.
Tue May 21 17:10:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 17:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 17:20:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 17:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 17:30:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 17:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 17:40:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 17:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 17:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 17:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 18:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 18:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 18:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 18:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 18:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 18:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 18:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 18:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 18:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 18:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 18:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 18:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 19:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 19:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 19:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 19:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 19:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 19:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 19:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 19:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 19:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 19:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 19:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 19:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 20:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 20:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 20:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 20:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 20:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 20:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 20:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 20:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 20:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 20:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 20:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 20:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 21:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 21:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 21:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 21:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 21:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 21:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 21:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 21:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 21:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 21:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 21:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 21:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 22:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 22:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 22:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 22:15:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 22:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 22:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 22:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 22:35:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 22:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 22:45:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 22:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 22:55:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 23:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 23:05:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 23:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 23:15:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 23:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 23:25:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 23:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 23:35:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 23:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 23:45:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 23:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 21 23:55:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 00:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 00:05:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 00:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 00:15:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 00:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 00:25:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 00:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 00:35:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 00:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 00:45:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 00:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 00:55:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 01:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 01:05:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 01:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 01:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 01:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 01:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 01:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 01:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 01:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 01:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 01:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 01:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 02:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 02:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 02:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 02:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 02:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 02:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 02:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 02:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 02:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 02:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 02:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 02:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 03:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 03:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 03:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 03:15:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 03:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 03:25:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 03:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 03:35:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 03:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 03:45:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 03:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 03:55:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 04:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 04:05:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 04:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 04:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 04:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 04:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 04:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 04:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 04:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 04:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 04:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 04:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 05:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 05:00:21 EDT 2013 95.47.118.Iniz is down 2 of 4 packets succeeded.
Wed May 22 05:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 05:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 05:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 05:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 05:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 05:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 05:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 05:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 05:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 05:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 05:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 06:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 06:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 06:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 06:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 06:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 06:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 06:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 06:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 06:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 06:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 06:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 06:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 07:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 07:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 07:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 07:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 07:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 07:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 07:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 07:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 07:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 07:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 07:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 07:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 08:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 08:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 08:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 08:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 08:20:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 08:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 08:30:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 08:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 08:40:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 08:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 08:50:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 08:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 09:00:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 09:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 09:10:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 09:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 09:20:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 09:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 09:30:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 09:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 09:40:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 09:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 09:50:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 09:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 10:00:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 10:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 10:10:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 10:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 10:20:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 10:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 10:30:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Wed May 22 11:40:11 EDT 2013 95.47.118.Iniz is down 1 of 4 packets succeeded.
Wed May 22 11:45:11 EDT 2013 95.47.118.Iniz is down 2 of 4 packets succeeded.
Wed May 22 11:50:11 EDT 2013 95.47.118.Iniz is down 3 of 4 packets succeeded.
Wed May 22 12:00:12 EDT 2013 95.47.118.Iniz is down 2 of 4 packets succeeded.
Wed May 22 12:05:11 EDT 2013 95.47.118.Iniz is down 3 of 4 packets succeeded.
Wed May 22 13:35:05 EDT 2013 151.236.10.Waveride is down 3 of 4 packets succeeded.
Wed May 22 16:40:08 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Wed May 22 18:20:14 EDT 2013 151.236.10.Waveride is down 1 of 4 packets succeeded.
Wed May 22 18:25:14 EDT 2013 151.236.10.Waveride is down 1 of 4 packets succeeded.
Wed May 22 19:10:05 EDT 2013 151.236.10.Waveride is down 3 of 4 packets succeeded.
Wed May 22 19:10:09 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Wed May 22 19:10:13 EDT 2013 95.47.118.Iniz is down 3 of 4 packets succeeded.
Wed May 22 19:10:17 EDT 2013 178.238.236.domVPS is down 3 of 4 packets succeeded.
Wed May 22 19:10:21 EDT 2013 37.247.53.Prometeus is down 3 of 4 packets succeeded.
Wed May 22 23:05:09 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Wed May 22 23:10:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Wed May 22 23:45:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Thu May 23 00:15:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Thu May 23 00:20:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Thu May 23 11:20:23 EDT 2013 178.238.236.domVPS is down 0 of 4 packets succeeded.
Thu May 23 15:10:14 EDT 2013 178.238.236.domVPS is down 3 of 4 packets succeeded.
Thu May 23 22:25:05 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Thu May 23 22:30:07 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Thu May 23 22:35:07 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Thu May 23 22:40:08 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Thu May 23 22:45:07 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Thu May 23 22:50:06 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Thu May 23 22:55:07 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Thu May 23 23:00:04 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Thu May 23 23:05:08 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Thu May 23 23:10:06 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Thu May 23 23:15:06 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Thu May 23 23:20:07 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Thu May 23 23:25:06 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Thu May 23 23:30:07 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Thu May 23 23:35:05 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Thu May 23 23:40:07 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Fri May 24 09:50:20 EDT 2013 95.47.118.Iniz is down 0 of 4 packets succeeded.
Fri May 24 10:00:11 EDT 2013 95.47.118.Iniz is down 1 of 4 packets succeeded.
Fri May 24 17:50:23 EDT 2013 178.238.236.domVPS is down 0 of 4 packets succeeded.
Fri May 24 19:55:17 EDT 2013 37.247.53.Prometeus is down 3 of 4 packets succeeded.
Sat May 25 20:25:17 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Sat May 25 20:30:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Sat May 25 23:25:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Sun May 26 02:30:20 EDT 2013 95.47.118.Iniz is down 0 of 4 packets succeeded.
Sun May 26 07:55:24 EDT 2013 178.238.236.domVPS is down 0 of 4 packets succeeded.
Sun May 26 08:00:23 EDT 2013 178.238.236.domVPS is down 0 of 4 packets succeeded.
Sun May 26 08:05:23 EDT 2013 178.238.236.domVPS is down 0 of 4 packets succeeded.
Sun May 26 08:10:23 EDT 2013 178.238.236.domVPS is down 0 of 4 packets succeeded.
Sun May 26 09:25:14 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Sun May 26 09:35:14 EDT 2013 151.236.10.Waveride is down 0 of 4 packets succeeded.
Sun May 26 13:35:09 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Sun May 26 13:40:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Sun May 26 17:30:12 EDT 2013 95.47.118.Iniz is down 2 of 4 packets succeeded.
Mon May 27 08:30:09 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 08:50:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 11:30:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Mon May 27 11:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Mon May 27 11:50:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 12:05:20 EDT 2013 95.47.118.Iniz is down 1 of 4 packets succeeded.
Mon May 27 12:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Mon May 27 13:10:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Mon May 27 13:20:08 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Mon May 27 13:30:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 13:40:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 14:00:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Mon May 27 14:20:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 14:45:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 15:00:17 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Mon May 27 15:35:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 15:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Mon May 27 15:55:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Mon May 27 16:15:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 16:25:08 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Mon May 27 16:40:08 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Mon May 27 19:05:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 19:15:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 19:25:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Mon May 27 19:30:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 19:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Mon May 27 20:15:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 20:20:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 20:30:09 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 20:40:18 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Mon May 27 20:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Mon May 27 21:25:17 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Mon May 27 21:40:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 21:45:18 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Mon May 27 21:45:22 EDT 2013 95.47.118.Iniz is down 3 of 4 packets succeeded.
Mon May 27 22:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Mon May 27 22:05:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Mon May 27 22:10:27 EDT 2013 37.247.53.Prometeus is down 0 of 4 packets succeeded.
Mon May 27 22:15:17 EDT 2013 37.247.53.Prometeus is down 1 of 4 packets succeeded.
Mon May 27 22:20:18 EDT 2013 37.247.53.Prometeus is down 2 of 4 packets succeeded.
Mon May 27 22:30:09 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 22:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Mon May 27 23:45:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Tue May 28 00:00:08 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Tue May 28 00:10:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 00:10:22 EDT 2013 95.47.118.Iniz is down 2 of 4 packets succeeded.
Tue May 28 00:20:09 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 00:25:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Tue May 28 00:30:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 00:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 00:40:09 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Tue May 28 00:55:17 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Tue May 28 01:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 01:05:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 01:20:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Tue May 28 01:25:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 01:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 01:35:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 01:45:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 02:00:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 02:10:09 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 02:15:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 02:35:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 02:45:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 02:55:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 03:00:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 03:10:17 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Tue May 28 03:15:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 03:20:17 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Tue May 28 03:25:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Tue May 28 03:30:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Tue May 28 03:40:17 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Tue May 28 03:50:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 03:55:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 04:05:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Tue May 28 04:10:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 04:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 04:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 04:30:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 04:40:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 04:50:09 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 05:00:17 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Tue May 28 05:05:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Tue May 28 05:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 05:25:09 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Tue May 28 05:30:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 05:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 05:45:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 05:50:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 06:00:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 06:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 06:15:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 06:25:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 06:30:18 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 06:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 06:45:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 06:55:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Tue May 28 07:05:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 07:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 07:15:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Tue May 28 07:20:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 07:25:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Tue May 28 07:35:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 07:45:09 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Tue May 28 07:50:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 08:00:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 08:05:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 08:10:05 EDT 2013 151.236.10.Waveride is down 2 of 4 packets succeeded.
Tue May 28 08:10:09 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 08:15:09 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 08:20:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 08:25:09 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 08:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 08:50:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Tue May 28 09:30:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 09:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 09:40:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Tue May 28 09:45:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 09:50:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Tue May 28 10:10:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.

Lies, Damned Lies and Statistics
If there’s one thing we can tell right off the bat VPS Database is one of those providers that won’t start your container automatically after the node boots. It’s also one of those providers that is constantly under DoS attack.

The first time it went down for an extended period I filed a ticket 10/05/13:

Hello,

I’ve noticed that my VPS has been up and down since 16h45EDT

Are there problems on the node/network?

Is there anything I can do to help?

Thank you.

ALL VPS are still up.

Kind Regards,
Victor Enciso – CE

Hello,

Could you please then explain why 37.0.122.VPSDatabase has been un-pingable for the past six hours?

The one time I was able to log into it briefly this afternoon it showed that the VPS had not crashed or been powered off.

However, there is clearly a networking problem.

Thanks

I did the following ;
Shut down.
Waited 60seconds
Boot
SSH and got a response back.

Kind Regards,
Victor Enciso – CEO

This is the moment where I learned that if your VM goes down for more than half an hour you probably need to log in and reboot it manually. Interestingly, SolusVM always seems to show these not-started containers as online regardless of the fact that they are not running.

On the 20th VPS Database customers received the following e-mail:

We have discovered that our Russia Server has been experiencing some Kernel Problems and right now we are investigating on what is going on.
Please be patient while we proceed on the issue.
We will report back when the issue has been fixed.

On the 25th we finally received word:

We have fixed the Kernel Issue we faced the last 3 days and the vps should be working properly now.
Sorry again for having the vps going up and down for the past 3 days.

Yep. 3 days’ work to pop in a new kernel.

Yesterday I got fed up and filed another ticket:

Hello,

Network connectivity seems to be an issue again today. The following output comes from a script that runs every 5 minutes:

Mon May 27 12:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Mon May 27 13:10:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Mon May 27 13:20:08 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Mon May 27 13:30:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 13:40:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 14:00:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Mon May 27 14:20:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 14:45:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 15:00:17 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Mon May 27 15:35:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 15:45:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.
Mon May 27 15:55:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Mon May 27 16:15:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 16:25:08 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Mon May 27 16:40:08 EDT 2013 37.0.122.VPSDatabase is down 1 of 4 packets succeeded.
Mon May 27 19:05:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 19:15:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 19:25:08 EDT 2013 37.0.122.VPSDatabase is down 2 of 4 packets succeeded.
Mon May 27 19:30:08 EDT 2013 37.0.122.VPSDatabase is down 3 of 4 packets succeeded.
Mon May 27 19:35:17 EDT 2013 37.0.122.VPSDatabase is down 0 of 4 packets succeeded.

Thanks for looking into this.

I saw from the statics that the vps went up and down 4 times.
As a reboot I would say.

Kind Regards,
Victor Enciso – CEO

Even if you believe I subconciously rebooted my VPS four times that doesn’t explain all of the 1, 2 and 3 out of 4 entries. That’s a considerable level of packet loss.

It’s downright scary that this man considers 75% packet loss acceptable and someone lets him in front of a keyboard. More infuriating is how every problem is casually brushed off like you’re the idiot. Anyone who sells IT services should hold this as an example of exactly how not to run your business.

Extrapolation
In the second week I started counting any response less than four out of four packets as a “down.” Let’s be really – and I mean really – fair and only count the 0-out-of-4 entries in combination with the first week of data (all 0-out-of-4s). I’m going to count each entry as 5 minutes, though we should bear in mind that the downtime in some cases could technically have been as low as five seconds.

  • VPS Database – 591 entries – 2955 minutes
  • Waveride – 45 entries – 225 minutes
  • domVPS – 19 entries – 95 minutes
  • Iniz – 11 entries – 55 minutes
  • Prometeus – 3 entries – 15 minutes

VPS Database certainly isn’t helped by the fact that I had to log in and manually boot the thing half a dozen times just to get it running again, but at almost 50 hours of downtime I think we can safely draw some conclusions.

Now to get an idea of who may be suffering from packet loss let’s take a look at the second week’s data alone:

  • VPS Database – 367 entries
  • Waveride – 23 entries
  • Iniz – 16 entries
  • domVPS – 8 entries
  • Prometeus – 5 entries

We can see things appear to scale proportionately.

Awards

  • The professionalism award goes to Iniz. Iniz is well established and their staff are courteous. They suffered a DoS (admirably) while I was recording this data and their management interface was down for about 2 days while they moved it to a different domain (and I’m assuming location, away from all the collateral damage) but I endured little downtime.
  • The uptime and value awards both go to Prometeus. Prometeus is 16 years old and I never had to file a ticket. As the statistics show, they performed better than any other provider. With 4T of transit and (in European terms) low latency from North America you can hardly go wrong.
  • The assclown award goes to VPS Database. Server bad. Service bad. Uptime bad bad bad. This is the only one that truly left a foul taste in my mouth.

Conclusions

  1. You get what you pay for.
  2. OpenVZ is rubbish, unless you’re a provider.
  3. If you’re doing anything that requires a modicum of reliability do not even consider a cheap VPS. Of course, if that’s the case you shouldn’t really consider a VPS of any sort. Get at least a cheap dedicated or colo and run your own Xen or KVM setup.

Special Mention
Although Minecrafted was not a part of my monitoring program I feel I should give them a special mention. I took out a VPS with them through their LEB promo at the same time I started with the OpenVZ providers and it has been a dream. They run Xen PV VMs on SSDs and I have been routing a low-latency AJAX based web chat via a reverse proxy, running RTMP streams and a minecraft server on one without so much as a peep from Icinga or a hiccup in the chat. Jolly good show.

Addendum
I took some justified grief on Low End Talk (the sister-site to LEB) when I posted this article, mostly because a single ping monitor can’t take into account things like routing problems that may be out of the provider’s control. I responded by creating Crossfire Ping Monitoring and you can see the first logs here:
crossfire
I think it vindicates my alleged “hate-on” for VPS Database. Please read the next series which was inspired by advice from LET users:

Convert a QCOW Image to Raw Disk Format (QEMU, Xen)

qemu-img convert -O raw diskimage.qcow diskimage.raw

Removing virbr0 or Why The Fsck is My Dom0 NATting?

I noticed one of my new Xen dom0s was coughing up our friend, the ip_conntrack: table full, dropping packet message today. If you like to get your money’s worth out of your dedis the RAM available to dom0 is probably limited – meaning a correspondingly low default ip_conntrack_max. I’m sure you can see how this might be a problem, even more so if it is lower than the ip_conntrack_max of your virtual machines.

None of my previous CentOS dedis had NAT/conntrack modules loaded by default and this dom0 had no need for NAT – being of a fully bridged configuration and routing only public IPs. My first guess was that this dedi’s redhatty initrd loaded the modules through the typical mash-everything-against-the-kernel-and-see-what-sticks approach so I tried removing the NAT and connection tracking related modules:

# rmmod iptable_nat
ERROR: Module iptable_nat is in use

OK, let’s take a look at the tables:

[root@cl-t067-252cl ~]# iptables-save
# Generated by iptables-save v1.3.5 on Sat Jul 21 21:27:40 2012
*nat
:PREROUTING ACCEPT [931:50495]
:POSTROUTING ACCEPT [446:25128]
:OUTPUT ACCEPT [7:502]
-A POSTROUTING -s 192.168.122.0/255.255.255.0 -d ! 192.168.122.0/255.255.255.0 -p tcp -j MASQUERADE --to-ports 1024-65535 
-A POSTROUTING -s 192.168.122.0/255.255.255.0 -d ! 192.168.122.0/255.255.255.0 -p udp -j MASQUERADE --to-ports 1024-65535 
-A POSTROUTING -s 192.168.122.0/255.255.255.0 -d ! 192.168.122.0/255.255.255.0 -j MASQUERADE 
COMMIT

It seems I have a subnet I was not aware of…

virbr0    Link encap:Ethernet  HWaddr 00:00:00:00:00:00  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

Who put that there? libvirt, apparently. According to that article not only is our problem ip_conntrack_max, but:

However, NAT slows down things and only recommended for desktop installations.

Seems highly logical to me. Their solution didn’t look very permanent so I first deleted the symlink in the autostart directory for “default”:

# cd /etc/libvirt/qemu/networks/autostart/
# ls -lsah
total 16K
8.0K drwx------ 2 root root 4.0K Jul 21 21:17 .
8.0K drwx------ 3 root root 4.0K May 14 09:18 ..
   0 lrwxrwxrwx 1 root root   14 Jul 21 21:17 default.xml -> ../default.xml
# mv default.xml
# cd ..
# cp default.xml ~/
# /etc/init.d/libvirtd restart

That didn’t do anything at all. Still had virbr0, still had the iptables rules and still had the kernel modules.

Reboot.

Apparently that was the wrong thing to do. All of my interfaces, bridges, etc seemed to come back up (except virbr0) and the NAT/conntrack modules were missing but not a single VM was routing.

On to their method:

# virsh net-destroy default
# virsh net-undefine default
# service libvirtd restart

Everything looks great. You still have the NAT/conntrack modules loaded but we should be able to take those out one by one.

# lsmod | grep nat
iptable_nat            40517  0 
ip_nat                 52973  2 ipt_MASQUERADE,iptable_nat
ip_conntrack           91749  4 ipt_MASQUERADE,iptable_nat,ip_nat,xt_state
nfnetlink              40457  2 ip_nat,ip_conntrack
ip_tables              55329  2 iptable_nat,iptable_filter
x_tables               50377  7 xt_physdev,ipt_MASQUERADE,iptable_nat,xt_state,ipt_REJECT,xt_tcpudp,ip_tables

Reboot.

Boned again.`Now default.xml is missing (I’m assuming that’s what net-destroy does) – good thing we made a backup first!

# cd /etc/libvirt/qemu/networks/
# cp ~/default.xml ./
# ln -s default.xml autostart/
# reboot

OK. Screw it. We’ll do it the hard way.

#!/bin/bash
ifconfig virbr0 down
iptables -t nat -D POSTROUTING -s 192.168.122.0/255.255.255.0 -d ! 192.168.122.0/255.255.255.0 -p tcp -j MASQUERADE --to-ports 1024-65535
iptables -t nat -D POSTROUTING -s 192.168.122.0/255.255.255.0 -d ! 192.168.122.0/255.255.255.0 -p udp -j MASQUERADE --to-ports 1024-65535
iptables -t nat -D POSTROUTING -s 192.168.122.0/255.255.255.0 -d ! 192.168.122.0/255.255.255.0 -j MASQUERADE
iptables -D INPUT -i virbr0 -p udp -m udp --dport 53 -j ACCEPT 
iptables -D INPUT -i virbr0 -p tcp -m tcp --dport 53 -j ACCEPT 
iptables -D INPUT -i virbr0 -p udp -m udp --dport 67 -j ACCEPT 
iptables -D INPUT -i virbr0 -p tcp -m tcp --dport 67 -j ACCEPT 
iptables -D FORWARD -d 192.168.122.0/255.255.255.0 -o virbr0 -m state --state RELATED,ESTABLISHED -j ACCEPT 
iptables -D FORWARD -s 192.168.122.0/255.255.255.0 -i virbr0 -j ACCEPT 
iptables -D FORWARD -i virbr0 -o virbr0 -j ACCEPT 
iptables -D FORWARD -o virbr0 -j REJECT --reject-with icmp-port-unreachable 
iptables -D FORWARD -i virbr0 -j REJECT --reject-with icmp-port-unreachable
rmmod iptable_nat
rmmod ipt_MASQUERADE
rmmod ip_nat
rmmod xt_state
rmmod ip_conntrack

HOW DO YOU LIKE ME NOW?!

Return top
foxpa.ws
Online Marketing Toplist
Internet
Technology Blogs - Blog Rankings

Internet Blogs - BlogCatalog Blog Directory

Bad Karma Networks

Please Donate!


Made in Canada  •  There's a fox in the Gibson!  •  2010-12