Posts Tagged ‘hosting’

Zero Day in Action: SolusVM, Robert Clarke and Juicy Allegations of Corporate Cyberwar

If you’re tuning in today you have the opportunity to watch a zero-day attack and response in action.

In the last 12 hours I’ve received a message from two different VPS providers explaining that they’ve taken down their SolusVM web-based VM management software due to a severe vulnerability:

Hello guys,

we learned about a nasty security leak in solusVM today and we decided to switch the SolusVM admin-panel off.
We hope that Solus as the company will soon release a patch that will fix most recent leaks as this is not the first one today.

Please check our status-page at for updates.

A couple of providers have already been hacked and their client’s data and passwords have been leaked or entire hosting-platforms have been wiped.That’s why we decided to shutdown the panel as a preventive measure. If you need reinstalls or reboots, just submit a ticket – we will try to help as fast as we can.We’d like to point out to the fact that this is not a technical flaw on our side.

Thanks a lot for your understanding!


It has come to our attention that SolusVm (the VPS control panel) may have some exploitable vulnerabilities which we are not aware of.
As a precaution we turned off our SolusVM panel untill a fix is released.
This is not the previous central backup vulnerability which we were patched against, but alledgedly newly discovered vulnerabilities that are about to be disclosed soon.
What does this mean for you:
1. No data is lost.
2. The VPSes themselves are up and running (unless unrelated incidents happen)
3. You can connect using SSH or your control panel, however, the console is part of solus and wont be available, so be careful not to get locked out of SSH for the next day or so.
4. Provisioning of new VPSes, while techincally possible, if done outside solus might result in various disfunctionalities, therefore you can opt for a refund or wait until we think it is safe enough to re-enable solus.
5. Billing panels are still available, but we are limited in what we can do. We use solus too for many tasks, but we will try our best to help you, so log a ticket if you need help.
6. There has been no database leak, no other compromise of any data and solus itself does not store those anyway except some basic things like name and IPs.
7. The billing panels take their data from Solus (traffic consumed, VM status) and is doing any action such as reboot, shutdown through solus too, therefore these functions will not be available and your VPS will appear as offline, when in fact it is not, use SSH for any urgent tasks you may have.
8. While your data is intact and VMs have not been touched, please remember we offer free FTP space and do a backup for the data you think is important enough to be saved. We are an unmanaged host and may or may not have back-ups in case of a disaster like a major hack, an earthquake or fire, for example. Biz plans benefit from offsite backups too.
9. We are using third party software (it is impossible not to, even linux kernel is a third party software we have no control of) and we are dependent on the respective vendors to keep their software secure, therefore, in spite of our best oefforts (and this is valid for everyone) we cannot be immune to hacks. Nobody is, so, one more reason to keep recent backups.

We are sorry for these problems, unfortunately, since we cant do anything to fix them, we choose to turn off the vulnerable software until a fix is released.
We will try to keep you updated here:

This is, as you can see, valid for other problems, as well.

Thank you for our understanding and support !


domVPS has also apparently shut down their SolusVM portal but has not yet issued a statement by e-mail.

You can watch Soluslabs’ response on their blog at So far a fix has been released for one vulnerability but at least one other has popped up:

We are aware of the current rumours regarding a further security issue with SolusVM as well as some snippets of code. We have been working hard to audit all of the SolusVM code to find any further potential security issues that may pose a threat.

At this moment we have been unable to locate any problems however we are continuing to search for any possible attack vectors. We have received a few blocks of code from some customers that are currently being reviewed. Should any issues be identified a patch will be released immediately along with further announcement.

In the meantime, we do not believe there to by any immediate threat to customers.

Further updates will be provided within the coming hours.

Thank you for your patience and continued support.

I sympathize. These poor buggers are going to endure a lot of ball-breaking and code sifting.

Unfortunately, Soluslabs doesn’t seem to be planning on releasing details of the exploit for another few days.

Fortunately, we don’t have to rely on them. From LEB:

Today has been an unfortunate day for many hosts and indeed a shocking eye-opener for anyone using SolusVM to offer VPS’ to the public. Earlier on today the website reported on a shocking SolusVM exploit that effects every SolusVM version – the now defunct/unused file centralbackup.php contained multiple blunders including SQL Injection, direct exec()ution of any command, and access to the SolusVM server-side binary which can execute any command. Unfortunately for hosts this was a surprise to say the least, and one of the first to be targetted seems to be RamNode.

An announcement from RamNode was soon released and it was confirmed that Robert Clarke, founder of ServerCrate, was behind the initial breach of security at RamNode via the exploit. “As you are all aware, this has been a nightmare for [us]. Robert Clarke ran the SolusVM exploit on our control panel early this morning. Someone, him or else, then logged into several nodes and wiped the data.”

Members of LowEndTalk did post findings that correlate with the above statement that Robert Clarke was behind the attack/intrusion. Evidence such as IP-matches & even confirmation that the IP was indeed Roberts’ home network (via the welcome page for a HP media server which clearly stated “Robert’s Pictures” with the hostname ‘’) – not especially good news considering Robert’s previously dubious history and not so great reputation in the industry. While Robert has admitted to the initial “testing” of the exploit he still protests his innocence and vehemently denies doing any of the damange. *Update* Robert has admitted to perpetrating it to several different people. It also appears he targeted BuyVM.

Now that’s just a good story.

At present, LowEndTalk (the forum cited above) is down. Whether that means they are being hosted on a VPS by an affected provider or it has earned the ire of its many DoS-keen members for burning Clarke or not is yet unknown.

Before anyone gets out the pitch forks let’s heed this LEB commenter’s wise words:

Now that everyone has all but burnt Robert at the stake, it is worth considering that this exploit appeared on the net to be then immediately broadcast in many locations, not least the home of the child VPS provider and DDoS hive that is Lowendtalk.

If Robert did cause the issues at Ramnode it is likely, or actually definite that he was simply one of a much larger group of people cutting their way through providers trying to get a “hit”, he was lucky as it were and found Ramnode, others are in the same position. I know of at least 4 with varying degrees of repair work required.

Whilst I am not condoning what he did, if he did it, it is easy to focus in an target him, yet from what Nick has said he is can only be sure Robert accessed something rather than did anything. I am sure all you providers can check your logs and see countless others “all of a sudden” waking up and becoming active in the apparent name of “just testing to check everything is ok”.

Innocent until proven guilty in a court of law, I always say!

Documentary for Dinner: TPB AFK: The Pirate Bay Away From Keyboard (2013)

The much-anticipated The Pirate Bay documentary covers the events surrounding the trial and conviction of TPB founders.

Mass Virtual Hosting Part Three: Disk Quotas (including NFS)

Disk quotas allow one to limit the amount of space a user or group may use on a particular filesystem. The traditional linux quota implementation allows two sorts of limit: soft limits, which the user/group may exceed for a given grace period and hard limits which may not be exceeded at all. Soft limits are great in situations where users may need significant amounts of storage only temporarily, as in the case with burning ISOs or intensive rendering software and other temporary-file generating activity. In a webhosting environment one typically offers different plans with a set limit of storage, so soft limits are probably redundant for our purposes. Since quotas can be applied to any user or group they can also be used to ensure particular daemons do not run roughshod over the filesystem, like apache access logs might during an HTTP GET denial of service attack.

If you will be using NFS to remotely mount filesystems you intend to implement quotas on you must implement them on the NFS server and use some sort of UID/GID consistency like NIS or libnss-mysql. Your kernel must be compiled with quota support or have it available as a module to enforce the limits. It is possible to set up quotas on a system running a quota-incapable kernel then activate them later by incorporating quota support. In menuconfig, set this option to module or compiled-in, if you choose to compile it as a module ensure that it is automatically loaded:

  • File systems
    • Quota support

You must also install the userspace tools, emerge quota on Gentoo. Next add usrquota and/or grpquota depending on your needs to the options field of the target filesystem(s), i.e:

/dev/sdX1               /mnt/storage     ext3            defaults,nosuid,noexec,nodev,noatime,usrquota,grpquota   0 0

In this example we are also disabling binary execution and some potentially dangerous filesystem options such as SUID and device files for security purposes. You may remount the filesystem to apply the changes:

# mount -o remount /mnt/storage

Now in the root of every filesystem to have quotas create and secure the quota files:

# touch /mnt/storage/aquota.user
# touch /mnt/storage/
# chmod 600 /mnt/storage/aquota.user
# chmod 600 /mnt/storage/
# /usr/sbin/quotacheck -avug

Unless you will be exclusively using XFS you must add the quota init script to your runlevel. On Gentoo:

# rc-update add quota boot

Next you must decide how often the quotas are checked, in other words how often the total recorded space users and groups are consuming  is updated. You must weigh the importance of accurate reporting against the potential resource load scanning the filesystem(s) may incur. Drop this scriptlet into a /etc/cron.* directory and chmod +x it:

/usr/sbin/quotacheck -avug

Quotas can be edited by the program edquota, with the -u flag and a user’s name or a -g flag and a group’s name respectively. Use the -f flag and the target filesystem’s mountpoint to restrict operations to one particular filesystem, otherwise edquota will default to all filesystems with quotas enabled. Edquota uses your EDITOR environment variable to load a temporary file containing a tabular representation of a user’s soft and hard quotas as well as currently used space. Simply change the soft and hard quota limits and save the file, the new values will be applied immediately. This is how user test’s quota looks like when edited with nano:

# edquota -f /mnt/storage -u test

Disk quotas for user test (uid 5000):
  Filesystem                   blocks       soft       hard     inodes     soft     hard
  /dev/sdX1                         0          0          0          0        0        0

It should be noted here that quotas can also be set on the number of inodes a user or group may use, effectively limiting the total number of files they can create. This is probably not practical for our needs, where space is the concern and we will mostly be hosting websites composed of relatively many, relatively small files. The blocks and inodes fields tell us how much the user was using the last time quotacheck was run while the soft and hard fields are their limits respectively. Once you have finished configuring the user or group’s quotas simply save and exit the editor and the changes will be saved.

We can use repquota to see a filesystem’s overall use on a per-user basis, simply specify the mountpoint of the filesystem in question:

# repquota /mnt/storage/
*** Report for user quotas on device /dev/sdX1
Block grace time: 7days; Inode grace time: 7days
                        Block limits                File limits
User            used    soft    hard  grace    used  soft  hard  grace
root      --  180240       0       0              7     0     0
test      +- 1738760     500     500  3days       5     0     0

The output is similar to edquota. Note the — and +- column: the first character indicates whether the user’s hard quota is over limit or under limit, denoted by the + and – symbols respectively, and the second character represents the soft quota. In this example we can see user test is very much over their 500 block hard limit. They will not be able to create any more files until they have cleared out enough space to put them back under the limit.

Quotas are fully enforced on an NFS server, but to share information about the quotas to NFS clients you must ensure rpc.rquotad is running. On gentoo alter /etc/conf.d/nfs to reflect:

# Optional services to include in default `/etc/init.d/nfs start`
# For NFSv4 users, you'll want to add "rpc.idmapd" here.
NFS_NEEDED_SERVICES="rpc.idmapd rpc.rquotad"

Then restart your nfs init script:

# /etc/init.d/nfs restart

On the NFS client change the share’s fstab column so that the options field contains quota, for example:

nfs-server:/mnt/storage        /home   nfs             rsize=32768,wsize=8192,soft,timeo=10,rw,intr,nosuid,noexec,nodev,quota          0 0

Remount the filesystem and you should be able to interface with the quotas on the remote NFS server. Be sure to use the -r flag when modifying quotas from the client(s).

Return top
Online Marketing Toplist
Technology Blogs - Blog Rankings

Internet Blogs - BlogCatalog Blog Directory

Bad Karma Networks

Please Donate!

Made in Canada  •  There's a fox in the Gibson!  •  2010-12