If you're tuning in today you have the opportunity to watch a zero-day attack and response in action.
In the last 12 hours I've received a message from two different VPS providers explaining that they've taken down their SolusVM web-based VM management software due to a severe vulnerability:
Hello guys,
we learned about a nasty security leak in solusVM today and we decided to switch the SolusVM admin-panel off.
We hope that Solus as the company will soon release a patch that will fix most recent leaks as this is not the first one today.
Please check our status-page at status.edis.at for updates.
A couple of providers have already been hacked and their client's data and passwords have been leaked or entire hosting-platforms have been wiped.That's why we decided to shutdown the panel as a preventive measure. If you need reinstalls or reboots, just submit a ticket - we will try to help as fast as we can.We'd like to point out to the fact that this is not a technical flaw on our side.
Thanks a lot for your understanding!
[Waveride]
It has come to our attention that SolusVm (the VPS control panel) may have some exploitable vulnerabilities which we are not aware of.
As a precaution we turned off our SolusVM panel untill a fix is released.
This is not the previous central backup vulnerability which we were patched against, but alledgedly newly discovered vulnerabilities that are about to be disclosed soon.
What does this mean for you:
1. No data is lost.
2. The VPSes themselves are up and running (unless unrelated incidents happen)
3. You can connect using SSH or your control panel, however, the console is part of solus and wont be available, so be careful not to get locked out of SSH for the next day or so.
4. Provisioning of new VPSes, while techincally possible, if done outside solus might result in various disfunctionalities, therefore you can opt for a refund or wait until we think it is safe enough to re-enable solus.
5. Billing panels are still available, but we are limited in what we can do. We use solus too for many tasks, but we will try our best to help you, so log a ticket if you need help.
6. There has been no database leak, no other compromise of any data and solus itself does not store those anyway except some basic things like name and IPs.
7. The billing panels take their data from Solus (traffic consumed, VM status) and is doing any action such as reboot, shutdown through solus too, therefore these functions will not be available and your VPS will appear as offline, when in fact it is not, use SSH for any urgent tasks you may have.
8. While your data is intact and VMs have not been touched, please remember we offer free FTP space and do a backup for the data you think is important enough to be saved. We are an unmanaged host and may or may not have back-ups in case of a disaster like a major hack, an earthquake or fire, for example. Biz plans benefit from offsite backups too.
9. We are using third party software (it is impossible not to, even linux kernel is a third party software we have no control of) and we are dependent on the respective vendors to keep their software secure, therefore, in spite of our best oefforts (and this is valid for everyone) we cannot be immune to hacks. Nobody is, so, one more reason to keep recent backups.
We are sorry for these problems, unfortunately, since we cant do anything to fix them, we choose to turn off the vulnerable software until a fix is released.
We will try to keep you updated here:
http://board.prometeus.net/viewforum.php?f=15
This is, as you can see, valid for other problems, as well.
Thank you for our understanding and support !
[Prometeus]
domVPS has also apparently shut down their SolusVM portal but has not yet issued a statement by e-mail.
You can watch Soluslabs' response on their blog at http://blog.soluslabs.com/. So far a fix has been released for one vulnerability but at least one other has popped up:
We are aware of the current rumours regarding a further security issue with SolusVM as well as some snippets of code. We have been working hard to audit all of the SolusVM code to find any further potential security issues that may pose a threat.
At this moment we have been unable to locate any problems however we are continuing to search for any possible attack vectors. We have received a few blocks of code from some customers that are currently being reviewed. Should any issues be identified a patch will be released immediately along with further announcement.
In the meantime, we do not believe there to by any immediate threat to customers.
Further updates will be provided within the coming hours.
Thank you for your patience and continued support.
I sympathize. These poor buggers are going to endure a lot of ball-breaking and code sifting.
Unfortunately, Soluslabs doesn't seem to be planning on releasing details of the exploit for another few days.
Fortunately, we don't have to rely on them. From LEB:
Today has been an unfortunate day for many hosts and indeed a shocking eye-opener for anyone using SolusVM to offer VPS’ to the public. Earlier on today the website localhost.re reported on a shocking SolusVM exploit that effects every SolusVM version – the now defunct/unused file centralbackup.php contained multiple blunders including SQL Injection, direct exec()ution of any command, and access to the SolusVM server-side binary which can execute any command. Unfortunately for hosts this was a surprise to say the least, and one of the first to be targetted seems to be RamNode.
...
An announcement from RamNode was soon released and it was confirmed that Robert Clarke, founder of ServerCrate, was behind the initial breach of security at RamNode via the exploit. “As you are all aware, this has been a nightmare for [us]. Robert Clarke ran the SolusVM exploit on our control panel early this morning. Someone, him or else, then logged into several nodes and wiped the data.”
Members of LowEndTalk did post findings that correlate with the above statement that Robert Clarke was behind the attack/intrusion. Evidence such as IP-matches & even confirmation that the IP was indeed Roberts’ home network (via the welcome page for a HP media server which clearly stated “Robert’s Pictures” with the hostname ‘clarkeone.homeserver.com’) – not especially good news considering Robert’s previously dubious history and not so great reputation in the industry. While Robert has admitted to the initial “testing” of the exploit he still protests his innocence and vehemently denies doing any of the damange. *Update* Robert has admitted to perpetrating it to several different people. It also appears he targeted BuyVM.
Now that's just a good story.
At present, LowEndTalk (the forum cited above) is down. Whether that means they are being hosted on a VPS by an affected provider or it has earned the ire of its many DoS-keen members for burning Clarke or not is yet unknown.
Before anyone gets out the pitch forks let's heed this LEB commenter's wise words:
Now that everyone has all but burnt Robert at the stake, it is worth considering that this exploit appeared on the net to be then immediately broadcast in many locations, not least the home of the child VPS provider and DDoS hive that is Lowendtalk.
If Robert did cause the issues at Ramnode it is likely, or actually definite that he was simply one of a much larger group of people cutting their way through providers trying to get a “hit”, he was lucky as it were and found Ramnode, others are in the same position. I know of at least 4 with varying degrees of repair work required.
Whilst I am not condoning what he did, if he did it, it is easy to focus in an target him, yet from what Nick has said he is can only be sure Robert accessed something rather than did anything. I am sure all you providers can check your logs and see countless others “all of a sudden” waking up and becoming active in the apparent name of “just testing to check everything is ok”.
Innocent until proven guilty in a court of law, I always say!
