Automatically Start Oracle VM VirtualBox and Virtual Machines on Solaris 11

As with starting the VirtualBox Web Service on Oracle Solaris, the documentation regarding starting VirtualBox and auto-starting virtual machines is rather convoluted. In particular it is not immediately clear if the autostart database directory provisions necessary under Linux are required under Solaris (they are not) or how to specify which VMs to automatically start, regardless of platform. Here I will once again attempt to straighten the instructions out, for my purposes - your mileage may vary.

First we must create a configuration file which defines those users (local system user accounts) may autostart virtual machines and after what delay they may be started (to prevent the over-congestion of resources at bootup).

First we need somewhere to store configuration files:
# mkdir /etc/vbox

Even if only one user account is used to run virtual machines on the host it is good to retain the default deny policy outlined in the official documentation's example configuration as restrictive assumptions are often safe assumptions - not that the compromise of any other account and subsequent automatic running of virtual machines is a particularly impactful security concern.

Drop the following in /etc/vbox/autostart.cfg:
# Default policy is to deny starting a VM, the other option is "allow". default_policy = deny # user is allowed to start virtual machines but starting them # will be delayed for 10 seconds user = { allow = true startup_delay = 10 }

Now we need to tell svccfg where to find the configuration file:
# svccfg -s svc:/application/virtualbox/autostart:default setprop config/config=/etc/vbox/autostart.cfg

A ways before covering how to set up autostart the documentation for VBoxManage modifyvm shows us how to specify those VMs which should be automatically started and after how long a delay (for a staggered configuration that avoids over-taxing your host's resources at startup):

8.8.9. Autostarting VMs During Host System Boot

These settings configure the VM autostart feature, which automatically starts the VM at host system boot-up. Note that there are prerequisites that need to be addressed before using this feature. See Section 9.21, “Starting Virtual Machines During System Boot”.

  • --autostart-enabled on|off: Enables and disables VM autostart at host system boot-up, using the specified user name.
  • --autostart-delay : Specifies a delay, in seconds, following host system boot-up, before the VM autostarts.

As the user that owns the virtual machines run:
$ VBoxManage modifyvm vmname --autostart-enabled on $ VBoxManage modifyvm vmname --autostart-delay seconds

This is also a good time to change your VMs' default frontend to headless, for reasons why please see A Word on Oracle VM VirtualBox Start Modes (defaultfrontend):
$ VBoxManage modifyvm vmname --defaultfrontend headless

You may wish to consider Preallocating VirtualBox VM RAM for Greater Stability and Avoiding Overcommitment:
$ VBoxManage setextradata vmname VBoxInternal/RamPreAlloc 1

When the configuration has been completed and you are ready to enable autostart, as root run:
# svcadm enable svc:/application/virtualbox/autostart:default

Automatically Start VirtualBox Web Service on Oracle Solaris 11

Per Oracle VM VirtualBox Administrator's Guide for Release 6.0: 2.19. Starting the Oracle VM VirtualBox Web Service Automatically:

The Oracle VM VirtualBox web service, vboxwebsrv, is used for controlling Oracle VM VirtualBox remotely. It is documented in detail in the Oracle VM VirtualBox Software Development Kit (SDK). See Chapter 4, Oracle VM VirtualBox Programming Interfaces.

You can learn more about how to interact with the VirtualBox Web Service through the SDK Programming Guide (API Reference).

The official documentation (VirtualBox Documentation: Chapter 9. Advanced Topics: 9.18.2. Oracle Solaris: Starting the Web Service With SMF) is somewhat convoluted. Here I will try to straighten it out, for my purposes - your mileage may vary.

First, configure the parameters. They are provided in the documentation as a table of Linux environment variables to be loaded in /etc/defaults/virtualbox; on Solaris we must provide these to svccfg on the command line both lower-cased and prefixed with config/.

Parameter Description Default
USER The user which the web service runs as
HOST The host to bind the web service to localhost
PORT The port to bind the web service to 18083
SSL_KEYFILE Server key and certificate file, in PEM format
SSL_PASSWORDFILE File name for password to server key
SSL_CACERT CA certificate file, in PEM format
SSL_CAPATH CA certificate path
SSL_DHFILE DH file name or DH key length in bits
SSL_RANDFILE File containing seed for random number generator
TIMEOUT Session timeout in seconds, 0 disables timeouts 300
CHECK_INTERVAL Frequency of timeout checks in seconds 5
THREADS Maximum number of worker threads to run in parallel 100
KEEPALIVE Maximum number of requests before a socket will be closed 100
ROTATE Number of log files, 0 disables log rotation 10
LOGSIZE Maximum log file size to trigger rotation, in bytes 1MB
LOGINTERVAL Maximum time interval to trigger log rotation, in seconds 1 day

For example:
svccfg -s svc:/application/virtualbox/webservice:default setprop config/host=localhost svccfg -s svc:/application/virtualbox/webservice:default setprop config/port=18083 svccfg -s svc:/application/virtualbox/webservice:default setprop config/user=user

The documentation provides the example for config/user as root. I could not more strongly advise against running any sort of network-exposed daemon as root and instead encourage you to configure a local user account with sufficient permissions to both administrate your virtual machines and run the web service.

After making any changes via svcconfig while the service is running you must run the following to make the changes live:
svcadm refresh svc:/application/virtualbox/webservice:default

To view the current configuration run:
# svcprop -p config svc:/application/virtualbox/webservice:default config/host astring localhost config/keyfile astring "" config/port integer 18083 config/user astring user

When your configuration is satisfactorily complete run:
svcadm enable svc:/application/virtualbox/webservice:default

To shut down a currently running instance of the service supply disable instead:
svcadm disable svc:/application/virtualbox/webservice:default

SSH Problem: ~v [LogLevel DEBUG]

You're logged in to a remote host via ssh, about to su or whatever and just as you begin typing your super secure, character set conscious password you get:
Password: ~v [LogLevel DEBUG] su: Authentication failed
You have just tripped an SSH escape sequence. From man ssh:

 When a pseudo-terminal has been requested, ssh supports a number of
 functions through the use of an escape character.

 A single tilde character can be sent as ~~ or by following the tilde by a
 character other than those described below.  The escape character must
 always follow a newline to be interpreted as special.  The escape
 character can be changed in configuration files using the EscapeChar
 configuration directive or on the command line by the -e option.

 The supported escapes (assuming the default ‘~’) are:

 ~V      Decrease the verbosity (LogLevel) when errors are being written
         to stderr.

 ~v      Increase the verbosity (LogLevel) when errors are being written
         to stderr.

The solution is simple: either choose a password that doesn't begin with an escape sequence or enter a double tilde: ~~

Install VirtualBox on Solaris 11

Make sure your system is up to date:
# pkg update

While in a production environment you may only run virtual machines via VBoxHeadless, given the functionality the Oracle VM VirtualBox Manager GUI provides (i.e. for emergencies, recoveries, fresh installations etc.) it is advisable to install the GUI desktop if you have not already done so:
# pkg install solaris-desktop

NOTE: You can start virtual machines in headless mode from the GUI by holding down the shift key while pressing the Start button.

Navigate to the VirtualBox Downloads page (https://www.virtualbox.org/wiki/Downloads) and download the Solaris 11 IPS hosts file.

Don't forget to also download the Oracle VM VirtualBox Extension Pack for later!

From the CLI install VirtualBox:
# wget https://download.virtualbox.org/virtualbox/6.1.28/VirtualBox-6.1.28-147628-Solaris.p5p # pkg install -g file:///export/home/user/Downloads/VirtualBox-6.1.28-147628-Solaris.p5p system/virtualbox

Now as root install the extpack:
# VBoxManage extpack install ~user/Downloads/Oracle_VM_VirtualBox_Extension_Pack-6.1.28.vbox-extpack

At the time of writing, the official VirtualBox documentation recommends adding any users that will be using the USB functionality inside VirtualBox to the vboxuser group however this is no longer the group automatically created during installation. Instead add your VirtualBox users to the vboxusers group:
# usermod -G vboxusers username

Solaris 11 Quick-and-Dirty VNC Server

Install the graphical desktop if it is not already available (i.e. fresh installation) then reboot:
# pkg install solaris-desktop ... # shutdown -i6 -g0 -y

Use vncpasswd to set the password used to access VNC desktops or you will be prompted to provide one when launching the server:
$ vncpasswd

Manually start the VNC server for a one-off session as the presently logged in user:
$ /usr/bin/vncserver

...or configure VNC for persistent access:

Enable XDMCP (X Display Manager Control Protocol) to permit logins to GDM (GNOME Display Manager) from remote terminals. Edit /etc/gdm/custom.conf to reflect:
[xdmcp] Enable=true

Restart GDM:
# svcadm restart gdm

Enable the Xvnc inetd service:
# inetadm -e xvnc-inetd

If you try connecting now you may encounter a situation where you are constantly redirected to the beginning of the login process upon successfully logging in which is crazymaking, to say the least. If you encounter this simply reboot the system:
# reboot

Now you may connect from a remote client:
$ vncviewer hostname[:port]

Include :port only if connecting to a running instance of vncserver (a port number will be shown in stdout after launch, sequentially starting at :1) otherwise inetd will launch Xvnc and load the GDM login interface.

It should be noted that your connection will not be encrypted by default. This is an unacceptable risk if you will be using VNC over anything but the same wired layer 2 network (arguably an unnecessary risk even then). This can be remedied on most vncviewer clients with the -via flag. From the TigerVNC for linux man page:

       -via gateway
              Automatically create encrypted TCP tunnel to the gateway machine before connection, connect to the host through that tunnel (TigerVNC-specific). By default,  this  option  invokes
              SSH  local  port  forwarding,  assuming  that SSH client binary can be accessed as /usr/bin/ssh. Note that when using the -via option, the host machine name should be specified as
              known to the gateway machine, e.g.  "localhost" denotes the gateway, not the machine where vncviewer was launched. The environment variable VNC_VIA_CMD can  override  the  default
              tunnel command of /usr/bin/ssh -f -L "$L":"$H":"$R" "$G" sleep 20.  The tunnel command is executed with the environment variables L, H, R, and G taken the values of the local port
              number, the remote host, the port number on the remote host, and the gateway machine respectively.

For example:
$ vncviewer -via ip-address ip-address