Make sure your system is up to date:
# pkg update

While in a production environment you may only run virtual machines via VBoxHeadless, given the functionality the Oracle VM VirtualBox Manager GUI provides (i.e. for emergencies, recoveries, fresh installations etc.) it is advisable to install the GUI desktop if you have not already done so:
# pkg install solaris-desktop

NOTE: You can start virtual machines in headless mode from the GUI by holding down the shift key while pressing the Start button.

Navigate to the VirtualBox Downloads page (https://www.virtualbox.org/wiki/Downloads) and download the Solaris 11 IPS hosts file.

Don't forget to also download the Oracle VM VirtualBox Extension Pack for later!

From the CLI install VirtualBox:
# wget https://download.virtualbox.org/virtualbox/6.1.28/VirtualBox-6.1.28-147628-Solaris.p5p # pkg install -g file:///export/home/user/Downloads/VirtualBox-6.1.28-147628-Solaris.p5p system/virtualbox

Now as root install the extpack:
# VBoxManage extpack install ~user/Downloads/Oracle_VM_VirtualBox_Extension_Pack-6.1.28.vbox-extpack

At the time of writing, the official VirtualBox documentation recommends adding any users that will be using the USB functionality inside VirtualBox to the vboxuser group however this is no longer the group automatically created during installation. Instead add your VirtualBox users to the vboxusers group:
# usermod -G vboxusers username