You've spend a lot of time getting a particular installation just right, whether it's a bare metal server, virtual machine, desktop workstation or laptop: the role is clearly defined and you'd like to replicate it as quickly as possible either with a fresh base installation or on a totally separate new host. This is particularly salient when upgrading major versions of Qubes Fedora TemplateVMs: generally speaking not a lot of customization goes into these base layers on which AppVMs and DisposableVMs etc. are meant to be built - except for all of the package management that goes into fleshing out a comfortable and usable default environment.
One option is to follow the Qubes documentation for upgrading a Fedora template in place: https://www.qubes-os.org/doc/template/fedora/upgrade/ however I am inclined to take advantage of the template packages as outlined at https://www.qubes-os.org/doc/templates/fedora/ both for the additional management capabilities (e.g. one-line reinstall, version management) and the clean start and distinction between images.
Although some articles recommend obtaining your package list from:
# dnf repoquery --userinstalled
acl-0:2.2.53-3.fc30.x86_64
adobe-release-x86_64-0:1.0-1.noarch
alternatives-0:1.11-4.fc30.x86_64
attr-0:2.4.48-5.fc30.x86_64
audit-libs-0:3.0-0.15.20191104git1c2f876.fc30.x86_64
basesystem-0:11-7.fc30.noarch
...
There are four issues with this approach:
- Specific architectures are specified. It's rare that you will be switching architectures but one of the most amazing aspects of Linux is its platform versatility and since things can slip into and out of noarch all the more reason to let the package manager's default settings handle the unforseeable. Agnosticism is next to godliness - but I might be biased :)
- Base packages, the packages that came preinstalled with the Minimal Server role or the TemplateVM etc. are included. I need a list of only those packages I have intentionally, specifically installed myself or I run the risk of trying to install deprecated, merged, removed, abandoned, unnecessary, etc. packages. This is of particular concern if I am upgrading to a new major release version and/or switching my base installed package set/"server role".
- Specific versions are specified which is begging for trouble even outside of the context of a global update (ask any Gentoo admin!)
- This is not a list of the packages that I have chosen to manually install; it is a list of every package installed after the base installation. In other words, it is every package I have chosen to install AND each one of its dependencies. Ask a Gentoo admin how they feel about explicitly installing dependencies!
The thing about dependencies is they like to change and when a dependency has been abandoned by an intentionally installed package yet is itself explicitly installed you are open to the liabilities (dependency hell (eek!), wasted space and update time, tool for intruders...) of keeping that package around and it can be quite unclear months or years after the fact if an abandoned dependency is safe to eliminate or if it provides the crucial library or shim or goo or magic smoke that makes some special, foreign or from-source software go~.
dnf history gets us a lot closer:
# dnf history
ID | Command line | Date and time | Action(s) | Altered
-------------------------------------------------------------------------------
24 | install tigervnc | 2020-07-29 21:15 | Install | 4
23 | install k3b | 2020-07-29 13:47 | Install | 28
22 | install deluge | 2020-07-28 23:06 | Install | 47
21 | install gnome-tweak-tool | 2020-07-28 10:14 | Install | 16
20 | install mlocate youtube- | 2020-07-25 03:57 | Install | 2
19 | install elinks links lyn | 2020-07-21 23:05 | Install | 1
18 | install nano psmisc nmap | 2020-07-21 22:58 | Install | 1
17 | install libreoffice | 2020-07-21 22:52 | Install | 99
16 | install vlc | 2020-07-20 23:31 | Install | 32
15 | install ffmpeg | 2020-07-20 23:10 | Install | 31
14 | upgrade --refresh | 2020-07-20 23:05 | Upgrade | 2 EE
13 | install kate gimp | 2020-07-20 01:07 | Install | 93
12 | install chromium | 2020-07-19 06:09 | Install | 13
11 | install screen sshfs nma | 2020-07-19 06:05 | Install | 19
10 | update | 2020-07-19 05:52 | I, O, U | 304 EE
9 | install -y --cacheonly - | 2019-12-25 18:19 | Install | 1
8 | install -y --cacheonly - | 2019-12-25 18:18 | Install | 4
7 | install -y --cacheonly - | 2019-12-25 18:17 | Install | 16 EE
6 | install -y --cacheonly - | 2019-12-25 18:16 | Install | 4 EE
5 | install -y --cacheonly - | 2019-12-25 18:12 | Install | 1
4 | install -y --cacheonly - | 2019-12-25 18:12 | Install | 1
3 | install -y --cacheonly - | 2019-12-25 18:11 | Install | 1
2 | install -y --cacheonly - | 2019-12-25 18:10 | Install | 125 EE
1 | install -y --cacheonly - | 2019-12-25 18:02 | Install | 785 EE
Unfortunately, even if we expand the terminal really far the Command line column is prone to truncating on long package lists. As an aside, I think it's really neat that we can use dnf history info number to zero in and look at, for example, the entries from 1 to 9 in this TemplateVM's history. They show us first the complete base installation at entry 1, then the standard constellation of packages Qubes adds to implement its unparalleled integration and enhancements at slot 2, then every package added and updated before this particular version of the official fedora-30 TemplateVM image was itself rolled into an RPM and deployed. It's always worth taking the time to get to know what you're made of - time permitting!
I must regretfully report that at the time of this writing, having spent hours digging through the dnf sqlite DBs, JSON files, API documentation etc it seems that while it was possible with yumdb to pull a simple list of explicitly user-installed packages free of their dependencies there is simply no facility in current dnf implementations to demarcate the difference between a user-installed package and a package that was installed as a dependency of one. We can at least address two issues: it is easy to get rid of the architecture and version information from our package list but it may be necessary to manually edit the list to remove dead packages, particularly if upgrading by major version number revisions. This is accomplished by using the --queryformat/--qf filter:
# dnf repoquery --userinstalled --qf "%{name}"
The same effect can be achieved through the dnf history userinstalled route via the application of sed:
# dnf history userinstalled | sed 's/-[0-9].*//' | sed '1d' | sed '/.kernel./d'
Direct an itemised list like the preceeding to a text file from stdout using the > operator, copy it to the receiving host and it can be easily edited and batch processed through xargs:
# < package_list.txt xargs dnf -y install
There is one more imperfect option which I have incidentally been using for years, it relies on:
- Your having used bash to perform most/all of your dnf install operations
- Gracefully loging out of/closing your shell session(s) afterwards (as opposed to exiting via SIGTERM or segfault or loss of power etc.)
- Not having exceeded the default .bash_history buffer length (very hard to do in a TemplateVM)
You guessed it...
# cat ~/.bash_history | grep "dnf install"
dnf install screen sshfs nmap links lynx nano whois bind-utils
dnf install chromium
dnf install kate gimp
dnf install vlc
dnf install ffmpeg
dnf install epel-release
dnf install nano net-misc psmisc nmap screen
dnf install nano psmisc nmap screen
dnf install elinks links lynx w3c
dnf install elinks links lynx
dnf install bind-utils whois sshfs
This method's saving grace is the ease with which it is copied and pasted between remote SSH sessions. I'm the kind of person who uses sudo bash and su so my installations will be found under the root user's .bash_history; if you are more of a sudoer type grep your regular user's log accordingly. Note that a little search-and-replace in a text-editor to add the -y flag to dnf would allow one to copy and paste the entire block into a remote session and let it run non-interactively.
Finally, as the Qubes documentation suggests, you can simply record the changes you make to your TemplateVMs. For instance, I have been compiling a shortlist of so-called favourite programs for RHEL/CentOS so I can quickly assemble an environment I'm used to on the numerous virtual machines I end up configuring month to month regardless of where I am and without having to hunt down an already configured image to crib off of. It's much less frustrating to get the utilities I reflexively expect to be available installed up front instead of as I notice their absence.
As for Debian/Ubuntu and derivatives that ship with dpkg the situation is not perfect - in that we are still going to end up with a list of base packages and dependencies that may need to be edited - but there are facilities built-in to address this simple type of migration.
On the egress system run:
# dpkg --get-selections > /tmp/package_list.txt
Then on the ingress system, after copying over the package_list.txt file, run:
# dpkg --set-selections < /tmp/package_list.txt
# apt-get -y update
# apt-get dselect-upgrade
Of course it is also possible to grep our .bash_history as with redhat but depending on your system and habits it might be necessary to check for more than one command:
# cat ~/.bash_history | grep "apt install"
# cat ~/.bash_history | grep "apt-get install"
# cat ~/.bash_history | grep "dpkg install"
![[attachment-YXjEH7]](/files/2021/12/21/winstall.jpg)