There's Something You Should Know about Private Internet Access (PIA) VPN

Roughly a decade ago I signed on with Private Internet Access as my first commercial VPN provider. At the time it was one of - if not the - biggest player in the arena. It enjoyed a universally good reputation both for network capacity and privacy, having emerged from two separate court cases producing - as it claimed and appeared to in fact be collecting - no logging data on its clients. It was also one of the first VPN providers to distribute value-added custom desktop and mobile client software with features one expects from all services today: the ability to easily select from many geographically disparate servers, preventing DNS query leakage, implementing a "kill switch" functionality to ensure application connections don't re-establish over the regular uplink when the VPN connection drops, etc.


Not being something I needed to use very often, my subscription seemed to roll over at the promotional rate I signed up at so I let it ride for a few years until the PayPal account it was attached to dissolved. Fast forward to a few weeks ago, a friend let me use their account to test it out again because I was looking for an easy geofence evasion solution so I could provide a romantic interest with entertainment unfairly blocked in Canada.

No sooner than I mentioned this in one of my chat groups I was admonished that "PIA was bought by a spammer." That's quite the accusation, but given numerous unfortunate buyouts over the years (LavaSoft's AdAware comes to mind) not an unreasonable one. It wasn't hard to find further echos of the insinuation on the web but some thoughtful digging I was able to piece together what I believe are the fair and objective facts regarding the situation. Nowhere are they summed up better, in my opinion, than in this incredibly well-written and patiently researched article by Sven Taylor of Restore Privacy: Strange Ties: Private Internet Access, Kape, and Crossrider. That article from 2019 was very recently followed-up one month ago on September 15, 2021 with Kape Technologies (Formerly Crossrider) Now Owns ExpressVPN, CyberGhost, Private Internet Access, Zenmate, and a Collection of VPN "Review" Websites then, incredibly, again just a couple of weeks ago on October 29, 2021 with Taking a Closer Look at Kape Technologies, Crossrider, and Malware.

To the best of my understanding - and with my apologies to those involved if I in any way am misconstruing the events - the following are the most pressing facts that I feel would be relevant to the reasonable consumer's due diligence in determining if PIA is worthy of one's custom today:

  1. PIA originated largely as it had appeared to: over a decade ago, justifiably gaining popular support, and it quickly grew to become a major player in the commercial VPN industry.
  2. Separately, Crossrider was founded in 2011 and brought to market a novel cross-platform, cross-browser development platform for the major browsers of the era.
  3. Crossrider's SDK facilitated monetization, as much commercial software does. The capability was effectively neutral. The implementation however, being in the hands of the extension developers and not Crossrider, could be abused. And it was - extensively; it provided an effective avenue for revenue to flow to developers employing a wide range of nefarious tactics including content injection and privacy invasion.

    From Kape's statement to RestorePrivacy:

    The Crossrider SDK and development platform was used by tens of thousands of independent developers to create cross-browser extensions, and unfortunately a small number of bad actors misused the platform to develop adware and malware. The team at the time attempted to combat the problem, including as a participant and supporter of the Clean Software Alliance, but ultimately decided to shut down Crossrider altogether in 2016 in the face of rising abuse.


So there we have it. A tragic case of a plucky tech upstart with a bright future that brought a new, effective and vital tool to market only to have it abused by bad actors which undeservedly devastated their reputation by mere association despite laudable efforts to combat their own platform's misuse and when that wasn't enough they nobly sacrificed their flagship product for the greater good at the expense of a bottom line they by all accounts could have sustained for the low price of looking the other way. Actually quite a heroic story when you think about it - and I don't mean to sarcastically disparage it. I have uncovered no evidence that indicates the events transpired any way other than how they are recounted by Kape Technologies, the rebranded, reorganized and refocused reincarnation of Crossrider that emerged from an apparently two year long restructuring initiated in 2016.

You know the old tune, surely: "We're just going to deep-six and lay low for a couple years then paradigm-shift from a company that enables countless faceless, unwashed and entirely unaccountable third parties - and ourselves, quite directly - to profit from data mining, ad injection and whatever the stuff EFF associates' nightmares are made of through thoughtfully and deliberately reshaping our core competencies to fabricate around ourselves a protective, mutagenic cocoon constructed from the modest pile of free-range circus clown bellybutton fluff that agglomerates after tapping out the debris left in our sacred collection of near-empty mason jars in which it is furnished - then adding to this the deposits swept from the conference room corners where they accumulate during our rigorous quarterly satanic team-building orgies then after slumbering long enough to completely digest all the stubborn teeth and hair of our dearly missed spate of interns we shall, unified now as one whole and more perfect being, courageously tear out of our serene confinement and emerge into the light, unafraid and triumphant whereupon we shall dare to boldly share with the world what has become a beautifully transformed, trustworthy security and privacy technology leader, that we might accept the hard-fought and righteously won adoration and adulation of the vile and undeserving plebeian masses. Amen."

...As one does.

If that were all there was to it - and if we were hapless, gullible schmucks - such misgivings, even fetid as these, could conceivably yet fail to pass the casual proponent's smell test and find their substance dismissed and derided summarily. Fear and uncertainty - no doubt spread at the tip of a competitor's sabre. Indeed, a nothingburger propped up by clueless trolls and seasoned with the succulent spray of haters' tears sent careening across forums, blog posts, comment feeds with all the verity and gravitas of the sweet morning's dew rolling off a duck's derrière. And yet, so broad and so deep are these brackish waters in which we find ourselves that, without even stirring, a rigid and perky turd announces its unwelcome presence in our porridge: other reasons to be skeptical of Kape Technologies abound - and despite all best efforts - refuse the indignity of any lesser fraction than that of our complete attention. Certainly less stark than the accusations of outright malware pushing but the story is far from over here.

  • In March 2017 Crossrider purchased CyberGhost VPN in its first step to enter the VPN market and re-imagine itself as a security and privacy focused player
  • In 2018 Crossrider purchased ZenMate VPN
  • After rebranding as Kape Technologies in 2018 Private Internet Access became its largest acquisition yet by customer base and dollar amount at a sizable USD$127M:

    From the Private Internet Access acquisition press release, courtesy of Business Wire November 19, 2019:

    LONDON--(, a consumer security software business, is delighted to announce the transformational acquisition of Private Internet Access (PIA), a leading US-based digital privacy company. This acquisition will significantly increase the company's presence in North America and doubles its existing user base to over 2 million paying customers with a truly global brand.

    This catapults Kape towards becoming the 'go-to' privacy company for consumers, paving the way to dominating the rapidly growing digital privacy space, which is already worth US$24 billion in 2019 and is expected to grow by 50% by 2022. According to the Breach Level Index, in the first half of 2018, more than 25 million records were compromised every day, which equates to 291 records every second. As technology develops, and more and more data is shared online, the need for online protection is increasing exponentially. The acquisition of PIA will see Kape's user base double to over 2 million paying subscribers with almost half of them in the US. The combined group is expected to be profitable and generate over US$120 million in revenues in 2020.

    As part of the transaction, Kape will add a number of encryption-based consumer software solutions to its privacy suite available on mobile, tablet and desktops, including: Plus Ultra, a software that speeds up internet connections; LibreBrowser, a completely private browser; and Private.sh, a private and encrypted search engine. This suite will provide people a truly private digital environment.

    Ido Erlichman, Chief Executive Officer of Kape, said: "This is a game-changing moment for both Kape and PIA, transforming our vision of creating a truly global privacy company into a reality."

    Ted Kim, Chief Executive Officer of LTMI (PIA's holding company), added: "We are excited to join forces with Kape to create a true pioneer in digital privacy with significant scale. This transaction brings us one step closer in realising our vision of a digitally private and secure world for our customers."

    Lumos Partners, LLC acted as the exclusive financial advisor and Baker Botts L.L.P served as legal counsel to PIA/LTMI. Bryan Cave Leighton Paisner acted as legal counsel for Kape.

    About Kape (AIM: Kape)

    Kape is a cybersecurity company focused on helping consumers around the world to have better experience and protection in their digital life. Kape develops and distributes a variety of digital products in the online security space. The Group utilises its proprietary digital distribution technology to optimise its reach and create a superb user experience. Kape offers products which provide online security, privacy and an optimal online experience. Kape's vision is to provide online autonomy for a secure and accessible personal digital life, with a team of over 350 people across seven locations worldwide.


    About PIA

    PIA was established in 2009 and is a security software business, based in Denver, Colorado. Since its inception, PIA has grown to become a leading VPN service provider focused on the consumer market, employing approximately 65, with 35% in an R&D capacity. PIA has over 1 million paying subscribers globally, with 48% of them based in the US.


    For Kape corporate public relations enquiries, contact:
    Vigo Communications
    Tel: +44 (0)20 7390 02347
    [email protected]

  • Just two months ago (from the time of writing) Kape made its biggest acquisition yet: USD$946M for ExpressVPN.

    From the ExpressVPN acquisition press release, courtesy of Reuters September 13, 2021:

    JERUSALEM, Sept 13 (Reuters) - British-Israeli digital security software provider Kape Technologies PLC (KAPE.L) said on Monday it was buying virtual private network (VPN) firm ExpressVPN for $936 million in a deal aimed at creating what it called a "premium consumer privacy and security player."

    Kape said the acquisition expands its customer base to more than 6 million from nearly 3 million and would create a tier one digital privacy and security firm best positioned to capitalise on the expected market growth.

    ExpressVPN, it said, has seen a compound annual growth rate (CAGR) of 35.1% over the past four years amid strong demand for consumer-friendly data privacy and security products.

    Consumers have increasingly turned to VPNs such as ExpressVPN to obscure their identities on the internet.

    "Controlling one's digital presence is at the forefront of every tech consumer's mind now, and Kape is more committed than ever to innovating and delivering the tools internet users need to protect their data and rights," said Ido Erlichman, chief executive of Kape Technologies.

    Dan Pomerantz, co-founder of ExpressVPN, said the firm will have more capital and resources to "accelerate our product development, deliver even more innovation to our users, and protect them from a wider range of threats."

    Kape said ExpressVPN will continue to operate day-to-day as an independent service.


Let us indulge in the least generous suspicions for a moment - after all the "tinfoil hat crowd" is, or one could be forgiven for expecting it to be, a core target demographic of commercial VPNs. Surely as a userbase expands so rapidly the pressure on any company (one might imagine moreso one with a history connected to advertising, user metrics, data mining and so on - regardless of how innocent or incidental that history may be) to leverage the data collection opportunities that arise into a revenue stream increases in stride. I don't think it's unreasonable to wonder why so many different VPN operations?. Certainly there is logic to buying out userbases and it clearly makes sense to perpetuate established brands but in simple economic terms it would surely make sense to consolidate not just the ownership and governance but also operations and resources. Instead Kape seems to go out of its way to reassure customers that each VPN service it buys will continue to be operated independently, with more or less unbroken continuity. Choosing not to cut out overhead, scale up core infrastructure and migrate users even at a trickle pace to a more homogeneous "master platform" could give one the impression that Kape is trying to obfuscate its position to the lay customer or even nurture a misleading sense of choice in an increasingly artificial market. On the other hand, there is merit to maintaining a level of independence between very similar units within a business: multiple discreet systems, if they are all reasonably efficient and generating revenue, bring clear and substantial benefits in terms of resiliency and an organization's ability to carry out experimentation, analysis and R&D. By not making any obvious changes immediately after a fresh acquisition Kape also benefits from not giving the existing userbase additional reasons to reevaluate their relationship; by seeming to do nothing they make it easier for even those customers that harbour reservations about the takeover by such an entity to also do nothing. And doing nothing beats the hell out of cancelling subscriptions.

Conversely, at best, one might interpret these events to be something akin to conventional media consolidation. Kape Technologies PLC is publicly traded on the London Stock Exchange. Throwing almost one billion American dollars at its latest prize it is quite clear the company that started in 2011 with a hot product yet had to shutter for two years when that one-trick tanked is no joke some ten years later.


Perhaps in the same vein as how many biotech megacaps aren't really pharmaceutical companies - they're investment funds whose core competency is just making money by acquiring the right microcaps - maybe today's Kape isn't really a technology company at all. Maybe it's really a simple investment fund that's found a niche in being very good or very lucky at buying the right growth equities. Operating on that assumption would imply security and performance aren't their real forté and it would be logical to focus on the kinds of metrics expected to suffer under the priorities of a volume and margin maximizing vehicle: capacity, reliability, human labour (quality of support), etc.

I will submit that my personal tinkering and benchmarking over the past few weeks has been deeply disappointing but I must qualify my input by admitting my tests have been by no means exhaustive or scientific. Over wired 100mbit/s synchronous links I can often sustain 60-70mbit/s over the default OpenVPN protocol with default settings but typically only when using automatically selected servers. It seems manually choosing a server even a few hundred kilometres away is almost always out of the question for streaming quality throughput much less consistency at even the lowest bitrates. That being said, equally anecdotally, it is interesting to note that negative reviews posted to reddit and the like receive what feels like unusually high (though genuine) response rates from users asserting their satisfaction. Comments regarding interaction with support and billing departments however seem decidedly negative and speak to a wide disappointment with obviously pre-canned responses and script-like interactions.

All of that being fine and dandy it's time to put our tinfoil hats back on because in my opinion the most troubling fact about Kape has nothing to do with difficult-to-substantiate fears of service quality being run into the ground. I couldn't blame you if you found my earlier allusion to a misleading sense of choice in an increasingly artificial market a little spooky and over the top. Are you sitting down?

From These VPN "Review" Websites are Actually Owned by VPNs by Sven Taylor of Restore Privacy May 20, 2021:

In March 2021, news broke that Kape had purchased Webselenese, which is the parent company of vpnMentor and Wizcase. These are two large VPN review websites that collectively get about 6.8 million visitors per month according to Ahrefs data (May 2021).


Now let's examine how the rankings changed after the acquisition.

The table below highlights the rankings on vpnMentor's homepage before and after the site was purchased by Kape. Notice the changes in CyberGhost and Private Internet Access.

Before ownership change

  1. NordVPN
  2. ExpressVPN
  3. Surfshark
  4. CyberGhost
  5. Private Internet Access

After ownership change

  1. ExpressVPN
  2. CyberGhost
  3. Private Internet Access
  4. IPVanish
  5. PrivateVPN

With vpnMentor.com, you can see that NordVPN and Surfshark have been completely removed from the top recommendations. Additionally, CyberGhost and Private Internet Access have gone up in the rankings to the #2 and #3 spots after the ownership changes.
We see similar developments with the before and after changes on Wizcase.com:

Before ownership change

  1. NordVPN
  2. ExpressVPN
  3. Surfshark
  4. CyberGhost
  5. Private Internet Access

After ownership change

  1. ExpressVPN
  2. CyberGhost
  3. Private Internet Access
  4. PrivateVPN
  5. HMA VPN

Just like with vpnMentor, we see that the parent company's brands were raised in the recommendations, while some competing brands were dropped.

So one more time, just so we're clear: they don't make malware. They're just shamelessly, gratuitously deceitful. I believe the technical term for elaborate propaganda like this is psyop.

It gets better. Come meet the team!

Primary shareholder of Kape Technologies Teddy Sagi. [Photo: Himself]

Teddy Sagi is an Israeli billionaire and the main man behind Kape Technologies PLC. He made much of his fortune in online gambling. As a mover, shaker and international man of mystery Teddy has been profiled by such prestigious publications as Forbes and Wikipedia and The Panama Papers where he has been linked to at least sixteen offshore accounts. It should be noted that no wrongdoing has been associated with the accounts. Yet,

The Financial Times reports Teddy served a nine-month prison sentence after being convicted in Israel of bribery and fraud in 1996.

According to The Jerusalem Post, just a little over one month ago Teddy avoided an assasination attempt. He blames "Iranian Terror". An unnamed source blames his Russian mob debts.

CEO of Kape Technologies Ido Erlichman. [Photo: Sharon Dery]

Ido Erlichman is a former undercover counterterrorist commando. He has served as CEO of Kape Technology for five years and by most accounts is responsible for the turnaround of Kape's fortunes.

Koby Menachemi though since departed, co-founded Crossrider back in 2011. Worthy of note for being a Unit 8200 (Israeli SIGINT) alum. Yikes. Koby and Teddy get a little spotlight in this article by Thomas Brewster of Forbes that details the murky ties between adware and Israeli intelligence figures: These Ex-Israeli Surveillance Agents Hijack Your Browser To Profit From Ads.

It should be noted that since military service is compulsory in Israel past affiliation with intelligence outfits is a more common trait than one might otherwise be accustomed to. But a past affiliation with intelligence outfits is a past affiliation with intelligence outfits. Ya dig?

The best for last: Mark Karplès is not affiliated with Kape. He was onboarded a few months before the PIA acquisition by co-founder Andew Lee in the position - of all things - as Chief Technology Officer. Mark rose to fame as the perpetrator of various frauds and mismanagement as CEO of the ill-fated Mt. Gox bitcoin exchange. If he is to be judged by the alleged coding, security and management style during his tenure at Mt. Gox it is questionable what value he could bring to an established, multi-million dollar VPN platform and his inexplicable placement at PIA has been the cause of numerous fits and cancellations among the userbase, unfortunately they have yielded little in the way of answers.


Please don't come away from this with the wrong impression; god knows I enjoy people with a little colour to their personalities. Some minor jail time here and a raging drug addiction or two there is the spice that makes folks interesting. But there is a preponderance of shadiness surrounding Kape and its properties that makes me instinctively wary. If I could get PIA to perform reasonably in my particular setting I might still consider using it for simple geofence hopping, but I would avoid using it for any purpose where my security and/or privacy were important. I would definitely never pay for it.

Now that you have the facts I hope you feel equipped to make an educated decision. If there are any important details I have left out or if I have any of the details wrong please reach out.

Good luck, be safe!

Replicate Installed Packages on New RHEL/Fedora/CentOS/Debian/Ubuntu Depoloyment

You've spend a lot of time getting a particular installation just right, whether it's a bare metal server, virtual machine, desktop workstation or laptop: the role is clearly defined and you'd like to replicate it as quickly as possible either with a fresh base installation or on a totally separate new host. This is particularly salient when upgrading major versions of Qubes Fedora TemplateVMs: generally speaking not a lot of customization goes into these base layers on which AppVMs and DisposableVMs etc. are meant to be built - except for all of the package management that goes into fleshing out a comfortable and usable default environment.

One option is to follow the Qubes documentation for upgrading a Fedora template in place: https://www.qubes-os.org/doc/template/fedora/upgrade/ however I am inclined to take advantage of the template packages as outlined at https://www.qubes-os.org/doc/templates/fedora/ both for the additional management capabilities (e.g. one-line reinstall, version management) and the clean start and distinction between images.

Although some articles recommend obtaining your package list from:
# dnf repoquery --userinstalled acl-0:2.2.53-3.fc30.x86_64 adobe-release-x86_64-0:1.0-1.noarch alternatives-0:1.11-4.fc30.x86_64 attr-0:2.4.48-5.fc30.x86_64 audit-libs-0:3.0-0.15.20191104git1c2f876.fc30.x86_64 basesystem-0:11-7.fc30.noarch ...
There are four issues with this approach:

  • Specific architectures are specified. It's rare that you will be switching architectures but one of the most amazing aspects of Linux is its platform versatility and since things can slip into and out of noarch all the more reason to let the package manager's default settings handle the unforseeable. Agnosticism is next to godliness - but I might be biased :)
  • Base packages, the packages that came preinstalled with the Minimal Server role or the TemplateVM etc. are included. I need a list of only those packages I have intentionally, specifically installed myself or I run the risk of trying to install deprecated, merged, removed, abandoned, unnecessary, etc. packages. This is of particular concern if I am upgrading to a new major release version and/or switching my base installed package set/"server role".
  • Specific versions are specified which is begging for trouble even outside of the context of a global update (ask any Gentoo admin!)
  • This is not a list of the packages that I have chosen to manually install; it is a list of every package installed after the base installation. In other words, it is every package I have chosen to install AND each one of its dependencies. Ask a Gentoo admin how they feel about explicitly installing dependencies!

The thing about dependencies is they like to change and when a dependency has been abandoned by an intentionally installed package yet is itself explicitly installed you are open to the liabilities (dependency hell (eek!), wasted space and update time, tool for intruders...) of keeping that package around and it can be quite unclear months or years after the fact if an abandoned dependency is safe to eliminate or if it provides the crucial library or shim or goo or magic smoke that makes some special, foreign or from-source software go~.

dnf history gets us a lot closer:
# dnf history ID | Command line | Date and time | Action(s) | Altered ------------------------------------------------------------------------------- 24 | install tigervnc | 2020-07-29 21:15 | Install | 4 23 | install k3b | 2020-07-29 13:47 | Install | 28 22 | install deluge | 2020-07-28 23:06 | Install | 47 21 | install gnome-tweak-tool | 2020-07-28 10:14 | Install | 16 20 | install mlocate youtube- | 2020-07-25 03:57 | Install | 2 19 | install elinks links lyn | 2020-07-21 23:05 | Install | 1 18 | install nano psmisc nmap | 2020-07-21 22:58 | Install | 1 17 | install libreoffice | 2020-07-21 22:52 | Install | 99 16 | install vlc | 2020-07-20 23:31 | Install | 32 15 | install ffmpeg | 2020-07-20 23:10 | Install | 31 14 | upgrade --refresh | 2020-07-20 23:05 | Upgrade | 2 EE 13 | install kate gimp | 2020-07-20 01:07 | Install | 93 12 | install chromium | 2020-07-19 06:09 | Install | 13 11 | install screen sshfs nma | 2020-07-19 06:05 | Install | 19 10 | update | 2020-07-19 05:52 | I, O, U | 304 EE 9 | install -y --cacheonly - | 2019-12-25 18:19 | Install | 1 8 | install -y --cacheonly - | 2019-12-25 18:18 | Install | 4 7 | install -y --cacheonly - | 2019-12-25 18:17 | Install | 16 EE 6 | install -y --cacheonly - | 2019-12-25 18:16 | Install | 4 EE 5 | install -y --cacheonly - | 2019-12-25 18:12 | Install | 1 4 | install -y --cacheonly - | 2019-12-25 18:12 | Install | 1 3 | install -y --cacheonly - | 2019-12-25 18:11 | Install | 1 2 | install -y --cacheonly - | 2019-12-25 18:10 | Install | 125 EE 1 | install -y --cacheonly - | 2019-12-25 18:02 | Install | 785 EE
Unfortunately, even if we expand the terminal really far the Command line column is prone to truncating on long package lists. As an aside, I think it's really neat that we can use dnf history info number to zero in and look at, for example, the entries from 1 to 9 in this TemplateVM's history. They show us first the complete base installation at entry 1, then the standard constellation of packages Qubes adds to implement its unparalleled integration and enhancements at slot 2, then every package added and updated before this particular version of the official fedora-30 TemplateVM image was itself rolled into an RPM and deployed. It's always worth taking the time to get to know what you're made of - time permitting!

I must regretfully report that at the time of this writing, having spent hours digging through the dnf sqlite DBs, JSON files, API documentation etc it seems that while it was possible with yumdb to pull a simple list of explicitly user-installed packages free of their dependencies there is simply no facility in current dnf implementations to demarcate the difference between a user-installed package and a package that was installed as a dependency of one. We can at least address two issues: it is easy to get rid of the architecture and version information from our package list but it may be necessary to manually edit the list to remove dead packages, particularly if upgrading by major version number revisions. This is accomplished by using the --queryformat/--qf filter:
# dnf repoquery --userinstalled --qf "%{name}"

The same effect can be achieved through the dnf history userinstalled route via the application of sed:
# dnf history userinstalled | sed 's/-[0-9].*//' | sed '1d' | sed '/.kernel./d'

Direct an itemised list like the preceeding to a text file from stdout using the > operator, copy it to the receiving host and it can be easily edited and batch processed through xargs:

# < package_list.txt xargs dnf -y install

There is one more imperfect option which I have incidentally been using for years, it relies on:

  • Your having used bash to perform most/all of your dnf install operations
  • Gracefully loging out of/closing your shell session(s) afterwards (as opposed to exiting via SIGTERM or segfault or loss of power etc.)
  • Not having exceeded the default .bash_history buffer length (very hard to do in a TemplateVM)

You guessed it...
# cat ~/.bash_history | grep "dnf install" dnf install screen sshfs nmap links lynx nano whois bind-utils dnf install chromium dnf install kate gimp dnf install vlc dnf install ffmpeg dnf install epel-release dnf install nano net-misc psmisc nmap screen dnf install nano psmisc nmap screen dnf install elinks links lynx w3c dnf install elinks links lynx dnf install bind-utils whois sshfs

This method's saving grace is the ease with which it is copied and pasted between remote SSH sessions. I'm the kind of person who uses sudo bash and su so my installations will be found under the root user's .bash_history; if you are more of a sudoer type grep your regular user's log accordingly. Note that a little search-and-replace in a text-editor to add the -y flag to dnf would allow one to copy and paste the entire block into a remote session and let it run non-interactively.

Finally, as the Qubes documentation suggests, you can simply record the changes you make to your TemplateVMs. For instance, I have been compiling a shortlist of so-called favourite programs for RHEL/CentOS so I can quickly assemble an environment I'm used to on the numerous virtual machines I end up configuring month to month regardless of where I am and without having to hunt down an already configured image to crib off of. It's much less frustrating to get the utilities I reflexively expect to be available installed up front instead of as I notice their absence.

As for Debian/Ubuntu and derivatives that ship with dpkg the situation is not perfect - in that we are still going to end up with a list of base packages and dependencies that may need to be edited - but there are facilities built-in to address this simple type of migration.

On the egress system run:
# dpkg --get-selections > /tmp/package_list.txt

Then on the ingress system, after copying over the package_list.txt file, run:
# dpkg --set-selections < /tmp/package_list.txt # apt-get -y update # apt-get dselect-upgrade

Of course it is also possible to grep our .bash_history as with redhat but depending on your system and habits it might be necessary to check for more than one command:
# cat ~/.bash_history | grep "apt install" # cat ~/.bash_history | grep "apt-get install" # cat ~/.bash_history | grep "dpkg install"

Install Windows 10 on Bare Metal (Optionally with Windows 7 or 8 License)

I have previously covered Installing Windows to a Qubes VM (Xen + QEMU HVM) due to the numerous caveats and considerations necessary to produce a comfortable and functional environment. I also have a very particular build order for installing Windows 10 on bare metal which I will outline here in case anyone finds it interesting but mostly to create a space where I can quickly fetch my tools.

In the early days of Windows 10 we were granted a free upgrade from Windows 7 and 8 under the assumption that it was a limited time offer. As many expected this turned out to be mere encouragement for quick adoption; to this day - even after the release of Windows 11 - it is possible to install a fully and legitimately licensed copy of Windows 10 using Windows 7 and 8 keys.

The key to keeping track of USB sticks is: attatch them to crap. Crap has a natural resistance to loss.

This build order begins with a Windows 7 installation for those cases where the product key sticker has been lost or damaged as it may be possible to recover keys stored in hardware or online or an alternative method of licensing and activation can be used which will still result in a fully and legitimately licensed, unmodified Windows 10 installation. If your product key is still intact/visible/recoverable via the presently installed operating system (if functional and accessible) feel free to skip straight ahead to preparing Windows 10 media.

NOTE: I recently wrote an article that covers how to include virtually every USB 3 and Mass Storage Controller driver missing from the official Windows 7 installation media, among other things if you so desire (applicable to all versions of Windows; XP and later). If you are putting together your Windows installation "forever build" I encourage you to check out Add Missing Hardware Support (Drivers) to Windows Install Media (DVDs, USB Sticks).

NOTE: If you perform a lot of Windows 10/11 installations you may want to check out my unattended Windows 10 and 11 installation answer file (AutoUnattend.xml) template. It can cut your install time down by over one half. And it can cut your frustration with answering the same questions about whether or not to enable Microsoft's spyware over and over by... well... all of it!

If you don't have Windows 10 installation media or images handy now or any time later, "in the field" don't sweat it! I've put together a list of sources at this easy-to-remember address:


Go time!

  1. If your machine lacks a product key sticker or the sticker is damaged try booting into the current installation of Windows, if possible. You can recover the product key by opening the Run prompt. Either press Meta (Windows Key) + R or click on Start Menu > Run then enter regedit and hit enter. Navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft then press Control + F to open the Search Dialogue. Enter DefaultProductKey in the Find what: field and check the Keys checkbox. In the right window pane you will see a ProductId entry. Record your product key; it is shown under the Data column.
  2. Obtain a clean, unmodified, Microsoft-distributed installation image for Windows 7 and either burn it to CD or write it to a USB stick (download the latest version of Rufus here, unless creating your media from Windows XP which will only work with versions 2.18 and lower)
  3. At this time you may wish to include an uncompressed copy of the contents of my bootdist.zip archive on your Windows 7 installation media.
  4. Install the edition of Windows 7 appropriate to your license; you will only be able to install the corresponding edition of Windows 10 later (a Windows 10 Home license is only valid for Windows 10 Home; a Windows 7 Professional, Enterprise and Ultimate license is only valid for Windows 10 Profesional or Enterprise)
  5. When asked for the product key enter the key shown on your license sticker; if your sticker is missing or damaged simply click Skip
  6. Generally speaking you will want to delete every existing partition on the primary detected drive and perform a complete, fresh installation (Advanced) to the Unallocated Space (a new partition of appropriate proportions will be automatically created).
  7. Do not enable automatic updates. The other configuration options, including username and password, are inconsequential at this point as long as you will remember how to log in once after installation is complete.
  8. Log in to the newly installed Windows 7 instance. You may need to locate network device drivers on a separate machine and copy them by USB or optical media. Connect to your local network and test the Internet connection by opening Internet Explorer or a command prompt and pinging a known working address, i.e. google.ca.
  9. Check your activation status:

    Click on Start, then Control Panel, then click on System and Security, and finally click on System. Then scroll all the way down to the bottom and you should see a section called Windows activation, which says "Windows is activated" and gives you the Product ID. It also includes the genuine Microsoft software logo.

    If you are instead presented with an option to Activate Windows Now you can either:

    • Enter the product key from your sticker, the previous installation or other records
    • Use an alternative method to license and activate Windows. It is important that you first:
      • Disable Microsoft Security Essentials/Windows Defender through the System and Security Control Panel if it is available and enabled on your system
      • Disable User Account Control (UAC):
        • Enter uac into the Start Menu > Run prompt
        • Click Change User Account Control settings
        • Move the slider down to Never Notify and click OK
      • Reboot the system
      • Note that my bootdist.zip archive has 32 and 64 bit portable versions of and conventional MSI-style installers for 7zip that allow one to extract files that (without the proper precautions) tend to be automatically erased when they show up on newly inserted USB sticks from most popular (optionally password-protected) compression formats.
  10. Once your System Control Panel shows that your system has been activated you must install a new browser; the version of Internet Explorer that ships with Windows 7 is too old to display Microsoft's own website for the Windows 10 Download Assistant. A Firefox installer is provided for convenience in my bootdist.zip archive but it will likely cease functioning with age; you are better off downloading a new installer directly from https://www.mozilla.org.
  11. After installing Firefox (or Chrome, Opera, whatever makes you happy) search for Windows 10 Update Assistant if the direct link https://www.microsoft.com/en-ca/software-download/windows10 no longer works. The Windows 10 Update Assistant is an executable that:
    1. Will download the installation ISO and automatically burn it to a disc or write it to a USB stick which can either be used for a cold-boot fresh installation or a live upgrade
    2. Optionally allow you to select software and settings that will be carried over if you choose to use the tool itself to do a live upgrade
    3. Perform a live upgrade without creating any installation media

    We want to take advantage of functions 3 and 1; first we will live-update the system to ensure that your license carries over if you used an alternative method to license and activate your Windows 7 installation, then we will cold-boot off the installation media so we can completely delete both the new Windows 10 installation and the Windows 7 remnants to ensure the maximum amount of space is reclaimed and you are left with a pristine, from-scratch, unmodified Windows 10 environment with a completely legitimate license.

    If for whatever reason the Microsoft website detects you are not running a version of Windows that can be upgraded with the Assistant (perhaps you could not find compatible network device drivers and are completing this step on a Linux machine) you will be directed instead to a plain ISO image file download page: https://www.microsoft.com/en-ca/software-download/windows10ISO. This is workable; after burning the ISO with your choice of DVD burning software either insert it into a running Windows installation to launch the upgrade assistant and live-migrate or cold-boot off the media to perform a from-scratch installation.

  12. It is important that when you open the Windows 10 Update Assistant application that you right-click and select Run as Administrator or the procedure will typically fail. First, use the tool to create installation media (USB stick preferred). The application will exit after it has finished burning your installation DVD or writing your USB stick. Re-open the application by right-clicking and selecting Run as Administrator again. This time live-update the running system to Windows 10, choosing to retain nothing, simply because it will make this process go faster.
  13. It is important that you unplug your ethernet connection if you have one; if you previously configured a wireless network in Windows 7 the installer will retain its settings after the first reboot - when the menu of available wireless networks is displayed be sure to disconnect from any currently connected network and when prompted click on "I don't have an Internet connection" or you will be sucked into creating or providing Microsoft Account credentials and additional configuration for an installation we are just going to overwrite shortly anyway.
  14. After the installation of Windows 10 has completed boot into your new Windows 10 environment and check your Internet connection status. At this time you may need to locate network device drivers, if you still have drivers you obtained for Windows 7 it is very likely that the same files will work under Windows 10. Generally it is most important to locate the network drivers where none are automatically found as Windows 10 does a terrific job of automatically downloading other system drivers via Windows Update once it has established an Internet connection.
  15. After confirming your Internet connection is working verify that your Windows 10 installation is fully licensed and activated: click the Start button then select Settings > Update & Security then select Activation.
  16. At this time you may wish to include an uncompressed copy of the contents of my bootdist.zip archive on your Windows 10 installation media.
  17. Reboot your system from the newly created Windows 10 installation media. Make sure your ethernet connection is unplugged!
  18. Go through the motions of installing Windows 10 again, deleting every partition and performing a full installation on the Unallocated Space; Performing a live-upgrade to Windows 10 from Windows 7 or 8 will guarantee your license carries over but it will also retain several gigabytes of old system files whether you want them or not and will make modifications to your system by "migrating" what is an otherwise barren Windows 7 installation. This final installation procedure will free up valuable hard drive space and ensure you are starting with an environment straight off the assembly line.
  19. Unless you want the "features" that come with Microsoft Account integration (increased privacy invasion, more convoluted authentication, etc) if you are prompted to join a wireless network or otherwise connect to the Internet at any time during the installation Skip or click on I don't have an Internet Connection - this will allow you to configure a local user account without having to sign up for or provide a Microsoft Account, configure an access PIN and other bothersome strings. If you're not sure, take my advice - you can always change your mind later.
  20. Once more, boot into your completed Windows 10 installation, check your Internet connection, install network drivers if necessary, verify that you are still fully licensed and activated. If not go back and repeat the relevant steps.
  21. Open Settings > Updates and configure your system updates to fit better with your usage pattern. The nice thing about using the Windows 10 Update Assistant is it rolls Windows Updates up to roughly the last month into its image so you should not have to wait long for things like graphics drivers to install (at which point you may right-click on the desktop and select Display Settings to adjust your resolution) and should only have to suffer one or two lengthly update-oriented power cycles; pay careful attention to your Power menu while running through the updates if you need to shut down or restart the machine without applying updates for whatever reason (i.e. being kicked out of the coffee shop, etc) since powering off during a pre-startup/post-shutdown update session can, at worst, render your machine unbootable.
  22. Right click on the Taskbar and open its Settings panel, enable the toggle for Small Icons which will save a lot of space (yet the icons are not much smaller and the margins are needlessly generous...) and most importantly: hide the enormous and wasteful searchbar. To use that functionality simply open the Start Menu and just start typing.
  23. My bootdist.zip archive contains an installer for W10Privacy which contains hundreds of privacy and optimization focused tweaks and automated tasks; I would encourage you to download the latest version directly from their website when putting together your own installation media supplement and take the time to go through each option. You only need to do this on one system then you are able to save the configuration profile and deploy it on every new installation. Please feel free to review the settings I have chosen in my included W10Privacy.ini file. The installer will give you the option of a typical system installation so you can easily go back and tinker with the settings but this is generally a run-once type of utility so you may prefer to take advantage of the portable binary option, at least once you've settled on a configuration profile. In any case remember to launch the application after it has been installed by right-clicking and selecting Run as Administrator; with so many options to load this can be a time consuming oversight on older systems.

I have taken to compiling lists of software I like to install as soon as a fresh installation of whatever OS is complete. The links go directly to the download pages of various essential software projects so I can rapidly download them all in one shot and the installers I download will always be the latest version, direct from their vendor. I end up with a system that is truly complete and doesn't require my running off at some later stage when I'm busy doing real work to obtain what I consider to be basic functionality.

To ensure that my installations are not bloated I organize the software into cascading profiles that layer functionality on top of eachother depending on what role a system is eventually expected to fulfill, starting with a Base profile that covers all the utilities I expect to be on every (in this case, Windows) installation and increasing in specialization from there. While these profiles obviously conform very tightly to what I personally do, like to use and what my expectations and definition of what any given "role" is hopefully others will get some use out of the time I have put into what is otherwise simply a time-saving measure for myself.

Please check out Windows Software @ foxpa.ws/windows-software if this concept appeals to you.

Recover a Solaris Service Stuck in Maintenance Mode

Something updated, a configuration file changed, there was a power outage, aliens descended from the heavens... and now a service that was running yesterday is nowhere to be found. Even the last bastion of an unraveling mind - turning it off and on again - has failed you. Welcome to SMF's maintenance mode.
# svcs | grep gdm maintenance 9:33:00 svc:/application/graphical-login/gdm:default

According to the official documentation (Managing Services (Overview) Introduction to SMF > SMF Concepts > Service States) a service enters the maintenance state when:

The service instance has encountered an error that must be resolved by the administrator.

This is done to prevent a critical error from running out of control or unnecessarily suspending system or service startup procedures when a service fails to start multiple times. The documentation (How to Restore a Service That Is in the Maintenance State) further recommends:

Determine if any process that are dependent to the service have not stopped.

Normally, when a service instance is in a maintenance state, all processes associated with that instance have stopped. However, you should make sure before you proceed. The following command lists all of the processes that are associated with a service instance as well as the PIDs for those processes.
# svcs -p service-name

(Optional) Kill any remaining processes.

Repeat this step for all processes that are displayed by the svcs command:

# pkill -9 process-name

Before we can clear our problem service we need to figure out what went wrong. We can take a detailed look with svcs -xv service-name:
# svcs -xv gdm svc:/application/graphical-login/gdm:default (GNOME Display Manager) State: maintenance since Sat Nov 20 09:33:00 2021 Reason: Method failed repeatedly. See: http://support.oracle.com/msg/SMF-8000-8Q See: man -M /usr/share/man -s 8 gdm See: /var/svc/log/application-graphical-login-gdm:default.log Impact: This service is not running.

It is so delightful to have a pertinent log file suggested in the middle of a potential crisis instead of having to sort through reams of journalctl vomit.

Once the issue has been resolved bring the service out of maintenance mode:
# svcadm clear service-name

Start it from the top:
# svcadm start service-name

And verify all is well:
# svcs -x gdm svc:/application/graphical-login/gdm:default (GNOME Display Manager) State: online since Sat Nov 20 10:06:16 2021 See: gdm(8) See: /var/svc/log/application-graphical-login-gdm:default.log Impact: None.

Update the Default Global Time Zone on Solaris 11

There are two ideologically correct methods to change the default system time zone on Solaris 11:

Using nsladm:

Install the nls-administration package if it is not already available:
# pkg install nls-administration

List the available time zones:
# nlsadm list-timezone

Finaly, set the correct time zone:
# nlsadm set-timezone timezone # nsladm set-timezone Canada/Eastern

Updating the timezone:default SMF property timezone/localtime:

# svccfg -s timezone:default setprop timezone/localtime = astring: Canada/Eastern # svcadm refresh timezone:default