Shell Scripting Variable Cheat Sheet

There are myriad implementations of the common UNIX shell. Often these shells are syntactically compatible and the following rules tend to apply when dealing with variables:

  • Names must be alphanumeric and may not start with a number.
  • Names are case sensitive and conventionally upper-case.
  • There may be no whitespace between the variable name, assignment operator and value when a variable is set.
  • Variable names begin with the $ character when referred to in read operations; this character is omitted when they are set.
  • Variables are scoped within the script they are defined in, unless they are exported.
  • Variables may be flagged readonly upon initialization.
  • Command substitution assigns the output of commands contained in $() at the time the variable is set.
  • Variables may be unset via the unset command.

MYVAR=1 COMPLEXVAR="A string containing spaces." readonly ROVAR="This value can not be changed later." CSVAR=$( ls /var | wc -l )
Special variables dynamically made available at runtime:

  • $0 - The name of the script.
  • $1 - $9 - The first 9 arguments to the script.
  • $# - How many arguments were passed to the script.
  • [email protected] - All the arguments supplied to the script.
  • $? - The exit status of the most recently run process.
  • $$ - The process ID of the current script.

You can see additional variables currently available to you by issuing env or export without arguments at the command line or from inside your script.
$ env SHELL=/bin/bash LANGUAGE=en_US.UTF-8 NO_AT_BRIDGE=1 PWD=/home LOGNAME=username XDG_SESSION_TYPE=tty HOME=/home/username LANG=en_CA.UTF-8 XDG_SESSION_CLASS=user TERM=xterm-256color USER=username SHLVL=1 XDG_SESSION_ID=c16 XDG_RUNTIME_DIR=/run/user/1000 LC_ALL=en_US.UTF-8 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus MAIL=/var/mail/username SSH_TTY=/dev/pts/4 TEXTDOMAIN=Linux-PAM _=/usr/bin/env

Special variables available to BASH scripts:

  • $EPOCHSECONDS - Each time this parameter is referenced it expands to the number of seconds since the Unix Epoch. (eg: 1577354438)
  • $EPOCHREALTIME - Same as $EPOCHSECONDS but includes microseconds. (eg: 1577354438.788521)
  • $SECONDS - Expands to the number of seconds since the shell was started. Assignment to this variable resets the count to the value assigned, and the expanded value becomes the value assigned plus the number of seconds since the assignment.
  • $RANDOM - Each time this parameter is referenced, a random integer between 0 and 32767 is generated. Assigning a value to this variable seeds the random number generator.
  • $LINENO - Returns the current line number in the Bash script.
  • $MACHTYPE - A string that fully describes the system type on which Bash is executing, in the standard GNU cpu-company-system format. (eg: arm-unknown-linux-gnueabihf)
  • $OSTYPE - A string describing the operating system Bash is running on. (eg: linux-gnueabihf)

Additional BASH-specific variables can be found at the BASH Reference Manual.

You can find a more in-depth look at variable syntax and example use cases at https://ryanstutorials.net/bash-scripting-tutorial/bash-variables.php.

Reset Forgotten Raspberry Pi Password

Changing a lost/forgotten Raspbian/Ubuntu MATE/etc password on a Raspberry Pi is a slightly different procedure than you are likely used to. There is no GRUB bootloader menu to intercept so you can interactively edit the kernel command line. Booting off a live cd/usb stick to chroot on another machine isn't going to work unless that machine uses the same ARM architecture and your boot environment uses a kernel that is compatible with your Pi's binaries.

  • Remove the SD card from your Raspberry Pi. Mount it on another machine.
  • Edit the cmdline.txt file and add init=/bin/sh to the end of the kernel command line.
  • Reinsert the SD card into the Pi and boot.
  • You will land at a root prompt (#). Issue the following command to remount the root filesystem in read/write mode: mount -o remount,rw /
  • If you are running Raspbian your username is likely "pi" - if you are running Ubuntu MATE or another flavour it will be whatever you chose at the time of installation. Issue the following to update the password: passwd <user>.
  • Launch init: exec /sbin/init. You should see the rest of your normal boot process complete and then be able to log in with your new password as usual.
  • With root privileges (using sudo), edit /boot/cmdline.txt and remove the string init=/bin/sh that you added on the other machine.
  • Now reboot your Pi by issuing: reboot with root privileges; your regular boot process should run and you will be able to log in with the new password again.

Install killall on CentOS/RHEL

It seems killall has been removed from the default toolset on Red Hat distros (i.e. CentOS and RHEL). The logic is sound: as you probably know killall does potentially dangerous different things on other UNIX variants; chiefly it will kill every single process on Solaris/SunOS with wonton disregard.

The preferred weapon of choice is now pkill, as in:
# pkill victim
...where victim is the name of the process(es) you wish to terminate.

But if you're like me, you don't use Solaris and for nigh on 20 years you've been conditioned to reach for killall when you need to quickly hunt down and slaughter a runaway process on your linux hosts.

Creatures of habit die hard and in old age ought be afforded some luxuries, thus:
# yum install psmisc
# killall thepetulantbastards

Install Red5 RTMP Server on CentOS/RHEL

Red5 is an open source RTMP server, the streaming protocol of choice for Adobe Flash. That makes it a free alternative to the commercially licensed Flash Media Server.

There are no officially supported RPMs for CentOS/RHEL, it's necessary to perform the installation manually. Red5 is a Java application and, as such, it is necessary to install the JDK. The current version of Red5 (1.1.1) will not work with Java versions later than 8. Agree to the license and download the JDK at https://www.oracle.com/technetwork/java/javase/downloads/index.html. Unlike recent versions of java, you will be forced to go through the exceedingly obnoxious step of creating an otherwise useless oracle account if you have not already done so (or if you have done this fifty times but can't remember any of them, as the case may be). It is important that you provide a working e-mail address because you will have to click on a validation link before you can log in. I tend to fill the rest of the fields out with dirty words because I am somewhat juvenile and deeply resentful of such things.

# rpm -iv jdk-8u212-linux-x64.rpm warning: jdk-8u212-linux-x64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY Preparing packages... jdk1.8-2000:1.8.0_212-fcs.x86_64 Unpacking JAR files... tools.jar... plugin.jar... javaws.jar... deploy.jar... rt.jar... jsse.jar... charsets.jar... localedata.jar...

Obtain the latest Red5 tarball from https://github.com/Red5/red5-server/releases and unpack it to /opt/:
# wget https://github.com/Red5/red5-server/releases/download/v1.1.1/red5-server-1.1.1.tar.gz # tar zxf red5-server-1.1.1.tar.gz # mv red5-server/ /opt/red5

The stock Red5 init script requires jsvc:
# yum install jsvc

Copy the Red5 SysV init script to /etc/init.d/ and the systemd .service file to /etc/systemd/system/ then enable the service to start on boot:
# cp red5 /etc/init.d/ # cp red5.service /etc/systemd/system/ # systemctl enable red5.service

Edit /etc/init.d/red5 and add the RED5_HOME variable before the FILE_PATH variable:
# The path to the folder containing daemon jar RED5_HOME="/opt/red5" FILE_PATH="/usr/local/$NAME" # If red5 home is set, use it if [ ! -z "$RED5_HOME" ]; then echo "Setting file path using RED5_HOME" FILE_PATH=$RED5_HOME; fi export RED5_HOME=$FILE_PATH; echo "Path $FILE_PATH";

If you have installed multiple versions of Java you can set the default with the alternatives command:
# alternatives --config java There is 1 program that provides 'java'. Selection Command ----------------------------------------------- *+ 1 /usr/java/jdk1.8.0_212-amd64/jre/bin/java Enter to keep the current selection[+], or type selection number:

Update the JAVA_HOME variable in /etc/init.d/red5; if your default version is anything other than 8 link directly to it instead of using the default symlink:
# The path to the folder containing the java runtime #JAVA_HOME="/usr/lib/jvm/default-java" JAVA_HOME="/usr/java/default" JAVA_HOME="/usr/java/jdk1.8.0_212-amd64"

The init script with Red5 1.1.1 is out of date; it references commons-daemon-1.0.15.jar while the package ships with commons-daemon-1.1.0.jar. Update the script to reflect:
#CLASS_PATH="$FILE_PATH/commons-daemon-1.0.15.jar:$FILE_PATH/red5-service.jar:$FILE_PATH/conf" CLASS_PATH="$FILE_PATH/commons-daemon-1.1.0.jar:$FILE_PATH/red5-service.jar:$FILE_PATH/conf"

...or you will encounter the following error in /opt/red5/log/red5-error.log:
Cannot find daemon loader org/apache/commons/daemon/support/DaemonLoader Service exit with a return value of 1

If you will be using Red5 in a production environment it's probably a good idea to allocate more than the default amount of ram to the JVM. It's also a very good idea to isolate Red5 from other services and resources, ideally in a virtual machine if not a fully dedicated server. If this is your use case, it is safe to allocate up to your total available memory minus one to two GB by editing /etc/init.d/red5 and adding the following arguments to the JVM_OPTS variable (you may need to scroll right):
# The amount of ram to allocate to the JVM (system total minus 1-2GB on a single-purpose system) RED5_RAM="-Xms2g -Xmx2g" ... JAVA_OPTS="$LOGGING_OPTS $SECURITY_OPTS $JAVA_OPTS $JVM_OPTS $RED5_RAM $TOMCAT_OPTS"

With the configuration we have set up so far we will be running Red5 as root. I know I don't have to tell you why exposing a complicated service running as root to the wild is a terrible idea. Whomever wrote the stock init script chose to utilize jsvc; among the best reasons to use a helper like this is to drop the service into a non-privileged user account. In fact, the very second line on jsvc's website explaining its purpose is:

Jsvc allows the application (e.g. Tomcat) to perform some privileged operations as root (e.g. bind to a port < 1024), and then switch identity to a non-privileged user.

Red5 doesn't ship with any documentation that indicates it will be running as root or suggesting that you should do otherwise. The init script isn't set up to be configured with an alternative account. Even the more current and expansive commercial "pro" edition of Red5's documentation has you configure the server as root.

Note the single jsvc instance.

From jsvc's website:

Jsvc uses 3 processes: a launcher process, a controller process and a controlled process. The controlled process is also the main java thread, if the JVM crashes the controller will restart it in the next minute. Jsvc is a daemon process so it should be started as root and the -user parameter allows to downgrade to an unprivilegded user.

We need to fix this.

First, create the non-privileged system account red5:
# useradd -d /opt/red5 -s /sbin/nologin -r red5

Do not give the new account a password: a non-privileged account should never be logged into.

Now edit /etc/init.d/red5 and instantiate the RED5_USER variable then place it in the command line thus:
# The user to run java under RED5_USER="red5" ... jsvc_exec() { cd $FILE_PATH $EXEC -home $JAVA_HOME -cp $CLASS_PATH -cwd $RED5_HOME $JAVA_OPTS -outfile $LOG_OUT -errfile $LOG_ERR -pidfile $PID -user $RED5_USER $1 $CLASS $ARGS }

Now you may start the service:
# service red5 start Setting file path using RED5_HOME Path /opt/red5 Starting the Red5 service... The Red5 service has started.

Now check your processes; you will note two instances of jsvc.exec: the controller process running as root and the larger (JVM) process running as red5:
# ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 28121 0.0 0.0 10736 356 ? Ss 22:35 0:00 jsvc.exec -home /usr/java/default -cp /opt/red5/commons-daemon-1.1.0.jar:/opt/red5/red5-service.jar:/opt/red5/conf - red5 28122 19.3 4.9 4288264 193616 ? Sl 22:35 0:10 jsvc.exec -home /usr/java/default -cp /opt/red5/commons-daemon-1.1.0.jar:/opt/red5/red5-service.jar:/opt/red5/conf -

Now we can see that Red5 is listening on its designated ports:
# lsof -i COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME jsvc 28122 red5 101u IPv6 713777 0t0 TCP *:macromedia-fcs (LISTEN) jsvc 28122 red5 108u IPv6 716369 0t0 TCP *:onscreen (LISTEN) jsvc 28122 red5 110u IPv6 716373 0t0 TCP *:distinct (LISTEN) ...or... # lsof -i -P COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME jsvc 28122 red5 101u IPv6 713777 0t0 TCP *:1935 (LISTEN) jsvc 28122 red5 108u IPv6 716369 0t0 TCP *:5080 (LISTEN) jsvc 28122 red5 110u IPv6 716373 0t0 TCP *:9999 (LISTEN)

As you may note, /etc/services does not accurately reflect our use case for some ports. You may update it at your discretion.

  • 1935 - RTMP
  • 5080 - HTTP tomcat web daemon for demo and custom FLEX webapps, management/statistics interface, REST API (pro version).
  • 9999 - RTMPS

If you are running firewalld or the default iptables policy it will be necessary to open the relevant ports, as we covered in Open a Port on CentOS/RHEL. Think carefully about whether you want to expose port 5080 and, if you do, where to.
# firewall-cmd --permanent --add-port=1935/tcp success # firewall-cmd --reload success

When I wrote Red5 Streaming Media Server Init Script for Gentoo 8 years ago I set Red5 up to use a non-privileged account as a matter of course because Red5 didn't ship with an OpenRC init script or installation instructions for Gentoo; I simply adhered to best practices. In writing this it dawns on me that there are very likely now thousands of Red5 and Red5 Pro installations in the wild exposing a complex Java application - complete with tomcat instance and demo webapps - while running as root.

For your convenience, an updated version of the init script for version 1.1.1:
#!/bin/sh # /etc/init.d/red5 ### BEGIN INIT INFO # Provides: Red5 # Required-Start: $remote_fs $syslog $network # Required-Stop: $remote_fs $syslog # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Starts the red5 service # Description: This file is used to start the daemon and should be placed in /etc/init.d # chkconfig: 2345 85 85 # processname: red5 ### END INIT INFO # Author: Sheldon Neilson <sheldon[AT]neilson.co.za> # Url: www.neilson.co.za # Date: 25/04/2013 # Modified by Paul Gregoire <mondain[AT]gmail.com> on 18 May 2016 # Modified by http://foxpa.ws on July 17 2019 for Red5 v.1.1.1 NAME="red5" DESC="Red5 service" # The path to Jsvc EXEC="/usr/bin/jsvc" # The user to run java under RED5_USER="red5" # The amount of ram to allocate to the JVM (system total minus 1-2GB on a single-purpose system) # RED5_RAM="-Xms2g -Xmx2g" RED5_RAM="-Xms512m -Xmx512m" # The path to the folder containing daemon jar RED5_HOME="/opt/red5" FILE_PATH="/usr/local/$NAME" # If red5 home is set, use it if [ ! -z "$RED5_HOME" ]; then echo "Setting file path using RED5_HOME" FILE_PATH=$RED5_HOME; fi export RED5_HOME=$FILE_PATH; echo "Path $FILE_PATH"; # The path to the folder containing the java runtime #JAVA_HOME="/usr/lib/jvm/default-java" JAVA_HOME="/usr/java/default" # Our classpath including our jar file and the Apache Commons Daemon library #CLASS_PATH="$FILE_PATH/commons-daemon-1.0.15.jar:$FILE_PATH/red5-service.jar:$FILE_PATH/conf" CLASS_PATH="$FILE_PATH/commons-daemon-1.1.0.jar:$FILE_PATH/red5-service.jar:$FILE_PATH/conf" # The fully qualified name of the class to execute CLASS="org.red5.daemon.EngineLauncher" # Any command line arguments to be passed to the our Java Daemon implementations init() method ARGS="9999" # The file that will contain our process identification number (pid) for other scripts/programs that need to access it. PID="/tmp/$NAME.pid" # System.out writes to this file... LOG_OUT="$FILE_PATH/log/$NAME-service.log" # System.err writes to this file... LOG_ERR="$FILE_PATH/log/$NAME-error.log" # JAVA options # You can set JVM additional options here if you want if [ -z "$JVM_OPTS" ]; then JVM_OPTS="-Xverify:none -XX:+TieredCompilation -XX:+UseBiasedLocking -XX:InitialCodeCacheSize=8m -XX:ReservedCodeCacheSize=32m -Dorg.terracotta.quartz.skipUpdateCheck=true" fi # Set up logging options LOGGING_OPTS="-Dlogback.ContextSelector=org.red5.logging.LoggingContextSelector -Dcatalina.useNaming=true" # Set up security options SECURITY_OPTS="-Djava.security.debug=failure" # Set up tomcat options TOMCAT_OPTS="-Dcatalina.home=$RED5_HOME" JAVA_OPTS="$LOGGING_OPTS $SECURITY_OPTS $JAVA_OPTS $JVM_OPTS $RED5_RAM $TOMCAT_OPTS" jsvc_exec() { cd $FILE_PATH $EXEC -home $JAVA_HOME -cp $CLASS_PATH -cwd $RED5_HOME $JAVA_OPTS -outfile $LOG_OUT -errfile $LOG_ERR -pidfile $PID -user $RED5_USER $1 $CLASS $ARGS } case "$1" in start) echo "Starting the $DESC..." # Start the service jsvc_exec echo "The $DESC has started." ;; stop) echo "Stopping the $DESC..." # Stop the service jsvc_exec "-stop" echo "The $DESC has stopped." ;; restart) if [ -f "$PID" ]; then echo "Restarting the $DESC..." # Stop the service jsvc_exec "-stop" # Start the service jsvc_exec echo "The $DESC has restarted." else echo "Daemon not running, no action taken" exit 1 fi ;; *) echo "Usage: /etc/init.d/$NAME {start|stop|restart}" >&2 exit 3 ;; esac

Open a Port on CentOS/RHEL

Along with numerous other changes, CentOS 7/RHEL 7 introduces firewalld:

Firewalld provides a dynamically managed firewall with support for network/firewall zones that define the trust level of network connections or interfaces. It has support for IPv4, IPv6 firewall settings, ethernet bridges and IP sets. There is a separation of runtime and permanent configuration options. It also provides an interface for services or applications to add firewall rules directly.

God knows learning iptables wasn't hard enough, here comes a big fat new layer to wrangle with the ostensible purpose of making things simpler while inherently adding a whole bunch of complexity and obscure inner workings. What could be more Red Hat? If your first inclination is to disable it and revert to pure iptables, power to you:
# systemctl stop firewalld # systemctl disable firewalld # systemctl mask --now firewalld # yum install iptables-services # systemctl start iptables # systemctl start iptables6 # systemctl enable iptables # systemctl enable iptables6
You are now free to use the conventional iptables configuration, i.e. issuing then dropping raw iptables commands to be run on boot into /etc/sysconfig/iptables
# iptables -I INPUT -p tcp -m tcp --dport 80 -j ACCEPT # iptables-save > /etc/sysconfig/iptables # service iptables restart

However if you choose to embrace the new paradigm it's simple enough to work with if you're willing to etch out some new space in your memory.

Open a port:
# firewall-cmd --permanent --add-port=80/tcp

Open a port range:
# firewall-cmd --permanent --add-port=80-81/tcp

After you have made the configuration change it is necessary to update the state of the firewall:
# firewall-cmd --reload

An interesting part of this new system for someone who is otherwise resentful may be the ability to open a service by name:
# firewall-cmd --permanent --add-service=http
In some cases this will execute additional operations, for example automatically loading relevant netfilter modules. Or, more interestingly, executing user-defined instructions...