Installing Webmin and Virtualmin from packages

Please note: this guide has not yet had a complete run-through from scratch.  If you hit problems, please e-mail alastair@plug.org.au

Do you want to install Webmin and Virtualmin from packages, i.e. not using the official install script?  The process is finicky, but can be done, as shown here.  One reason you might wish to to this is that you might not want the default "kitchen sink" installation and all the dependencies it brings in.  (Or you might not like how the official install script modifies /etc/apt/sources.list rather than adding a file called /etc/apt/sources.list.d/webmin.list .  Or you might be concerned about the fact that the install script is served from a non-HTTPS URL and is therefore open to modification in transit or other security breaches.  Et cetera.)

The end result of this process is a server that uses PHP, fcgid and Apache.  Unlike a standard Virtualmin installation, it is a "bare bones" server that can be used for web hosting only.  It assumes that other features like e-mail hosting and DNS are provided by the cloud or separate servers.
This guide assumes that you are using Debian or Ubuntu.  It might be possible to apply some of the instructions below on other systems, but many won't work unless substitute commands are used. Note: older versions of Debian (wheezy and earlier) and Ubuntu (precise and earlier) don't have the apt command; you can use aptitude instead.  (FYI, aptitude is installable in recent OS releases if you prefer it.)

Install Linux

This is out of scope for this guide.

Linux Preparation

As an alternative to the steps in the sub-sections below, you can install the required packages manually, but there are a lot of them to remember.  http://al.id.au/svn/tools/debian/apache-fcgid/  has a script called setup-fcgid, but note that it currently works only on older releases of Ubuntu (pre Xenial) and Debian (pre Stretch).

Setup LAMP and Postfix

1. Run sudo tasksel
2. Move text cursor to "LAMP server" and press Enter
3. Move text cursor to "Mail server" and press Enter
• Virtualmin won't work without Postfix etc.
• Note that Webmin's Exim support is listed as experimental, so on Debian (where the default MTA is Exim) you might want to install Postfix instead if you have no preference
4. Press Tab to go to the <Ok> button and press Enter
5. When prompted to set a MySQL server password, do so unless you will be uninstalling MySQL as per the next sub-section; in this case, you will be prompted multiple times, so press Enter each time
6. When the Postfix configuration dialog pops up, read the intro carefully and then choose the option that applies to you

Install extra packages

If you are running Ubuntu Xenial (or newer) or Debian Stretch (or newer), run this command:

sudo apt install libapache2-mod-fcgid apache2-suexec-custom php-cgi

Otherwise, run this command:

sudo apt install libapache2-mod-fcgid apache2-suexec-custom php5-cgi

Alter the PHP configuration

This is needed to prevent Apache's built-in PHP module from trying to run PHP files.

If you are running Ubuntu Xenial (or newer) or Debian Stretch (or newer), run this command:

sudo rm /etc/apache2/mods-enabled/php7.0.conf

Otherwise, run this command:

sudo rm /etc/apache2/mods-enabled/php5.conf

Remove MySQL server if necessary

Note: Only do this if you'll be using a separate database server.

1. Run sudo apt purge mysql-server mysql-server-5.7 mysql-server-core-5.7

Webmin and Virtualmin Packages

Keys

1. Run mkdir /tmp/virtualmin-keys
2. Run wget -N -P /tmp/virtualmin-keys http://software.virtualmin.com/lib/{RPM-GPG-KEY-webmin,RPM-GPG-KEY-virtualmin,RPM-GPG-KEY-virtualmin-6}
3. Run shasum -a 256 /tmp/virtualmin-keys/{RPM-GPG-KEY-webmin,RPM-GPG-KEY-virtualmin,RPM-GPG-KEY-virtualmin-6}
4. Confirm that you get the output shown below.  (This is necessary because software.virtualmin.com doesn't have a valid HTTPS certificate and thus only works from the non-HTTPS URL.  Therefore you are required to perform extra checking to make sure you'll get the real packages.)

Required output:

36a563bec98a9894065d5f45fbfe58ef51985aecc561569e6288b009ef28f251  /tmp/virtualmin-keys/RPM-GPG-KEY-webmin
08294ec28b36249adf8d51352153235d2467b081d2f3e133771da9100b6fbc81  /tmp/virtualmin-keys/RPM-GPG-KEY-virtualmin
d8bd1baa45a96a837efe1cd535f8a9325aff18751e8571cf3e792c5ea3ffb039  /tmp/virtualmin-keys/RPM-GPG-KEY-virtualmin-6

APT config

1. Run sudoedit /etc/apt/sources.list.d/webmin.list
2. Paste in the contents shown below
1. Note: the lines may appear to contain a single URL, but they don't, so don't remove any spaces
3. Modify the two placeholders as follows:
1. {{OS}} -- debian or ubuntu
2. {{RELEASE}} -- the codename of your Distro release
• You can find this by running lsb_release --short --codename
• For stretch, use jessie instead (the directory for stretch is missing from the server) 

This is the contents of the file mentioned above:

deb http://software.virtualmin.com/gpl/{{OS}}/ virtualmin-{{RELEASE}} main
deb http://software.virtualmin.com/gpl/{{OS}}/ virtualmin-universal main

Installation 

1. Run sudo apt-get update
2. Run sudo apt install webmin-virtual-server virtualmin-base

Configuration

First, configure the apache2-suexec-custom package:

1. Run sudoedit /etc/apache2/suexec/www-data
2. Change the first line to /home
3. Save and quit (press Ctrl-X)

Next, log into Webmin and configure it:

1. Go to https://host.address:10000/
• Replace host.address with the IP Address or full host name of your server
2. Log in with your normal Unix account
3. In the left-hand menu, click on Servers and then Apache Webserver
4. Click on the Global configuration tab in the right-hand section
5. Click on Configure Apache Modules
6. Tick the actions, fcgid and suexec boxes
7. Click [Enable Selected Modules]

You can now click on Virtualmin at the top of the left-hand menu and create "Virtual Servers", which are simply managed Apache vhosts.

Caveats

If you want to host a site named after the primary address of the server, it might not work, because the Apache default vhost will try to serve those requests instead.  (This only happens if the server's hostname is configured to match the primary address.)

To fix this, do this:

1. Run sudoedit /etc/apache2/sites-available/000-default.conf 
2. Uncomment the ServerName directive
• The value should be nonsense, as the point is to set it to something that will never match a real vhost
3. Save and quit (press Ctrl-X)
4. Run sudo service apache2 reload

Resizing a disk on a cloud compute instance (AWS or Google)

Resizing a disk on a cloud compute instance

Diagnosis

If you get an error when installing/upgrading a package that looks like this:

dpkg: error processing archive /var/cache/apt/archives/linux-image-3.13.0-62-generic_3.13.0-62.102_amd64.deb (--unpack):
 cannot copy extracted data for './boot/vmlinuz-3.13.0-62-generic' to '/boot/vmlinuz-3.13.0-62-generic.dpkg-new': failed to write (No space left on device)

...then your root file system is full. This can also be checked like this:

$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/xvda1            129G   129G   0M 100% /

If this has happened, one way to proceed is to add a new file system and move data to it (e.g. so that /home is a separate file system that is mounted into the tree but not using any of the allocated space on the root file system any more).  This method is not covered here.

Another way is to expand the size of the existing root file system.  That is what this article covers.  The phases below apply to any file system that has become full.  (No longer do you need to shut down the instance or unmount the file system, snapshot the volume and create a new, larger one from the snapshot.)

How to fix 

Warning: A mistake or mis-interpretation when following these steps could destroy your data.  Please use caution, and unless you are an intermediate or above Linux user, consider asking a system administrator who is familiar with the tools presented to perform the task.

Phase 1 -- Expand virtual disk

If you are using Amazon Web Services:

1. Log in to the AWS console
2. Make sure you are in the EC2 service section
3. Click on Volumes in the left-hand menu
4. Find the volume that is attached to your instance (sort by the Attachment column)
• Ensure you have found the correct one if that instance has multiple volumes
5. Right-click on the volume and choose Modify
6. Alter the size
7. Click [Save]

 If you are using Google Compute Engine:

1. Log in to the console
2. In the drop-down menu, choose the project that your compute instance belongs to
3. In the Resources box, click on Compute Engine
4. Click on the name of your instance in the list
5. Scroll down to the "Boot disk and local disks" section
6. Click on the relevant disk
7. Click [EDIT] in the toolbar
8. Change the Size value
9. Click [Save]

Other virtualisation systems (like VMware) or storage management systems (like LVM), may allow you to expand a disk volume.  If your host uses one of these systems, research and apply whatever method is available to you, then continue with the next phases.

Phase 2a -- Expand partition

In the steps below, replace /dev/sda with the device name of the root file system, minus the partition number.

1. Log in via SSH 
2. Run lsblk to determine the device of the root file system
• See examples below
• The device name is given in the first column, and in this case is not preceded with "/dev/"
• The relevant disk or partition is the one with "/" in the MOUNTPOINT column
3. If the root file system's device is a disk (not a partition) skip to "Phase 3 -- Expand file system"
• Virtual disks under Linux have device names like "/dev/sda" or "/dev/xvdf"
• Partitions are a way of subdividing disks; they have device names like "/dev/sda1" or "/dev/xvdf2"
• A file system can reside on a partition or use the whole disk; in the latter case there is no partition table and so trying to list/manipulate it won't work
4. Tell Linux to detect the new disk size
1.  N/A -- this is done automatically by the kernel
5. Install parted
1.  Run "sudo apt-get update ; sudo apt-get install parted"
6. Run parted to expand the partition
1. Run "sudo parted /dev/sda"
• Observe the note above about /dev/sda
2. Check if there is more than one partition on the disk:
1. At the parted prompt, Run "print list"
• Take note of the disk size displayed on the second line of output
2. If the root file system's partition (see step 2) is not the last one on the disk, follow the steps at "Phase 2b -- Moving and expanding partitions" and then return here.
3.  At the parted prompt, Run "resize 1 start size"
• If appropriate, replace "1" with the root file system's partition number
• Replace "start" with the partition's existing Start value from the table show by "print list" above
• Replace "size" with the disk size value noted above
4. Follow the instructions provided by parted
5. At the parted prompt, Run "quit"
7. Tell Linux to update its view of the partitions
1.  Run "sudo kpartx -u /dev/sda"
• Observe the note above about /dev/sda

lsblk examples

$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
xvda 202:0    0  15G  0 disk /

 or:

$ lsblkNAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda      8:0    0  10G  0 disk
└─sda1   8:1    0  10G  0 part /

Phase 2b -- Moving and expanding partitions

Warning: Skip to Phase 3 unless Phase 2a directed you to follow these steps.

TBA

Phase 3 -- Expand file system

This is unnecessary if your root file system's device is a partition, as you will have resized both in Phase 2.

Firstly, run "df -hT" and take note of the value for the root file system in the Type column.

If the file system type is ext4, follow these steps:

1. Run "sudo apt-get update ; sudo apt-get install e2fsprogs"
2. Run "sudo resize2fs /dev/sda"
• Observe the note in Phase 2a about /dev/sda
3. Follow the instructions provided

 If the file system type is xfs, follow these steps:

1. Run "sudo apt-get update ; sudo apt-get install xfsprogs"
2. Run "sudo xfs_growfs /dev/sda"
• Observe the note in Phase 2a about /dev/sda
3. Follow the instructions provided

If the file system type is rootfs, look for the other line in the df output that is also mounted on "/" (don't ask me why there's two) and follow the steps above for that file system type.

If  the file system type is none of the above, do a web search for the type and the word "resize".

Phase 4 -- Verification

Run "df -h" and you should see that you now have plenty of free space on your root file system.

Dynamically generating host entries for SSH client config

I will at some point in the future blog about how cool Ansible inventory files are, and how I re-purposed them for Shepherd.
But for now, given that I have all these Ansible inventory files lying around, why not use them to generate my ssh config file, instead of having to update it by hand every time I add a new host to my collection?

Background: the command-line OpenSSH client reads both /etc/ssh/ssh_config and ~/.ssh/config to get the default values for options as well as any host-specific options.  This means that you can define the hosts that you'll be logging into using friendly names.  You can also set global settings, or just settings for specific groups of hosts using wildcards.

I use a Makefile that reads all the inventory files that I have specified and converts them into host entries.  It combines these with the contents of all the files in ~/.ssh/config.d (this is non-standard) and creates ~/.ssh/config .  Note that you can't edit this file manually any more; if you do, either your changes will be overwritten or it will prevent updates to one of the source files from being used (if the target file is newer than the source, which is how make determines if a dependency has changed and therefore the target has to be rebuilt).

Here is a sanitised version of my Makefile:

# User options
CREATED_FILES=xyz.conf mine.conf
LOGLEVEL_TWEAK_REGEXP=xyz.net

.PHONY: all
all: config

# User dependencies
tmp/xyz.conf: ~/Work/XYZ/Hosts/hosts.txt
tmp/mine.conf: ~/Transfer.unison/Geekery/Hosts/hosts.txt


# Actual targets and variables
CREATED_FILE_PATHS = $(CREATED_FILES:%=tmp/%)
config: tmp/000_header.conf $(CREATED_FILE_PATHS) config.d/*.conf
        if [ ! -f $@.bak ] ; then cp -p $@ $@.bak ; fi
        for file in tmp/000_header.conf $(CREATED_FILE_PATHS) ; do cat $$file ; echo ; done > $@
        for file in config.d/*.conf ; do echo "# $(CURDIR)/$$file" ; cat $$file ; echo ; echo ; done >> $@

tmp:
        mkdir $@

$(CREATED_FILE_PATHS):
    echo "# $^" > '$@'
    sed -e '/^#/d' \
        -e 's/[[:space:]]*#.*//' \
        -e '/ansible_ssh_host=/ !d' \
        -e 's/\(cloud_provider\|cloud_region\|cloud_instance_id\)=\([^[:space:]]*\)[[:space:]]*//g' \
        -e 's/^\([^[:space:]]*\)[[:space:]]*alias=\([^[:space:]]*\)[[:space:]]*/Host \1 \2\n/' \
        -e 's/^\(Host [^\n]*\)[[:space:]]*alias=\([^[:space:]]*\)[[:space:]]*/\1 \2\n/' \
        -e 's/^\([^H][^[:space:]]*\)[[:space:]]*/Host \1\n/' \
        -e 's/ansible_ssh_host=\([^[:space:]]*\)[[:space:]]*/  HostName \1\n/' \
        -e 's/ansible_ssh_user=\([^[:space:]]*\)[[:space:]]*/  User \1\n/' \
        -e 's/ansible_ssh_private_key_file=\("[^"]*"\)/  PubkeyAuthentication yes\n  IdentitiesOnly yes\n  IdentityFile \1/' \
        -e 's/ansible_[^[:space:]]*=[^[:space:]]*[[:space:]]*//g' \
        -e 's/^\(Host [^\n]*\($(LOGLEVEL_TWEAK_REGEXP)\)[^\n]*\)/\1\n  LogLevel Error/' \
        -e 's/\n\([[:space:]]*HostName [^\n]*\($(LOGLEVEL_TWEAK_REGEXP)\)[^\n]*\)/\n  LogLevel Error\n\1/' \
        -e 's/$$/\n/' \
      '$^' >> '$@'

tmp/000_header.conf: | tmp
    echo '# see ssh_config(5)' > $@
    echo '#' >> $@
    echo '# DO NOT EDIT -- ANY CHANGES WILL BE OVERWRITTEN BY Makefile' >> $@

Usage 

To "run" the Makefile, use this command:

make -C .ssh

That's it.  You can run it as often as you like, and if no source files have been modified, the target file won't be re-created.

Options

CREATED_FILES is the list of files that are to be created in ~/.ssh/tmp before being combined together (with the other files) to make the target file.  You also have to provide dependency lines (these are a cut-down make rule) that specify which inventory file is used to make each temporary file.

LOGLEVEL_TWEAK_REGEXP allows you specify hosts for which the "LogLevel Error" option will be set, which prevents /etc/issue.net from being blatted onto your terminal every time you run scp or rsync.  The regexp is matched against both the host title (including aliases) and the SSH hostname.

Dependency lines

Note that each file in CREATED_FILES has a matching line (including "tmp/") that specifies the source file.  Think of these as a pair; every time you add or remove a file from CREATED_FILES you have to add or remove the dependency line.

Caveats

At least one file must be in .ssh/config.d -- but it can be empty.  I recommend having a file in this subdirectory called zzz_global.conf so its contents will come after all your host stanzas.  This the place to put your Host * stanza with its options, e.g. "ServerAliveInterval 30" which is great for preventing your ADSL modem from dropping connections.

If you don't have anything to put in LOGLEVEL_TWEAK_REGEXP, comment out the two lines that use it by putting "##" after the first single quote of each sed line.  Yes, sed suppports comments!

How to bootstrap Python VirtualEnvs with a specific Python version

This shows how to create a "bootstrap" VirtualEnv, which will allow the creation of (mostly) self-contained VirtualEnvs that are used for projects.

1. Install OS dependencies like libsqlite3-dev
• Otherwise you won't be able to import sqlite3
• See Django problem: sqlite error when running"manage.py migrate" for what sort of error this results in
2. Build Python from source:
1. Download from https://www.python.org/download/releases
2. Extract tarball
3. cd into directory
4. Run "./configure --prefix=/opt/python3"
5. Run "make"
6. Run "sudo make install"
3. Manually grab the latest virtualenv module to avoid dependency problems with the existing Python installation:
1. Download virtualenv-X.Y.Z.tar.gz from https://pypi.python.org/pypi/virtualenv#downloads
2. Extract into /opt/virtualenv-X.Y.Z
4. Create and configure the "bootstrap" VirtualEnv:
1. E.g. run "/opt/virtualenv-15.0.3/virtualenv.py -p /opt/python3/bin/python3 lib/py3venv"
2. Run "lib/py3venv/bin/pip install virtualenv"

Now, whenever you need to create a self-contained VirtualEnv for a project, run this command, substituting the placeholder for the desired output directory:

lib/py3venv/bin/virtualenv env_dir

The VirtualEnv directory that is created will contain its own Python executable, but will rely on the global packages in /opt/python3.

See The "Virtual Environments and Packages" section of the Python tutorial for more info.

Resume from hibernate failed silently on Debian

After I changed my swap partition, my computer hibernated fine but would not resume, i.e. it booted normally, as if I had done a hard power-off beforehand.

This happened because hibernating saves the contents of RAM etc. into your swap partition.  Therefore, <initramfs>/conf/conf.d/resume now contained an incorrect UUID (of the old, no-longer-existing partition).  See Debian linux mint, resume after hibernation fails for how to fix this.  Don't forget to modify your /etc/fstab and also regenerate the blkid cache by running "sudo blkid -p".

(See Wikipedia.org's Initrd article for background and the initramfs-tools package for how initramfs is managed under Debian.)

You have to shut down after changing your swap partition.  For some reason the kernel's power management subsystem won't hibernate to the new swap partition until after the next boot.  If you try, you'll get the error "Cannot find swap device, try swapon -a" on the console.

PS -- Don't be fooled by "PM: Resume from disk failed." in /var/log/kern.log (this is a normal error that you get when you boot after shutting down).  Debian's initramfs will check for the presence of /dev/disk/by-uuid/$resume and /sys/power/resume and only run <initramfs>/bin/resume if they both exist.  If they don't, it doesn't do anything and just continues to boot.

User defaults on Linux

These are the places where you have to change things like the home directory base:

• /etc/default/useradd (even on non-Debian systems)
  (change these with useradd -D)
  
• /etc/adduser.conf
• Webmin's Users and Groups module configuration

You'd think there'd be one place all this was stored, wouldn't you?

See also /etc/login.defs .