Dynamically generating host entries for SSH client config

I will at some point in the future blog about how cool Ansible inventory files are, and how I re-purposed them for Shepherd.
But for now, given that I have all these Ansible inventory files lying around, why not use them to generate my ssh config file, instead of having to update it by hand every time I add a new host to my collection?

Background: the command-line OpenSSH client reads both /etc/ssh/ssh_config and ~/.ssh/config to get the default values for options as well as any host-specific options.  This means that you can define the hosts that you'll be logging into using friendly names.  You can also set global settings, or just settings for specific groups of hosts using wildcards.

I use a Makefile that reads all the inventory files that I have specified and converts them into host entries.  It combines these with the contents of all the files in ~/.ssh/config.d (this is non-standard) and creates ~/.ssh/config .  Note that you can't edit this file manually any more; if you do, either your changes will be overwritten or it will prevent updates to one of the source files from being used (if the target file is newer than the source, which is how make determines if a dependency has changed and therefore the target has to be rebuilt).

Here is a sanitised version of my Makefile:

# User options
CREATED_FILES=xyz.conf mine.conf
LOGLEVEL_TWEAK_REGEXP=xyz.net

.PHONY: all
all: config

# User dependencies
tmp/xyz.conf: ~/Work/XYZ/Hosts/hosts.txt
tmp/mine.conf: ~/Transfer.unison/Geekery/Hosts/hosts.txt


# Actual targets and variables
CREATED_FILE_PATHS = $(CREATED_FILES:%=tmp/%)
config: tmp/000_header.conf $(CREATED_FILE_PATHS) config.d/*.conf
        if [ ! -f $@.bak ] ; then cp -p $@ $@.bak ; fi
        for file in tmp/000_header.conf $(CREATED_FILE_PATHS) ; do cat $$file ; echo ; done > $@
        for file in config.d/*.conf ; do echo "# $(CURDIR)/$$file" ; cat $$file ; echo ; echo ; done >> $@

tmp:
        mkdir $@

$(CREATED_FILE_PATHS):
    echo "# $^" > '$@'
    sed -e '/^#/d' \
        -e 's/[[:space:]]*#.*//' \
        -e '/ansible_ssh_host=/ !d' \
        -e 's/\(cloud_provider\|cloud_region\|cloud_instance_id\)=\([^[:space:]]*\)[[:space:]]*//g' \
        -e 's/^\([^[:space:]]*\)[[:space:]]*alias=\([^[:space:]]*\)[[:space:]]*/Host \1 \2\n/' \
        -e 's/^\(Host [^\n]*\)[[:space:]]*alias=\([^[:space:]]*\)[[:space:]]*/\1 \2\n/' \
        -e 's/^\([^H][^[:space:]]*\)[[:space:]]*/Host \1\n/' \
        -e 's/ansible_ssh_host=\([^[:space:]]*\)[[:space:]]*/  HostName \1\n/' \
        -e 's/ansible_ssh_user=\([^[:space:]]*\)[[:space:]]*/  User \1\n/' \
        -e 's/ansible_ssh_private_key_file=\("[^"]*"\)/  PubkeyAuthentication yes\n  IdentitiesOnly yes\n  IdentityFile \1/' \
        -e 's/ansible_[^[:space:]]*=[^[:space:]]*[[:space:]]*//g' \
        -e 's/^\(Host [^\n]*\($(LOGLEVEL_TWEAK_REGEXP)\)[^\n]*\)/\1\n  LogLevel Error/' \
        -e 's/\n\([[:space:]]*HostName [^\n]*\($(LOGLEVEL_TWEAK_REGEXP)\)[^\n]*\)/\n  LogLevel Error\n\1/' \
        -e 's/$$/\n/' \
      '$^' >> '$@'

tmp/000_header.conf: | tmp
    echo '# see ssh_config(5)' > $@
    echo '#' >> $@
    echo '# DO NOT EDIT -- ANY CHANGES WILL BE OVERWRITTEN BY Makefile' >> $@

Usage 

To "run" the Makefile, use this command:

make -C .ssh

That's it.  You can run it as often as you like, and if no source files have been modified, the target file won't be re-created.

Options

CREATED_FILES is the list of files that are to be created in ~/.ssh/tmp before being combined together (with the other files) to make the target file.  You also have to provide dependency lines (these are a cut-down make rule) that specify which inventory file is used to make each temporary file.

LOGLEVEL_TWEAK_REGEXP allows you specify hosts for which the "LogLevel Error" option will be set, which prevents /etc/issue.net from being blatted onto your terminal every time you run scp or rsync.  The regexp is matched against both the host title (including aliases) and the SSH hostname.

Dependency lines

Note that each file in CREATED_FILES has a matching line (including "tmp/") that specifies the source file.  Think of these as a pair; every time you add or remove a file from CREATED_FILES you have to add or remove the dependency line.

Caveats

At least one file must be in .ssh/config.d -- but it can be empty.  I recommend having a file in this subdirectory called zzz_global.conf so its contents will come after all your host stanzas.  This the place to put your Host * stanza with its options, e.g. "ServerAliveInterval 30" which is great for preventing your ADSL modem from dropping connections.

If you don't have anything to put in LOGLEVEL_TWEAK_REGEXP, comment out the two lines that use it by putting "##" after the first single quote of each sed line.  Yes, sed suppports comments!