Tech Snippets

Tags: #selfhosting #letsencrypt #nginx #npm #prosody #python

Especially when self-hosting, the wonderful Nginx Proxy Manager (NPM) is often used as a reverse proxy, and for generating and auto-renewing SSL certificates. But how to export the right certificate to another machine that needs it, like the Prosody XMPP server?

This can get rather complicated, since NPM internally stores its configuration and certificates in numbered files instead of using the domain names for filenames.

Well, here is a possible solution.

I run Prosody as an XMPP chat server (not on the Nginx Proxy Manager machine) and need to transfer the SSL certifcates to Prosody. Here is a quick-and-dirty Python implementation of how this could be done.

1. Check all NPM’s numbered .conf files (1.conf, 2.conf, …) for the domain you need the certificate for. The domain name is on the server_name line.
2. Find the location where NPM stores the full chain certificate and the private key. These are on the ssl_certificate and ssl_certificate_key lines, respectively.
3. Open an SSH connection to your target machine (have an SSL key set up; using passwords in clear text is a security risk!) and use sftp to copy the certificate and key files over to your target machine. This will use readable filenames such as chat.mydomain.com.fullchain.pem and chat.mydomain.com.privkey.pem.
4. To avoid file permission problems, run this script once every night, via root’s crontab. Once a day should give enough safety margin, since NPM renews the certificates early enough before their expiry date.

Here is the code I used. It assumes you run NPM in a Docker container, and have your files in user admin’s npm folder. The domain we look for in this case is chat.mydomain.com, and we wish to copy the files to the machine prosody in the local network (resolvable by DNS, otherwise use an IP address), into user myusername’s certs folder. Adapt to your needs.

Note: You might have to install the paramiko and scp Python modules before:

sudo pip3 install paramiko scp

The code is far from perfect, it currently lacks error checking in case the domain or certificate locations aren’t found, but it has been working in daily use for some time now.

#!/usr/bin/python3
import re
import os
from pathlib import Path
from paramiko import SSHClient
from scp import SCPClient
from paramiko import AutoAddPolicy, RSAKey, SSHClient

proxy_hosts_dir = '/home/admin/npm/data/nginx/proxy_host'
archive_dir = '/home/admin/npm/letsencrypt/live'
target_host = 'chat.mydomain.net'
host_file = ''

for filename in os.listdir(proxy_hosts_dir):
    file = os.path.join(proxy_hosts_dir, filename)
    if os.path.isfile(file) and filename.endswith('.conf'):
        with open(file) as f:
            content = f.read();
            m = re.search(r'server_name\s+(.*);', content)
            if m:
              s = m.group(1).split()
              basename = ('_').join(s).replace("*", "x")
              if target_host in s:
                f = re.search(r'ssl_certificate\s+/etc/letsencrypt/live(.*);', content)
                if f:
                  fullchain = archive_dir + f.group(1)
                p = re.search(r'ssl_certificate_key\s+/etc/letsencrypt/live(.*);', content)
                if p:
                  privkey = archive_dir + p.group(1)

client = SSHClient()
client.load_system_host_keys()
client.set_missing_host_key_policy(AutoAddPolicy())
client.connect('prosody', username='myusername')
sftp = client.open_sftp()
sftp.put(fullchain, '/home/myusername/certs/' + target_host + '.fullchain.pem')
sftp.put(privkey, '/home/myusername/certs/' + target_host + '.privkey.pem')
sftp.close()

This script I run via root’s crontab on the NPM machine every night at 3:00 a.m. like so:

# send certificate and key to Prosody XMPP server
0 3 * * * /home/admin/bin/ssl-chat.py

And on the Prosody machine, at 3:10 a.m., also via root’s crontab:

# install the new cert+key sent over from Nginx proxy
10 3 * * * prosodyctl --root cert import chat.mydomain.net /home/myusername/certs/

I use Prosody’s prosodyctl cert update functionality here, since it’s the most reliable way to update certificates with Prosody.

Note: prosodyctl makes backups of both the certificate and key files, so you’ll eventually end up with hundreds of backup files in /etc/prosody/certs. To avoid that, I add an extra delete command in Prosody’s root crontab which will clean out all backups at 3:09 a.m., just before the installation of the new certificate and key:

# set bash as cron shell (below doesn't work with sh)
SHELL=/bin/bash
# remove old certificate and key backups
9 3 * * * rm /etc/prosody/certs/*.{crt,key}.bkp~* > /dev/null

This will leave us with the new certificate and key, plus one backup of each.

Works well so far. Maybe this can help others, too.