MlmmjWebArchiver

Summary

mlmmj-WebArchiver is a script for publishing the archives of mailing lists using openbsd.mlmmj. It generates the pages using php to display the mailing list archives of each of the configured mailing lists.

Download and Install

The script can be found here https://git.cryptomilk.org/projects/mlmmj-webarchiver.git/.

The script itself generates the website pages, so DO NOT download and install this in /var/www/htdocs. Download it into a directory you will install it from. ~/src, ~/local ~/projects, or similar locations would work.

You'll want to clone it like this:

git clone https://git.cryptomilk.org/projects/mlmmj-webarchiver.git
cd mlmmj-webarchiver/
ls -la

The output will look something like this:

planetofnix$ git clone https://git.cryptomilk.org/projects/mlmmj-webarchiver.git 
Cloning into 'mlmmj-webarchiver'...
remote: Enumerating objects: 93, done.
remote: Counting objects: 100% (93/93), done.
remote: Compressing objects: 100% (92/92), done.
remote: Total 93 (delta 47), reused 0 (delta 0), pack-reused 0
Receiving objects: 100% (93/93), 32.66 KiB | 388.00 KiB/s, done.
Resolving deltas: 100% (47/47), done.
planetofnix$ cd mlmmj-webarchiver/
planetofnix$ ls -la
total 144
drwxr-xr-x  4 izzyb  izzyb    512 May  3 17:29 .
drwxr-xr-x  8 izzyb  izzyb    512 May  3 17:28 ..
drwxr-xr-x  8 izzyb  izzyb    512 May  3 17:29 .git
-rw-r--r--  1 izzyb  izzyb    364 May  3 17:29 ChangeLog
-rw-r--r--  1 izzyb  izzyb  35147 May  3 17:29 LICENSE
-rw-r--r--  1 izzyb  izzyb   1656 May  3 17:29 README
drwxr-xr-x  4 izzyb  izzyb    512 May  3 17:29 mlmmj-webarchiver
-rwxr-xr-x  1 izzyb  izzyb   8815 May  3 17:29 mlmmj-webarchiver.sh

The README has some basic instructions I found not very useful at first. I'll explain the process clearer here. First thing you want to do is copy the mlmmj-webarchiver folder to /etc/. The mlmmj-webarchiver.sh is a shell script that you setup to run via cron (see below). It should probably go in /usr/local/bin, but I prefer to keep everything together so recommend putting it in /etc/mlmmj-webarchiver.

doas cp -av mlmmj-webarchiver /etc/
doas cp mlmmj-webarchiver.sh /etc/mlmmj-webarchiver/
doas chown -R root.daemon /etc/mlmmj-webarchiver

Prerequisites

These instructions assume you already have a working mlmmj mailing list, mail server and web server. You'll also need to install mhonarc via pkg_add

doas pkg_add mhonarc

Bug Fix

There is a bug in the script that needs to be fixed. Not sure if its a bug, or just a configuration problem with httpd, but this fixes it. Edit /etc/mlmmj-webarchiver/index.php.newlist and change line 19 as follows:

  $archived_months .= "<a href=\"$file\">$file</a><br />\n";

  change to

  $archived_months .= "<a href=\"$file/index.html\">$file</a><br />\n";

You can make other customization to these files as needed.

Configuring

/etc/mlmmj-webachriver/mlmmj-webarchiver.conf

You should edit /etc/mlmmj-webarchiver/mlmmj-webarchiver.conf and make any changes needed. As a minimum you'll want to change WEBARCHIVE_OUT to be inside your web servers chroot environment.

WEBARCHIVE_OUT="/var/www/htdocs/mlmmj-webarchiver/"

/etc/httpd.conf

You'll need to configure /etc/httpd.conf with a section something like this:

# mlmmj-webarchiver Mailing list web archive configuration.
server "archive.ircnow.org" {
        listen on * tls port 443
        tls {
                certificate "/etc/ssl/mail.ircnow.org.fullchain.pem"
                key "/etc/ssl/private/mail.ircnow.org.key"
        }
        location "/.well-known/acme-challenge/*" {
                root "/acme"
                request strip 2
        }
        location "*.php" {
                fastcgi socket "/run/php-fpm.sock"
        }
        root "/htdocs/mlmmj-webarchiver/"
        connection max request body 104857600
        hsts
        directory {
                index "index.php"
        }
}
server "archive.ircnow.org" {
        listen on * port 80

        location "/.well-known/acme-challenge/*" {
                root "/acme"
                request strip 2
        }
        location "*.php" {
                fastcgi socket "/run/php-fpm.sock"
        }

        root "/htdocs/mlmmj-webarchiver/"
        connection max request body 104857600
        hsts
        directory {
                index "index.php"
        }
}
# End mlmmj-webarchiver Configuration

Crontab

You should add the following line to roots crontab

*/10    *       *       *       *       /etc/mlmmj-webarchiver/mlmmj-webarchiver.sh >>/var/log/mlmmj-webarchiver.log 2>&1

This will check for new mail or configuration every 10 minutes and update the site accordingly. Output will be logged in /var/log/mlmmj-webarchiver.log

Adding Lists to the WebArchive.

The README file mentions control files, but doesn't specify that it means in the /var/spool/mlmmj/ directory. You'll find a folder for each mailing list in there, with a folder called control in it. To have the mailing list published, you need to add a file to the control/ directory named webarchive. The file can be empty, which means it will use the default directory configured with WEBARCHIVE_OUT above. A simple touch would work:

doas touch /var/spool/mlmmj/codeforce/control/webarchive

Its probably better to specify a path for clarity even if its redundant by editing it or copying it from another published archive. Or with some doas magic like this:

doas ksh -c 'echo /var/www/htdocs/mlmmj-webachiver/ > /var/spool/mlmmj/codeforce/control/webarchive'

Then you'll know when looking at the file that it's related to the mlmmj-webachriver script. You should probably also create an webarchiveprotected file as described in the README