wiki.accu.org

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Moving hosting away from Bytemark ======

Bytemark's new owners (for the last few years) IoMart are looking to tackle their steadily declining share prices by stopping all support on Bytemark's BigV hosting system. They offer to replace our VPS will one from their standard fleet, but as they are no longer the technical-lead organisation they were, but a set of financial engineers, I would like, if we are to move the hosting, to put work into moving it to a different organisation. Specifically I have in mind [[https://mythic-beasts.com|Mythic Beasts]] to whom I have recently moved my personal hosting from Bytemark.

===== Current situation =====

A summary of our current hosting position is this. We have a single VPS, ''dennis.accu.org'', with 2Gb RAM and a single vCPU. This runs the following services:

  * Main ACCU website.
  * The old ACCU website, now used only for membership administration until a better solution is found.
  * The ads server.
  * The blog aggregator - a WordPress site.
  * Git repository (Gitea - thing a GitLab type system) and main website build.
  * A small wiki.
  * Conference organisation website and tooling.
  * Email, mailing lists and mailing list admin website.

Disc space - we currently have 44Gb main store and 100Gb archive store - was a perennial problem until a recent expansion of the archive store.

Publication of popular articles in particularly Overload can cause load spikes that seriously degrade website performance. We don't have much headroom.

On the plus side, ioMart have not yet stopped Bytemark's generous provision of the above (except for the recent increase in storage) gratis. They have, though, started withdrawing their support for some more high profile sites they previously supported, so I think we cannot depend on our support continuing indefinitely.

===== Goals =====

For me, a change in hosting should aim to achieve the following goals.

  - Relieve storage pressure.
  - Split main website hosting onto a separate host. At present, any CPU or IO heavy operation on the host (such as making changes to the website and building the website) impacts website performance.
  - Introduce host configuration management and configuration version management.
  - Ensure configuration is managed in a way that allows services to further distributed across separate hosts as requirements change.
  - Add a CI tool to manage builds of the main website.

We also need to update the OS version, and this in turn will mean moving the mailing lists from MailMan2 to MailMan3. This is a considerable change.

===== Trial work =====

I run a personal server setup that is not dissimilar to the ACCU setup, though with considerably less traffic. I have recently completed the process of moving this from Bytemark to Mythic Beasts. Specifics are:

  * Two hosts configured.
  * All certificate generation done on the main host and distributed to other host.
  * Gitea and Jenkins moved.
  * Main website moved.
  * Music part of Morris side website moved. This includes a substantial build process.
  * Move two WordPress websites.
  * Move email hosting.
  * Move two small MailMan mailing lists.

===== New hosts =====

Almost uniquely, I believe, Mythic offer hosting on Raspberry Pis. Pi hosting means you get a dedicated, not virtual, host with network attached storage, at an rather attractive price. I have two hosts, one a Pi3 w/ 1Gb RAM and the other a Pi4 w/ 4Gb RAM, both with 100Gb network attached storage. For these I am forking out the princely sum of 15GBP per month (inc VAT). That's all together, not each.

The one downside of this offering is that only IPv6 connectivity is provided. Mythic observe that the annual cost of a single IPv4 address now exceeds the cost of the entire computer. They do, however, provide website IPv4 proxying, which I find works very well, and also IMAP proxying, plus NAT64 to handle outbound traffic to IPv4 addresses. Inbound email needs to be passed through their servers, which they configure to forward to the Pi.

The Mythic email servers do run all incoming email through ''rspamd'' and reject anything from a host on the SpamHaus ZEN blacklist. So email arrives locally with headers reporting spam status, and I propose to use these just to deliver suspicious emails into Junk if delivering to a local mailbox, or just discard if destined for a mailing list. On my server, I continue to run ''rspamd'' locally, so I can train its filters, but won't initially configure that for ACCU unless spam volumes to ACCU mailboxes make it advisable.

I am also considering changing all current ACCU forwarding aliases (e.g. ''secretary@'', ''chair@'') to proper IMAP mailboxes. This will enable communication continuity when officers changes, but also solve problems with alias forwarding, which has become unreliable in the modern email world, as the forwarded messages do not come from expected servers and can be treated as spam.

I am using SaltStack for configuration management, and (of course) keeping Salt configurations under version control. Configuration for my personal sites can be viewed [[https://git.lunch.org.uk/jim/MythicSalt|here]] if you're curious.

Having successfully complete this personal project, I suggest ACCU considered following a similar-ish course.

I suggest ACCU rents one RPi4 with 8Gb RAM and 200Gb network attached storage. This would host the website build infrastructure (Git/Gitea, and add Jenkins for CI), email/mailing list, certificate handling, membership database. Backup archives would be created on host and ''rsync''d by certificate access to offsite backups - in the first instance my home server, but other volunteers are welcome.

For the foreseeable future we need to keep the old website running for the membership system. This in turn has to use an antediluvian version of PHP, which I think realistically means we need to run it on our own host. We can continue to run it for the foreseeable future on the Bytemark server, and also look at using that to continue to server the main website. Alternatively, I suggest renting a second RPi4 with 4Gb, but only 100Gb network storage. This will host this wiki, the membership system and database, the World of Code Wordpress blog aggregator, and, initially at least, the main website.

We should then look at improving the main website hosting by deploying onto a hosting service & CDN. All I can do here is to hand-wave at the [[https://gohugo.io/hosting-and-deployment/|relevant Hugo documentation]].

Alternatively, if offers of hosting (such as Guy has made in the past) stand, we can deploy copies of the main website among a fleet of servers.

The cost of 2 Mythic RPi4s as above, paid annually, would cost £168 and £111 inc VAT. Permitted bandwidth would be 3Tb and 2Tb per month respectively, which should comfortably accommodate our needs.

====== accu.org domain ======

As part of the moving process we would also need to move hosting the ''accu.org'' domain to Mythic Beasts to enable generation of Let's Encrypt certificates. This would cost £15.50 + VAT for a single year, or £75 + VAT for 5 years.