Tuesday 6th February 2018

All Systems Server Outage

We have now resolved an issue affecting all Transporters systems and websites.

Our primary database server suffered a hardware failure and upon restarting the database was unable to start.

We managed to restore the database to full operations without any data loss, however it required a slow and manual process.

Typically in a situation like this one of our backup databases, which are running 24/7 with live up to date data, would have kicked in. However the issue that affected the primary database also got passed through replication to the backups and corrupted them as well, preventing these from running.

We do run frequent snapshots of all databases so could have restored from a recent back, but this was not required.

We are conducting a full review to determine how we can prevent a similar combination of simultaneous extreme issues causing problems in the future.