host10 domain server down 2/13/2021 – 12:43am [RESOLVED]

One of our servers that hosts customer domains, host10.cruzio.com, is currently offline.  Engineering is working to recover the server.  Incoming email will be queued on secondary servers for delivery once the server is back online.

UPDATE: Recover continues as of 4:30am.  No ETA for restoration at this time.  Thanks for your patience.

UPDATE at 6:00am – Data recovery is taking much longer than we’d hoped.  We anticipate that host10 will be offline until sometime Saturday afternoon.  We will update this status if that changes.

UPDATE: 5:30pm 2/13/21 – We continue with data recovery.  Data stores appear to be fine, but are taking a long time to transfer.  We still hope to have this repaired before the day is over.

UPDATE: 2:00am 2/14/21 – We are still recovering data.  There are a huge number of small files in the mail storage area that are taking much longer than expected.  Hoping that data restoration will complete by mid-morning today.

UPDATE: 9:45am 2/14/21 – System is back up and online.  Engineering will do a debrief to analyze what went wrong and how to make recovery operations go more quickly.  Thanks for your patience.