Downtime this pm

[Paul]

Apologies for the downtime earlier, we're unsure of the exact cause as yet but in short a vital networking connection broke which took everything down.

[Paul]

Ok, so some more info - last night we had an upgrade on our firewall, at that point it looks like a cable wasn't replaced properly and that eventually disconnected this pm. Our hosting company Rackspace (who were responsible for the work done overnight) then took approx 1hr 45 to diagnose and fix the problem (eg go to the firewall, notice the cable not in and push it in), which is obviously pretty unacceptable, but there we have it.

 

Apologies again...

[Jax]

Bit shocking as RackSpace have in my mind been quite reputable in I.T. circles in the past, and such a thing as that is normally something I would expect anyone in the field to check in the first few minutes (first thing I do in a network is normally PING a network device that stops responding, as long as that has not been disabled), anything in the SLA that may help for the future?

[The PIT]

Probably was in the socket but not pushed home. If the upgrade to the firewall may not have been hardware related so the cable may not have been removed during the upgrade so may have been in a dodgy state for some time.

Stll a red faced engineer walking around somewhere.