Thoughts: A reflection on an incident

I was reminded of a very important lesson recently due to a minor outage.

Double check everything.

In the face of an incident. Never trust wholeheartedly what someone is telling you until given adequate proof. I am not saying that people will openly lie. (At least I hope not). It is just that high-stress environments such as incident can lead to sloppiness. We want resolution, and a small part of our brains want desperately to avoid the blame. These two things can mix into shaky truths. Things that can derail the resolution of the problem.

We had a minor outage. I happened to be responsive at the time it was reported and was pulled into the War Room. Things were messy. The first piece of information that I was given was incomplete. The next piece sent us down a rabbit hole. We did not respond in a respectable time. The problem was simple, but well masked behind some confusing decisions and lots and lots of mistakes.

I did not double-check what I was being told. We wasted time.

Our failures are important. Each one should serve some kind of lesson. I missed an opportunity to drive a quick resolution because I fell into the trap of immediacy and didn’t double-check a piece of information I was given.

I will be more diligent in the future.